Cfrgtkky

Suppose we have a point mass prior,

$$theta sim begin{cases} I(theta=1) ,& prob=frac{1}{2} \
Gamma(c,c), & prob=frac{1}{2} end{cases}$$

Then if we are asked

$lim_{c to infty} P(theta=1)$

Now here is the issue, since Gamma is a continuous distribution, it seems that in the case of gamma,we will never have $theta=1$.

To me it thus seems that regardless of the value of c, the $p(theta=1)=frac{1}{2}$

However, we also have that since expected value of a $gamma(a,b)=frac{a}{b}$ so that the expected value of the gamma is 1 when we have $a=b=c$

But, by Markov, for $X sim Gamma(c,c)$

$lim_{c to infty} Pr(|X-mu| lt epsilon) to 1$ for any $epsilon gt 0$

So is $lim_{c to infty}P(theta=1) =1$ , or is $lim_{c to infty}P(theta=1)=frac{1}{2}$

As even though the markov inequality holds, it is a continous distirbution, so we will never have it exactly equal to 1.

Thanks all

Your specified prior distribution is a mixture of a continuous and discrete part. Let $I sim text{Bern}(1/2)$ be the indicator that the parameter is taken from the gamma distribution. Then using the law of total probability you have:

$$begin{equation} begin{aligned}
mathbb{P}(theta = 1|c)
&= mathbb{P}(theta = 1|c, I=0) cdot mathbb{P}(I=0) + mathbb{P}(theta = 1|c, I=1) cdot mathbb{P}(I=1) \[6pt]
&= frac{1}{2} cdot mathbb{P}(theta = 1|c, I=0) + frac{1}{2} cdot mathbb{P}(theta = 1|c, I=1) \[6pt]
&= frac{1}{2} + frac{1}{2} cdot mathbb{P}(theta = 1| theta sim text{Ga}(c,c)) \[6pt]
&= frac{1}{2}. \[6pt]
end{aligned} end{equation}$$

(Note that the last step comes from recognising that the gamma is a continuous distribution, so the probability of a specific point is zero under this distribution.) This result holds for all values of $c$, so you are correct that it also holds in the limit:

$$lim_{c rightarrow infty} mathbb{P}(theta = 1|c) = lim_{c rightarrow infty} frac{1}{2} = frac{1}{2}.$$

Your later use of Chebychev's inequality shows that as $c rightarrow infty$ you get $theta rightarrow 1$ (convergence in probability), but this does not change the fact that $mathbb{P}(theta = 1|c) = 1/2$ for all $c > 0$. (To understand the reason for this more fully, have a read about the distinction between convergence in probability and almost-sure convergence.)

Disclaimer The above answer is completely fine if the question
is about the prior probability that $θ=1$. While there is no
indication whatsoever in the text of the question to the alternative, namely
that the question is about the posterior probability that $θ=1$, a recent
exchange with students about a very similar question in my book, The
Bayesian Choice (Exercise 5.9), leads me to propose the extension
to the posterior setting.

In this posterior case, when $Xsim mathcal{E}(theta)$ [for instance] the probability that $θ=1$ is
begin{align*}
mathbb{P}(theta=1|x)&= frac{e^{-x}}{e^{-x}+int_0^infty theta e^{-theta x} pi_c(theta)text{d}theta}\
&=frac{e^{-x}}{e^{-x}+ int_0^infty theta e^{-theta x} c^c theta^{c-1}e^{-c theta} Gamma(c)^{-1}text{d}theta}\
&=frac{e^{-x}}{e^{-x}+c^c (c+x)^{-c-1} Gamma(c+1)Gamma(c)^{-1}}\
&=frac{e^{-x}}{e^{-x}+c^{c+1} (c+x)^{-c-1}}\
&=frac{e^{-x}}{e^{-x}+ (1+x/c)^{-c-1}}\
end{align*}
which converges to
$$frac{e^{-x}}{e^{-x}+e^{-x}}=frac{1}{2}$$
when $c$ goes to infinity.