Cfrgtkky

I want to use MH to get samples from $p(theta mid y) approx p(y mid theta) p(theta)$. Let's assume $theta$ is heavily constrained and I transform $theta$ to $f(theta)$ so I can sample from an unconstrained space.

The new posterior becomes $p(f(theta) mid y) approx p(y mid f(theta) ) p(f(theta)) ,times, |det(J_{f^{-1}}(y)) |$. Note that I only changed the prior term (Pushforward measure) and left the likelihood term unchanged as it is a probability distribution on $y$, not on $theta$.

(1) My question now is: can I - in the Metropolis Hastings acceptance ratio - just evaluate

$$frac{p(y mid theta^star) }{ p(y mid theta) } ,times, frac{p(f(theta^star)) mid det{ J_{f^{-1}}( theta^star)} mid }{ p(f(theta)) mid det{ J_{f^{-1}}( theta )} mid }$$

? This term makes me nervous, because I transformed theta, evaluate the pdf of the transformed prior, but then transform it back and evaluate the likelihood of the parameter in the original space. However, I cannot evaluate the first term of this equation:

$$frac{p(y mid f(theta^star)) }{ p(y mid f(theta)) } ,times, frac{p(f(theta^star)) leftlvert det{ J_{f^{-1}}( theta^star)} rightrvert }{ p(f(theta)) leftlvert det{ J_{f^{-1}}( theta )} rightrvert }.$$

I could somehow reverse engineer this problem, i.e. define priors on $f(theta)$ and then map $f(theta)$ to $theta$. The Jacobian of the inverse transform then becomes the Jacobian of the transform of my original problem. That way I could evaluate all terms. However, I originally wanted to give some meaning to my priors for $theta$, not for some unconstrained $f(theta)$.

EDIT: Problem solved and clarified - thank you, I should have seen this myself! Please also see
linked stackexchange thread to this post for further
clarification.

You should notice that what you denote $p(y|f(theta))$ is actually the same as $p(y|theta)$ [if you overlook the terrible abuse of notations]. As you mention, changing the parameterisation does not modify the density of the random variable at the observed value $y$ and there is no Jacobian associated with that part.

With proper notations, if
begin{align*}
theta &sim pi(theta)qquadqquad&text{prior}\
y|theta &sim f(y|theta)qquadqquad&text{sampling}\
xi &= h(theta) qquadqquad&text{reparameterisation}\
dfrac{text{d}theta}{text{d}xi}(xi) &= J(xi)qquadqquad&text{Jacobian}\
y|xi &sim g(y|xi)qquadqquad&text{reparameterised density}\
xi^{(t+1)}|xi^{(t)} &sim q(xi^{(t+1)}|xi^{(t)}) qquadqquad&text{proposal}
end{align*}
the Metropolis-Hastings ratio associated with the proposal $xi'sim q(xi'|xi)$ in the $xi$ parameterisation is
$$
underbrace{dfrac{pi(theta(xi'))J(xi')}{pi(theta(xi))J(xi)}}_text{ratio of priors}times
underbrace{dfrac{f(y|theta(xi')}{f(y|theta(xi))}}_text{likelihood ratio}times
underbrace{dfrac{q(xi|xi')}{q(xi'|xi)}}_text{proposal ratio}
$$
which also writes as
$$dfrac{pi(h^{-1}(xi'))J(xi')}{pi(h^{-1}(xi))J(xi)}times
dfrac{g(y|xi')}{g(y|xi)}times
dfrac{q(xi|xi')}{q(xi'|xi)}
$$