Question

While doing Bayesian inference, consider estimating the posterior distribution of the model parameter (m), given data (d). Assume that Prior and Likelihood are proportional to Gaussian functions given by \[ {Prior} \propto \exp(-0.5(m - 1)^2) \] \[ {Likelihood} \propto \exp(-0.5(m - 3)^2) \]

The mean of the posterior distribution is (Answer in integer)

Hint:

When the prior and likelihood are Gaussian, the posterior is also Gaussian. Its mean lies between the prior and likelihood means, weighted by their precisions (inverse variances).

Answer 1

Step 1: Apply Bayes' theorem.
The posterior distribution \( P(m|d) \) is proportional to the product of the prior \( P(m) \) and the likelihood \( P(d|m) \): \[ P(m|d) \propto P(m) \times P(d|m) \propto \exp(-0.5(m - 1)^2) \times \exp(-0.5(m - 3)^2) \] \[ P(m|d) \propto \exp(-0.5 [(m - 1)^2 + (m - 3)^2]) \] Step 2: Expand the exponent. \[ (m - 1)^2 + (m - 3)^2 = m^2 - 2m + 1 + m^2 - 6m + 9 = 2m^2 - 8m + 10 \] \[ P(m|d) \propto \exp(-0.5 [2m^2 - 8m + 10]) = \exp(-(m^2 - 4m + 5)) \] Step 3: Complete the square in the exponent. \[ m^2 - 4m + 5 = (m^2 - 4m + 4) + 1 = (m - 2)^2 + 1 \] \[ P(m|d) \propto \exp(-[(m - 2)^2 + 1]) \propto \exp(-(m - 2)^2) \] Step 4: Identify the mean of the posterior distribution.
The posterior distribution is proportional to a Gaussian with mean \( \mu_{post} = 2 \). Alternatively, using the formula for the mean of the posterior for Gaussian prior and likelihood:
Prior mean \( \mu_p = 1 \), prior variance \( \sigma_p^2 = 1 \). Likelihood mean \( \mu_l = 3 \), likelihood variance \( \sigma_l^2 = 1 \). Posterior mean \( \mu_{post} = \frac{\frac{1}{\sigma_p^2} \mu_p + \frac{1}{\sigma_l^2} \mu_l}{\frac{1}{\sigma_p^2} + \frac{1}{\sigma_l^2}} = \frac{\frac{1}{1} \times 1 + \frac{1}{1} \times 3}{\frac{1}{1} + \frac{1}{1}} = \frac{1 + 3}{2} = \frac{4}{2} = 2 \) The mean of the posterior distribution is 2.