[SOLVED] PML Homework 04

A bivariate Gibbs sampler for a vector 𝑥 = (𝑥₁, 𝑥₂) draws iteratively from the conditional distributions in the following way:

choose a starting value 𝑝(𝑥₁|𝑥⁽₂⁰⁾) for each iteration 𝑖:

draw 𝑥(21(𝑖𝑖)) from  from 𝑝𝑝((𝑥𝑥21||𝑥𝑥(1(2𝑖𝑖−))1)) draw 𝑥

As in other MCMC algorithms, samples from the first few iterations are usually discarded. This is known as the “warmup” phase. Suppose that the conditional distributions are

𝑥₁|𝑥₂ ∼ 

2      ,

𝑥₂|𝑥₁ ∼ 

where 𝜌, 𝜇₁, 𝜇₂, 𝜎₁, 𝜎₂ are real valued parameters and |𝜌| < 1.

1. (code) Implement a Gibbs sampler which takes as inputs the number of warmup draws warmup , the number of iterations iters and the parameters rho, mu1, mu2, sigma1, sigma2 .
2. (code) Set 𝜌 = 0.3, 𝜇1 = 0.5, 𝜇2 = 0.1, 𝜎1 = 1, 𝜎2 = 1.5 and plot the distributions of 𝑥1 and 𝑥2.
3. (theory) What is the joint distribution of 𝑥 = (𝑥₁, 𝑥₂)?

Exercise 2

(code)

Let 𝜃₁ and 𝜃₂ be real valued parameters of the function

𝑓(𝑥, 𝜃1, 𝜃2) = 𝑒𝜃1𝑥𝑥+ 𝜃2 .

We observe some noisy realizations (𝑥_𝑜𝑏𝑠, 𝑦_𝑜𝑏𝑠) of this function and want to infer the posterior distribution of 𝜃₁ and 𝜃₂.

1. Generate 30 observations (𝑥𝑜𝑏𝑠, 𝑦𝑜𝑏𝑠) as follows:

𝜃₁ = 0.5

𝜃₂ = 3

𝜌 ∼ (0, 0.3²)

𝑥_𝑜𝑏𝑠 ∼ (0, 10)

𝑦_𝑜𝑏𝑠 = 𝑓(𝑥_𝑜𝑏𝑠, 𝜃1, 𝜃2) + 𝜌

and plot them using seaborn sns.scatterplot function.

2. Given the observations from (1.) and the generative model

𝜃₁ ∼ Exp(2.)

𝜃₂ ∼ Exp(0.2)

𝛾 ∼ (0, 0.5)

𝑦̂ = 𝑓(𝑥, 𝜃₁, 𝜃₂)

𝑦 ∼ (𝑦̂ , 𝛾)

use pyro NUTS to infer the posterior distribution of 𝜃₁, 𝜃₂, 𝛾. Set warmup_steps=2000 and num_samples=100 .

3. Look at convergence checks on the traces and discuss the quality of your estimates. Remeber that you can extract posterior samples from an MCMC object using get_samples() method (notebook 04).
4. Use sns.scatterplot ^{to plot the observed couples} (𝑥_𝑜𝑏𝑠, 𝑦_𝑜𝑏𝑠) and sns.lineplot to plot the learned function 𝑓 with the posterior estimates 𝜃₁, 𝜃₂.