Mean field variational inference
In this problem, you will investigate mean field approximate inference algorithms (Koller & Friedman1 11.5). Consider the Markov network in the above figure. Define edge potentials …
•
2 min read
Markov chain Monte Carlo sampling
Inverse CDF sampling A simple sampling method adopted by many of the standard math libraries is the inverse probability transform: draw $u \sim \text{Unif}(0, 1)$, then draw $x\sim …
•
3 min read
Approximate inference via Gibbs sampling
Consider a setting in which there are $D$ diseases and a patient either has ($d_i=1$) or does not have ($d_i=0$) each disease. The hospital can measure $S$ symptoms, where $s_j=1$ …
•
4 min read
Parameter learning in probabilistic graphical models
Parameter learning in Bayesian networks and Markov random fields Cost of learning CRF parameters Consider the process of gradient-ascent training for a conditional random field …
•
4 min read
Learning maximum likelihood tree structure with the Chow-Liu algorithm
Write a function ChowLiu(X) -> A where X is a D by N data matrix containing a multivariate data point on each column that returns a Chow-Liu maximum likelihood tree for X. The tree …
•
1 min read
Expectation-maximization for a Markov chain mixture model
Assume that a sequence $v_1,\ldots,v_T \in \{1,\dots,V\}$ is generated by a Markov chain. For a single chain of length $T$, we have $$ p(v_1,\dots,v_T) = p(v_1)\prod_{t=1}^{T-1} …
•
8 min read
Learning edge direction in a Bayesian network model
Our interest here is to discuss a method to learn the direction of an edge in a belief network. Consider a distribution $$ P(x,y | \theta,M_{y\to x}) = …
•
3 min read