Ising models with hidden Markov structure: applications to probabilistic inference in machine learning

Abstract In this paper, we investigate tree-indexed Markov chains (Gibbs measures) defined by a Hamiltonian that couples two Ising layers: hidden spins <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mrow> <mml:mi>s</mml:mi> <mml:mo stretchy="false">(</mml:mo> <mml:mi>x</mml:mi> <mml:mo stretchy="false">)</mml:mo> <mml:mo>∈</mml:mo> <mml:mo fence="false" stretchy="false">{</mml:mo> <mml:mo>±</mml:mo> <mml:mn>1</mml:mn> <mml:mo fence="false" stretchy="false">}</mml:mo> </mml:mrow> </mml:math> and observed spins <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mrow> <mml:mi>σ</mml:mi> <mml:mo stretchy="false">(</mml:mo> <mml:mi>x</mml:mi> <mml:mo stretchy="false">)</mml:mo> <mml:mo>∈</mml:mo> <mml:mo fence="false" stretchy="false">{</mml:mo> <mml:mo>±</mml:mo> <mml:mn>1</mml:mn> <mml:mo fence="false" stretchy="false">}</mml:mo> </mml:mrow> </mml:math> on a Cayley tree. The Hamiltonian incorporates Ising interactions within each layer and site-wise emission couplings between layers, extending hidden Markov models to a bilayer Markov random field. Specifically, we explore translation-invariant Gibbs measures (TIGMs) of this Hamiltonian on Cayley trees. Under certain explicit conditions on the model’s parameters, we demonstrate that there can be up to three distinct TIGMs. Each of these measures represents an equilibrium state of the spin system. These measures provide a structured approach to inference on hierarchical data in machine learning. They have practical applications in tasks such as denoising, weakly supervised learning, and anomaly detection. The Cayley tree structure is particularly advantageous for exact inference due to its tractability.