We propose a new approach to the theoretical analysis of Loopy Belief
Propagation (LBP) and the Bethe free energy (BFE) by establishing a formula to
connect LBP and BFE with a graph zeta function. The proposed approach is
applicable to a wide class of models including multinomial and Gaussian types.
The connection derives a number of new theoretical results on LBP and BFE. This
paper focuses two of such topics.

A kernel method is proposed for realizing Bayes' rule, based on
representations of probability distributions in reproducing kernel Hilbert
spaces (RKHS). The empirical RKHS embeddings of the conditional probabilities
and prior are expressed as feature mappings of samples, and an RKHS embedding
of the posterior distribution is computed, again based on a feature mapping of
a sample. This kernel Bayes' rule can be applied to a wide variety of
nonparametric Bayesian inference problems. As an example, the approach is used
in filtering with a nonparametric state-space model.

We propose a new approach to the analysis of Loopy Belief Propagation (LBP)
by establishing a formula that connects the Hessian of the Bethe free energy
with the edge zeta function. The formula has a number of theoretical
implications on LBP. It is applied to give a sufficient condition that the
Hessian of the Bethe free energy is positive definite, which shows
non-convexity for graphs with multiple cycles. The formula clarifies the
relation between the local stability of a fixed point of LBP and local minima
of the Bethe free energy.

We introduce two graph polynomials and discuss their properties. The one is a
polynomial of two variables, motivated by performance analysis of the Bethe
approximation of the Ising partition function. The other polynomial of one
variable is obtained by its specialization. It is shown that these polynomials
satisfy deletion-contraction relations and are essentially new examples of
V-function, which is introduced by Tutte (1947, Proc. Cambridge Philos. Soc.
43, 26-40).