Many models of interest in the natural and social sciences have no
closed-form likelihood function, which means that they cannot be treated using
the usual techniques of statistical inference. In the case where such models
can be efficiently simulated, Bayesian inference is still possible thanks to
the Approximate Bayesian Computation (ABC) algorithm. Although many refinements
have since been suggested, the technique suffers from three major shortcomings.
First, it requires introducing a vector of "summary statistics", the choice of
which is arbitrary and may lead to strong biases.

We consider the generic problem of performing sequential Bayesian inference
in a state-space model with observation process $(y_{t})$, state process
$(x_{t})$ and fixed parameter $\theta$. An idealized approach would be to apply
the \emph{iterated batch importance sampling} (IBIS) algorithm of
\citet{Chopin:IBIS}. This is a sequential Monte Carlo algorithm \emph{in the

A Monte Carlo algorithm is said to be adaptive if it can adjust automatically
its current proposal distribution, using past simulations. The choice of the
parametric family that defines the set of proposal distributions is critical
for a good performance. We treat the problem of constructing such parametric
families for adaptive sampling on multivariate binary spaces.

We introduce a new class of Sequential Monte Carlo (SMC) methods, which we
call free energy SMC. This class is inspired by free energy methods, which
originate from Physics, and where one samples from a biased distribution such
that a given function $\xi(\theta)$ of the state $\theta$ is forced to be
uniformly distributed over a given interval.

Because of their multimodality, mixture posterior densities are difficult to
sample with standard Markov chain Monte Carlo (MCMC) methods. We propose a
strategy to enhance the sampling of MCMC in this context, using a biasing
procedure which originates from computational statistical physics. The
principle is first to choose a "reaction coordinate", that is, a direction in
which the target density is multimodal. In a second step, the marginal
log-density of the reaction coordinate is estimated; this quantity is called
"free energy" in the computational statistical physics literature.

This is the compilation of our comments submitted to the Journal of the Royal
Statistical Society, Series B, to be published within the discussion of the
Read Paper of Andrieu, Doucet and Hollenstein.

We are grateful to all discussants (Bernardo, Gelman, Kass, Lindley, Senn,
and Zellner) of our re-visitation for their strong support in our enterprise
and for their overall agreement with our perspective. Further discussions with
them and other leading statisticians showed that the legacy of Theory of
Probability is alive and lasting.