In the Bayesian community, an ongoing imperative is to develop efficient
algorithms. An appealing approach is to form a hybrid algorithm by combining
ideas from competing existing techniques. This paper addresses issues in
designing hybrid methods by considering selected case studies: the delayed
rejection algorithm, the pinball sampler, the Metropolis adjusted Langevin
algorithm, and the population Monte Carlo algorithm. We observe that even if
each component of a hybrid algorithm has individual strengths, they may not
contribute equally or even positively when they are combined.
The development of statistical methods and numerical algorithms for model
choice is vital to many real-world applications. In practice, the ABC approach
can be instrumental for sequential model design; however, the theoretical basis
of its use has been questioned. We present a measure-theoretic framework for
using the ABC error towards model choice and describe how easily existing
rejection, Metropolis-Hastings and sequential importance sampling ABC
algorithms are extended for the purpose of model checking.
Approximate Bayesian computation (ABC), also known as likelihood-free
methods, have become a favourite tool for the analysis of complex stochastic
models, primarily in population genetics but also in financial analyses. We
advocated in Grelaud et al.
The Savage-Dickey ratio is known as a specialised representation of the Bayes
factor (O'Hagan and Forster, 2004) that allows for a functional plugging
approximation of this quantity. We demonstrate here that it is a generic
approximation method instead of an identity imposing constraints on the prior
distributions, while incidentally clarifying the measure-theoretic bases of the
method. We provide furthermore a general framework to produce a converging
approximation of the Bayes factor that is unrelated with the earlier approach
of Verdinelli and Wasserman (1995).
In this note we attempt to trace the history and development of Markov chain
Monte Carlo (MCMC) from its early inception in the late 1940's through its use
today. We see how the earlier stages of the Monte Carlo (MC, not MCMC) research
have led to the algorithms currently in use. More importantly, we see how the
development of this methodology has not only changed our solutions to problems,
but has changed the way we think about problems.