We derive rates of contraction of posterior distributions on nonparametric
models resulting from sieve priors. The aim of the paper is to provide general
conditions to get posterior rates when the parameter space has a general
structure, and rate adaptation when the parameter space is, e.g., a Sobolev
class. The conditions employed, although standard in the literature, are
combined in a novel way. The results are applied to density, regression,
nonlinear autoregression and Gaussian white noise models.
Bayesian inference is attractive for its coherence and good frequentist
properties. However, it is a common experience that eliciting a honest prior
may be difficult and, in practice, people often take an {\em empirical Bayes}
approach, plugging empirical estimates of the prior hyperparameters into the
posterior distribution. Even if not rigorously justified, the underlying idea
is that, when the sample size is large, empirical Bayes leads to "similar"
inferential answers. Yet, precise mathematical results seem to be missing.
For a Gaussian time series with long-memory behavior, we use the FEXP-model
for semi-parametric estimation of the long-memory parameter $d$. The true
spectral density $f_o$ is assumed to have long-memory parameter $d_o$ and a
FEXP-expansion of Sobolev-regularity $\be > 1$. We prove that when $k$ follows
a Poisson or geometric prior, or a sieve prior increasing at rate
$n^{\frac{1}{1+2\be}}$, $d$ converges to $d_o$ at a suboptimal rate. When the
sieve prior increases at rate $n^{\frac{1}{2\be}}$ however, the minimax rate is
almost obtained.
For many decades, statisticians have made attempts to prepare the Bayesian
omelette without breaking the Bayesian eggs; that is, to obtain probabilistic
likelihood-based inferences without relying on informative prior distributions.
A recent example is Murray Aitkin's recent book, {\em Statistical Inference},
which presents an approach to statistical hypothesis testing based on
comparisons of posterior distributions of likelihoods under competing models.
Aitkin develops and illustrates his method using some simple examples of
inference from iid data and two-way tests of independence.
A stationary Gaussian process is said to be long-range dependent (resp.
anti-persistent) if its spectral density $f(\lambda)$ can be written as
$f(\lambda)=|\lambda|^{-2d}g(|\lambda|)$, where $0< d < 1/2 (resp. -1/2 < d <
0), and g is continuous. We propose a novel Bayesian nonparametric approach for
the estimation of the spectral density of such processes. Within this approach,
we prove posterior consistency for both d and g, under appropriate conditions
on the prior distribution.
This chapter provides a overview of Bayesian inference, mostly emphasising
that it is a universal method for summarising uncertainty and making estimates
and predictions using probability statements conditional on observed data and
an assumed model (Gelman 2008).
This introduction to Bayesian statistics presents the main concepts as well
as the principal reasons advocated in favour of a Bayesian modelling. We cover
the various approaches to prior determination as well as the basis asymptotic
arguments in favour of using Bayes estimators. The testing aspects of Bayesian
inference are also examined in details.
Published exactly seventy years ago, Jeffreys's Theory of Probability (1939)
has had a unique impact on the Bayesian community and is now considered to be
one of the main classics in Bayesian Statistics as well as the initiator of the
objective Bayes school. In particular, its advances on the derivation of
noninformative priors as well as on the scaling of Bayes factors have had a
lasting impact on the field. However, the book reflects the characteristics of
the time, especially in terms of mathematical rigor.
In this paper, we investigate the asymptotic properties of nonparametric
Bayesian mixtures of Betas for estimating a smooth density on $[0,1]$. We
consider a parametrization of Beta distributions in terms of mean and scale
parameters and construct a mixture of these Betas in the mean parameter, while
putting a prior on this scaling parameter. We prove that such Bayesian
nonparametric models have good frequentist asymptotic properties.
We are grateful to all discussants (Bernardo, Gelman, Kass, Lindley, Senn,
and Zellner) of our re-visitation for their strong support in our enterprise
and for their overall agreement with our perspective. Further discussions with
them and other leading statisticians showed that the legacy of Theory of
Probability is alive and lasting.
We are grateful to all discussants (Bernardo, Gelman, Kass, Lindley, Senn,
and Zellner) of our re-visitation for their strong support in our enterprise
and for their overall agreement with our perspective. Further discussions with
them and other leading statisticians showed that the legacy of Theory of
Probability is alive and lasting.
In this paper, we study the asymptotic posterior distribution of linear
functionals of the density. In particular, we give general conditions to obtain
a semiparametric version of the Bernstein-Von Mises theorem. We then apply this
general result to nonparametric priors based on infinite dimensional
exponential families. As a byproduct, we also derive adaptive nonparametric
rates of concentration of the posterior distributions under these families of
priors on the class of Sobolev and Besov spaces.