This paper introduces and analyzes a stochastic search method for parameter
estimation in linear regression models in the spirit of Beran and Millar
(1987). The idea is to generate a random finite subset of a parameter space
which will automatically contain points which are very close to an unknown true
parameter. The motivation for this procedure comes from recent work of
Duembgen, Samworth and Schuhmacher (2011) on regression models with log-concave
error distributions.
We study the approximation of arbitrary distributions P on d-dimensional
space by distributions with log-concave density. Approximation means minimizing
a Kullback-Leibler type functional. We show that such an approximation exists
if, and only if, P has finite first moments and is not concentrated on some
hyperplane. Furthermore we show that this approximation depends continuously on
P with respect to Mallows' distance D_1. This result implies consistency of the
maximum likelihood estimator of a log-concave density under fairly general
conditions.
We present theoretical properties of the log-concave maximum likelihood
estimator of a density based on an independent and identically distributed
sample in $\mathbb{R}^d$. Our study covers both the case where the true
underlying density is log-concave, and where this model is misspecified. We
begin by showing that for a sequence of log-concave densities, convergence in
distribution implies much stronger types of convergence -- in particular, it
implies convergence in Hellinger distance and even in certain exponentially
weighted total variation norms.