The performance of multiple hypothesis testing is known to be affected by the
statistical dependence among random variables involved. The mechanisms
responsible for this, however, are not well understood. We study the effects of
the dependence structure of a finite state hidden Markov model (HMM) on the
likelihood ratios critical for optimal multiple testing on the hidden states.
Various convergence results are obtained for the likelihood ratios as the
observations of the HMM form an increasing long chain.
Two extensions of generalized linear models are considered. In the first one,
response variables depend on multiple linear combinations of covariates. In the
second one, only response variables are observed while the linear covariates
are missing. We derive stochastic Lipschitz continuity results for the loss
functions involved in the regression problems and apply them to get bounds on
estimation error for Lasso. Multivariate comparison results on Rademacher
complexity are obtained as tools to establish the stochastic Lipschitz
continuity results.
For regularized estimation, the upper tail behavior of the random Lipschitz
coefficient associated with empirical loss functions is known to play an
important role in the error bound of Lasso for high dimensional generalized
linear models. The upper tail behavior is known for linear models but much less
so for nonlinear models. We establish exponential type inequalities for the
upper tail of the coefficient and illustrate an application of the results to
Lasso likelihood estimation for high dimensional generalized linear models.
In a recent work (arXiv:0910.2517), for nonlinear models with sparse
underlying linear structures, we studied the error bounds of
$\ell_0$-regularized estimation. In this note, we show that
$\ell_1$-regularized estimation in some important cases can achieve the same
order of error bounds as those in the aforementioned work.
We study the estimation of $\beta$ for the nonlinear model $y =
f(X\sp{\top}\beta) + \epsilon$ when $f$ is a nonlinear transformation that is
known, $\beta$ has sparse nonzero coordinates, and the number of observations
can be much smaller than that of parameters ($n\ll p$). We show that in order
to bound the $L_2$ error of the $L_0$ regularized estimator $\hat\beta$, i.e.,
$\|\hat\beta - \beta\|_2$, it is sufficient to establish two conditions.