Grouping structures arise naturally in many statistical modeling problems.
Several methods have been proposed for variable selection that respect grouping
structure in variables. Examples include the group LASSO and several concave
group selection methods. In this article, we give a selective review of group
selection concerning methodological developments, theoretical properties, and
computational algorithms. We pay particular attention to group selection
methods involving concave penalties. We address both group selection and
bi-level selection methods.
We propose a new penalized method for variable selection and estimation that
explicitly incorporates the correlation patterns among predictors. This method
is based on a combination of the minimax concave penalty and Laplacian
quadratic associated with a graph as the penalty function. We call it the
sparse Laplacian shrinkage (SLS) method. The SLS uses the minimax concave
penalty for encouraging sparsity and Laplacian quadratic penalty for promoting
smoothness among coefficients associated with the correlated predictors.
A number of variable selection methods have been proposed involving nonconvex
penalty functions. These methods, which include the smoothly clipped absolute
deviation (SCAD) penalty and the minimax concave penalty (MCP), have been
demonstrated to have attractive theoretical properties, but model fitting is
not a straightforward task, and the resulting solutions may be unstable.
In regression problems where covariates can be naturally grouped, the group
Lasso is an attractive method for variable selection since it respects the
grouping structure in the data. We study the selection and estimation
properties of the group Lasso in high-dimensional settings when the number of
groups exceeds the sample size. We provide sufficient conditions under which
the group Lasso selects a model whose dimension is comparable with the
underlying model with high probability and is estimation consistent.
We consider a nonparametric additive model of a conditional mean function in
which the number of variables and additive components may be larger than the
sample size but the number of nonzero additive components is "small" relative
to the sample size. The statistical problem is to determine which additive
components are nonzero. The additive components are approximated by truncated
series expansions with B-spline bases. With this approximation, the problem of
component selection becomes that of selecting the groups of coefficients in the
expansion.