We study the problem of learning high dimensional regression models
regularized by a structured-sparsity-inducing penalty that encodes prior
structural information on either input or output sides. We consider two widely
adopted types of such penalties as our motivating examples: 1) overlapping
group lasso penalty, based on the l1/l2 mixed-norm penalty, and 2) graph-guided
fusion penalty.
We consider the optimization problem of learning regression models with a
mixed-norm penalty that is defined over overlapping groups to achieve
structured sparsity. It has been previously shown that such penalty can encode
prior knowledge on the input or output structure to learn an
structured-sparsity pattern in the regression parameters. However, because of
the non-separability of the parameters of the overlapping groups, developing an
efficient optimization method has remained a challenge.
We consider the problem of learning a structured multi-task regression, where
the output consists of multiple responses that are related by a graph and the
correlated response variables are dependent on the common inputs in a sparse
but synergistic manner. Previous methods such as l1/l2-regularized multi-task
regression assume that all of the output variables are equally related to the
inputs, although in many real-world problems, outputs are related in a complex
manner.
We consider the problem of learning a sparse multi-task regression with an
application to a genetic association mapping problem for discovering genetic
markers that influence expression levels of multiple genes jointly. In
particular, we consider the case where the structure over the outputs can be
represented as a tree with leaf nodes as outputs and internal nodes as clusters
of the outputs at multiple granularity, and aim to recover the common set of
relevant inputs for each output cluster.