We investigate fast methods that allow to quickly eliminate variables
(features) in supervised learning problems involving a convex loss function and
a $l_1$-norm penalty, leading to a potentially substantial reduction in the
number of variables prior to running the supervised learning algorithm. The
methods are not heuristic: they only eliminate features that are {\em
guaranteed} to be absent after solving the learning problem. Our framework
applies to a large class of problems, including support vector machine
classification, logistic regression and least-squares.