Most of the non-asymptotic theoretical work in regression is carried out for
the square loss, where estimators can be obtained through closed-form
expressions. In this paper, we use and extend tools from the convex
optimization literature, namely self-concordant functions, to provide simple
extensions of theoretical results for the square loss to the logistic loss. We
apply the extension techniques to logistic regression with regularization by
the $\ell_2$-norm and regularization by the $\ell_1$-norm, showing that new
results for binary classification through logistic regression can be easily
derived from corresponding results for least-squares regression.