Divergence estimators based on direct approximation of density-ratios without
going through separate approximation of numerator and denominator densities
have been successfully applied to machine learning tasks that involve
distribution comparison such as outlier detection, transfer learning, and
two-sample homogeneity test. However, since density-ratio functions often
possess high fluctuation, divergence estimation is still a challenging task in
practice. In this paper, we propose to use relative divergences for
distribution comparison, which involves approximation of relative
density-ratios. Since relative density-ratios are always smoother than
corresponding ordinary density-ratios, our proposed method is favorable in
terms of the non-parametric convergence speed. Furthermore, we show that the
proposed divergence estimator has asymptotic variance independent of the model
complexity under a parametric setup, implying that the proposed estimator
hardly overfits even with complex models. Through experiments, we demonstrate
the usefulness of the proposed approach.