Robust and reliable covariance estimation plays a decisive role in financial
applications. An important class of estimators is based on Factor models. Here,
we show by extensive Monte Carlo simulations that covariance matrices derived
from the statistical Factor Analysis model exhibit a systematic error, which is
similar to the well-known systematic error of the spectrum of the sample
covariance matrix. Moreover, we introduce the Directional Variance Adjustment
(DVA) algorithm, which diminishes the systematic error.