Given a random binary sequence $X^{(n)}$ of random variables, $X_{t},$
$t=1,2,...,n$, for instance, one that is generated by a Markov source (teacher)
of order $k^{*}$ (each state represented by $k^{*}$ bits). Assume that the
probability of the event $X_{t}=1$ is constant and denote it by $\beta$.
Consider a learner which is based on a parametric model, for instance a Markov
model of order $k$, who trains on a sequence $x^{(m)}$ which is randomly drawn
by the teacher.

We investigate a population of binary mistake sequences that result from
learning with parametric models of different order. We obtain estimates of
their error, algorithmic complexity and divergence from a purely random
Bernoulli sequence. We study the relationship of these variables to the
learner's information density parameter which is defined as the ratio between
the lengths of the compressed to uncompressed files that contain the learner's
decision rule. The results indicate that good learners have a low information
density$\rho$ while bad learners have a high $\rho$.