We study notions of robustness of Markov kernels and probability distribution
of a system that is described by $n$ input random variables and one output
random variable. Markov kernels can be expanded in a series of potentials that
allow to describe the system's behaviour after knockouts. Robustness imposes
structural constraints on these potentials. Robustness of probability
distributions is defined via conditional independence statements. These
statements can be studied algebraically. The corresponding conditional
independence ideals are related to binary edge ideals.
The closure of a discrete exponential family is described by a finite set of
equations corresponding to the circuits of an underlying oriented matroid.
These equations are similar to the equations used in algebraic statistics,
although they need not be polynomial in the general case. This description
allows for a combinatorial study of the possible support sets in the closure of
an exponential family. If two exponential families induce the same oriented
matroid, then their closures have the same support sets.
We improve recently published results about resources of Restricted Boltzmann
Machines (RBM) and Deep Belief Networks (DBN) required to make them Universal
Approximators. We show that any distribution p on the set of binary vectors of
length n can be arbitrarily well approximated by an RBM with k-1 hidden units,
where k is the minimal number of pairs of binary vectors differing in only one
entry such that their union contains the support set of p. In important cases
this number is half of the cardinality of the support set of p.
The concept of effective complexity of an object as the minimal description
length of its regularities has been initiated by Gell-Mann and Lloyd. Based on
their work we gave a precise definition of effective complexity of finite
binary strings in terms of algorithmic information theory in our previous
paper. Here we study the effective complexity of strings generated by
stationary processes. Sufficiently long typical process realizations turn out
to be effectively simple under any linear scaling with the string's length of
the parameter $\Delta$ which determines the minimization domain.
This work presents a novel learning method in the context of embodied
artificial intelligence and guided self-organisation, which is free of
assumptions about the world and restrictions on the underlying model. The
learning rule is derived from the principle of maximising the predictive
information in the sensori-motor loop. It is evaluated in six experiments in
which individually controlled robots with different control paradigms are
physically connected to chains of varying length. The robots have no form of
direct communication.
This work presents a novel learning method in the context of embodied
artificial intelligence and guided self-organisation, which is free of
assumptions about the world and restrictions on the underlying model. The
learning rule is derived from the principle of maximising the predictive
information in the sensori-motor loop. It is evaluated in six experiments in
which individually controlled robots with different control paradigms are
physically connected to chains of varying length. The robots have no form of
direct communication.