Gaussian and quadratic approximations of message passing algorithms on graphs
have attracted considerable recent attention due to their computational
simplicity, analytic tractability, and wide applicability in optimization and
statistical inference problems. This paper presents a systematic framework for
incorporating such approximate message passing (AMP) methods in general
graphical models.
This paper considers the problem of detecting the support (sparsity pattern)
of a sparse vector from random noisy measurements. Conditional power of a
component of the sparse vector is defined as the energy conditioned on the
component being nonzero. Analysis of a simplified version of orthogonal
matching pursuit (OMP) called sequential OMP (SequOMP) demonstrates the
importance of knowledge of the rankings of conditional powers.
The distortion-rate performance of certain randomly-designed scalar
quantizers is determined. The central results are the mean-squared error
distortion and output entropy for quantizing a uniform random variable with
thresholds drawn independently from a uniform distribution. The distortion is
at most 6 times that of an optimal (deterministically-designed) quantizer, and
for a large number of levels the output entropy is reduced by approximately
(1-gamma)/(ln 2) bits, where gamma is the Euler-Mascheroni constant.
We consider the optimal quantization of compressive sensing measurements
following the work on generalization of relaxed belief propagation (BP) for
arbitrary measurement channels. Relaxed BP is an iterative reconstruction
scheme inspired by message passing algorithms on bipartite graphs. Its
asymptotic error performance can be accurately predicted and tracked through
the state evolution formalism.
Minimum mean squared error (MMSE) estimators of signals from samples
corrupted by jitter (timing noise) and additive noise are nonlinear, even when
the signal prior and additive noise have normal distributions. This paper
develops stochastic algorithms based on Gibbs sampling and slice sampling to
approximate optimal MMSE estimators in this Bayesian formulation. Simulations
demonstrate that these nonlinear algorithms can improve significantly upon the
linear MMSE estimator.
This paper examines the problem of estimating the parameters of a bandlimited
signal from samples corrupted by random jitter (timing noise) and additive iid
Gaussian noise, where the signal lies in the span of a finite basis. For the
presented classical estimation problem, the Cramer-Rao lower bound (CRB) is
computed, and an Expectation-Maximization (EM) algorithm approximating the
maximum likelihood (ML) estimator is developed. Simulations are performed to
study the convergence properties of the EM algorithm and compare the
performance both against the CRB and a basic linear estimator.
Frame permutation quantization (FPQ) is a new vector quantization technique
using finite frames. In FPQ, a vector is encoded using a permutation source
code to quantize its frame expansion. This means that the encoding is a partial
ordering of the frame expansion coefficients. Compared to ordinary permutation
source coding, FPQ produces a greater number of possible quantization rates and
a higher maximum rate. Various representations for the partitions induced by
FPQ are presented and reconstruction algorithms based on linear programming and
quadratic programming are derived.
Permutation codes are a class of structured vector quantizers with a
computationally-simple encoding procedure. Here we provide an extension that
preserves the computational simplicity but yields improved operational
rate--distortion performance. The new class of vector quantizers has a codebook
comprising several permutation codes as subcodes. Methods for designing good
code parameters are given. One method depends on optimizing the rate allocation
in a shape--gain vector quantizer with gain-dependent wrapped spherical shape
codebook.
The replica method is a non-rigorous but widely-accepted technique from
statistical physics used in the asymptotic analysis of large, random, nonlinear
problems. This paper applies the replica method to non-Gaussian maximum a
posteriori (MAP) estimation. It is shown that with random linear measurements
and Gaussian noise, the asymptotic behavior of the MAP estimate of an
n-dimensional vector decouples as n scalar MAP estimators. The result is a
counterpart to Guo and Verdu's replica analysis of minimum mean-squared error
estimation.
The replica method is a non-rigorous but widely-accepted technique from
statistical physics used in the asymptotic analysis of large, random, nonlinear
problems. This paper applies the replica method to non-Gaussian maximum a
posteriori (MAP) estimation. It is shown that with random linear measurements
and Gaussian noise, the asymptotic behavior of the MAP estimate of an
n-dimensional vector decouples as n scalar MAP estimators. The result is a
counterpart to Guo and Verdu's replica analysis of minimum mean-squared error
estimation.