Thinning, Entropy, and the Law of Thin Numbers

P Harremoës, OT Johnson, I. Kontoyiannis

Research output: Contribution to journalArticle (Academic Journal)peer-review

26 Citations (Scopus)


Rényi's thinning operation on a discrete random variable is a natural discrete analog of the scaling operation for continuous random variables. The properties of thinning are investigated in an information-theoretic context, especially in connection with information-theoretic inequalities related to Poisson approximation results. The classical Binomial-to-Poisson convergence (sometimes referred to as the “law of small numbers”) is seen to be a special case of a thinning limit theorem for convolutions of discrete distributions. A rate of convergence is provided for this limit, and nonasymptotic bounds are also established. This development parallels, in part, the development of Gaussian inequalities leading to the information-theoretic version of the central limit theorem. In particular, a “thinning Markov chain” is introduced, and it is shown to play a role analogous to that of the Ornstein-Uhlenbeck process in connection to the entropy power inequality.
Translated title of the contributionThinning, Entropy and the Law of Thin Numbers
Original languageEnglish
Pages (from-to)4228 - 4244
Number of pages17
JournalIEEE Transactions on Information Theory
Issue number9
Publication statusPublished - Sept 2010


Dive into the research topics of 'Thinning, Entropy, and the Law of Thin Numbers'. Together they form a unique fingerprint.

Cite this