Kullbach Distribution


< Back to Probability Distributions

I love a good mystery. I found a reference to the Kullbach distribution while researching unusual probability distributions. Of course, the first thing I did was to perform a Google search for “Kullbach distribution”. I got exactly one result from Haight [1]

So, what exactly is this mysterious distribution? A delve into the Index to the Distributions of Mathematical Statistics (IDMS) [1] provides a further clue:

The notation GM stands for geometric mean, and Type III is a skewed distribution similar to the binomial distribution [2], which does have the gamma function in the denominator.

The entry “[2]251” refers to page 251 of Kendall’s The Advanced Theory of Statistics , Volume 1 [3]:

But, page 251 of Kendall’s work does not have the “Kullbach distribution” listed, nor is there an entry for “Kullbach” in the back index.

The penny started to drop. This was a typo. A look at a later edition of the Index to the Distributions revealed that the spelling should be Kullback distribution.

But, what is the “Kullbach Distribution?”

Solomon Kullback wrote about relative entropy in 1951 [4], where he described it as

“the mean information for discrimination between H1 and H2 per observation from μ1

The asymmetric “directed divergence” (a distance metric) is now called the Kullback–Leibler divergence.

I did find reference to the formula listed in IDMS in Kullback’s 1931 paper An Application of Characteristic Functions to the Distribution Problem of Statistics [5] under a section titled “distribution of the geometric mean”, which is obtained from that of the arithmetic mean μ by the transformation of μ = log gn .

kullbach distribution
Kullback’s 1931 formula.

Mystery solved!

References

[1] Haight, F. (1958). Index to the Distributions of Mathematical Statistics. National Bureau of Standards Report.

[2] Abramowitz, M. and Stegun, I. A. (Eds.). (1972). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, 9th printing. New York: Dover.

[3] KENDALL, M. G. The Advanced Theory of Statistics , Volume 1 , London: Charles Griffin and Co., 1945,

[4] Kullback, S.; Leibler, R.A. (1951). “On information and sufficiency”. Annals of Mathematical Statistics22 (1): 79–86. doi:10.1214/aoms/1177729694. JSTOR 2236703. MR 0039968.

[5] Kullback, S. (1934). An Application of Characteristic Functions to the Distribution Problem of Statistics. The Annals of Mathematical Statistics5(4), 263–307. http://www.jstor.org/stable/2957612 p.277.


One response to “Kullbach Distribution”

  1. delve into the Index to the Distributions of Mathematical Statistics (IDMS) [1] provides a further clue

    “Which?”