Information distance estimation between mixtures of multivariate Gaussians

C. T. J. Dodson; C. T. J. Dodson

doi:10.3934/Math.2018.4.439

AIMS Mathematics

2018, Volume 3, Issue 4: 439-447. doi: 10.3934/Math.2018.4.439

Previous Article Next Article

Research article Topical Sections

Information distance estimation between mixtures of multivariate Gaussians

C. T. J. Dodson ^,

School of Mathematics, University of Manchester, Manchester, M13 9PL, UK

Received: 25 July 2018 Accepted: 28 September 2018 Published: 19 October 2018

There are e cient software programs for extracting from large data sets and image sequences certain mixtures of probability distributions, such as multivariate Gaussians, to represent the important features and their mutual correlations needed for accurate document retrieval from databases. This note describes a method to use information geometric methods for distance measures between distributions in mixtures of arbitrary multivariate Gaussians. There is no general analytic solution for the information geodesic distance between two k-variate Gaussians, but for many purposes the absolute information distance may not be essential and comparative values su ce for proximity testing and document retrieval. Also, for two mixtures of di erent multivariate Gaussians we must resort to approximations to incorporate the weightings. In practice, the relation between a reasonable approximation and a true geodesic distance is likely to be monotonic, which is adequate for many applications. Here we consider some choices for the incorporation of weightings in distance estimation and provide illustrative results from simulations of di erently weighted mixtures of multivariate Gaussians.
- information geometry,
- multivariate spatial covariance,
- Gaussian mixtures,
- geodesicdistance,
- approximations
Citation: C. T. J. Dodson. Information distance estimation between mixtures of multivariate Gaussians[J]. AIMS Mathematics, 2018, 3(4): 439-447. doi: 10.3934/Math.2018.4.439

Related Papers:

Abstract

There are e cient software programs for extracting from large data sets and image sequences certain mixtures of probability distributions, such as multivariate Gaussians, to represent the important features and their mutual correlations needed for accurate document retrieval from databases. This note describes a method to use information geometric methods for distance measures between distributions in mixtures of arbitrary multivariate Gaussians. There is no general analytic solution for the information geodesic distance between two k-variate Gaussians, but for many purposes the absolute information distance may not be essential and comparative values su ce for proximity testing and document retrieval. Also, for two mixtures of di erent multivariate Gaussians we must resort to approximations to incorporate the weightings. In practice, the relation between a reasonable approximation and a true geodesic distance is likely to be monotonic, which is adequate for many applications. Here we consider some choices for the incorporation of weightings in distance estimation and provide illustrative results from simulations of di erently weighted mixtures of multivariate Gaussians.

References

[1]	S-I. Amari and H. Nagaoka, Methods of Information Geometry, Oxford, American Mathematical Society, Oxford University Press, 2000.
[2]	K. Arwini and C. T. J. Dodson, Information Geometry Near Randomness and Near Independence, Lecture Notes in Mathematics, Springer-Verlag, New York, Berlin, 2008.
[3]	C. Atkinson and A. F. S. Mitchell, Rao's distance measure, Sankhya: Indian Journal of Statistics, 43 (1981), 345–365.
[4]	J. Cao, D. Mao, Q. Cai, et al. A review of object representation based on local features, Journal of Zhejiang University-SCIENCE C (Computers & Electronics), 14 (2013), 495–504.
[5]	T. Craciunescu and A. Murari, Geodesic distance on Gaussian manifolds for the robust identification of chaotic systems, Nonlinear Dynam, 86 (2016), 677–693.
[6]	P. S. Eriksen, Geodesics connected with the Fisher metric on the multivariate normal manifold. In C.T.J. Dodson, Editor, Proceedings of the Geometrization of Statistical Theory Workshop, Lancaster (1987), 225–229.
[7]	F. Nielsen and F. Barbaresco, Geometric Science of Information, LNCS 8085, Springer, Heidelberg, 2013.
[8]	A. Murari, T. Craciunescu, E. Peluso, et al. Detection of Causal Relations in Time Series A ected by Noise in Tokamaks Using Geodesic Distance on Gaussian Manifolds, Entropy 19 (2017), 569.
[9]	F. Nielsen, V. Garcia and R. Nock, Simplifying Gaussian mixture models via entropic quantization. In Proc. 17^th European Signal Processing Conference, Glasgow, Scotland 24-28 August 2009, 2012–2016.
[10]	A. Shabbir, G. Verdoolaege and G. Van Oost. Multivariate texture discrimination based on geodesics to class centroids on a generalized Gaussian manifold. In F. Nielsen and F. Barbaresco (Eds) Geometric Science of Information, LNCS 8085, Springer, Berlin-Heidelberg, 2013,853–860.
[11]	L. T. Skovgaard, A Riemannian geometry of the multivariate normal model, Scand. J. Stat., 11 (1984), 211–223.
[12]	J. Soldera, C. T. J. Dodson and J. Scharcanski, Face recognition based on texture information and geodesic distance approximations between multivariate normal distributions, Measurement Science and Technology, 2018.
[13]	G. Verdoolaege and A. Shabbir. Color Texture Discrimination Using the Principal Geodesic Distance on a Multivariate Generalized Gaussian Manifold, International Conference on Networked Geometric Science of Information, Springer, Cham Berlin-Heidelberg, 2015,379–386.

Reader Comments

Your name:*

Email:*
© 2018 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)