Families of alpha- beta- and gamma- divergences: Flexible and robust measures of Similarities

Andrzej Cichocki, Shun ichi Amari

Research output: Contribution to journalArticlepeer-review

249 Citations (SciVal)

Abstract

In this paper, we extend and overview wide families of Alpha-, Beta- and Gamma-divergences and discuss their fundamental properties. In literature usually only one single asymmetric (Alpha, Beta or Gamma) divergence is considered. We show in this paper that there exist families of such divergences with the same consistent properties. Moreover, we establish links and correspondences among these divergences by applying suitable nonlinear transformations. For example, we can generate the Beta-divergences directly from Alpha-divergences and vice versa. Furthermore, we show that a new wide class of Gamma-divergences can be generated not only from the family of Beta-divergences but also from a family of Alpha-divergences. The paper bridges these divergences and shows also their links to Tsallis and Rényi entropies. Most of these divergences have a natural information theoretic interpretation.

Original languageEnglish
Pages (from-to)1532-1568
Number of pages37
JournalEntropy
Volume12
Issue number6
DOIs
Publication statusPublished - Jun 2010
Externally publishedYes

Keywords

  • Csiszár-Morimoto and Bregman divergences
  • Extended Itakura-Saito like divergences
  • Generalized divergences
  • Similarity measures
  • Tsallis and Rényi entropies

Fingerprint

Dive into the research topics of 'Families of alpha- beta- and gamma- divergences: Flexible and robust measures of Similarities'. Together they form a unique fingerprint.

Cite this