Exponential machines

A. Novikov, M. Trofimov, I. Oseledets

    Research output: Contribution to journalArticlepeer-review

    6 Citations (Scopus)

    Abstract

    Modeling interactions between features improves the performance of machine learning solutions in many domains (e.g. recommender systems or sentiment analysis). In this paper, we introduce Exponential machines (ExM), a predictor that models all interactions of every order. The key idea is to represent an exponentially large tensor of parameters in a factorized format called tensor train (TT). The tensor train format regularizes the model and lets you control the number of underlying parameters. To train the model, we develop a stochastic Riemannian optimization procedure, which allows us to fit tensors with ¼ 2 56 entries. We show that the model achieves state-of-the-art performance on synthetic data with high-order interactions and that it works on par with high-order factorization machines on a recommender system dataset MovieLens 100 K.

    Original languageEnglish
    Pages (from-to)789-797
    Number of pages9
    JournalBulletin of the Polish Academy of Sciences: Technical Sciences
    DOIs
    Publication statusPublished - 2018

    Keywords

    • Factorization machines
    • Riemannian optimization
    • Tensor decomposition
    • Tensor train

    Fingerprint

    Dive into the research topics of 'Exponential machines'. Together they form a unique fingerprint.

    Cite this