Fast ConvNets Using Group-Wise Brain Damage

Vadim Lebedev, Victor Lempitsky

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    235 Citations (Scopus)

    Abstract

    We revisit the idea of brain damage, i.e. the pruning of the coefficients of a neural network, and suggest how brain damage can be modified and used to speedup convolutional layers in ConvNets. The approach uses the fact that many efficient implementations reduce generalized convolutions to matrix multiplications. The suggested brain damage process prunes the convolutional kernel tensor in a group-wise fashion. After such pruning, convolutions can be reduced to multiplications of thinned dense matrices, which leads to speedup. We investigate different ways to add group-wise prunning to the learning process, and show that severalfold speedups of convolutional layers can be attained using group-sparsity regularizers. Our approach can adjust the shapes of the receptive fields in the convolutional layers, and even prune excessive feature maps from ConvNets, all in data-driven way.

    Original languageEnglish
    Title of host publicationProceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
    PublisherIEEE Computer Society
    Pages2554-2564
    Number of pages11
    ISBN (Electronic)9781467388504
    DOIs
    Publication statusPublished - 9 Dec 2016
    Event29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016 - Las Vegas, United States
    Duration: 26 Jun 20161 Jul 2016

    Publication series

    NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
    Volume2016-December
    ISSN (Print)1063-6919

    Conference

    Conference29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
    Country/TerritoryUnited States
    CityLas Vegas
    Period26/06/161/07/16

    Fingerprint

    Dive into the research topics of 'Fast ConvNets Using Group-Wise Brain Damage'. Together they form a unique fingerprint.

    Cite this