Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis

Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    357 Citations (Scopus)

    Abstract

    The recent work of Gatys et al., who characterized the style of an image by the statistics of convolutional neural network filters, ignited a renewed interest in the texture generation and image stylization problems. While their image generation technique uses a slow optimization process, recently several authors have proposed to learn generator neural networks that can produce similar outputs in one quick forward pass. While generator networks are promising, they are still inferior in visual quality and diversity compared to generation-by-optimization. In this work, we advance them in two significant ways. First, we introduce an instance normalization module to replace batch normalization with significant improvements to the quality of image stylization. Second, we improve diversity by introducing a new learning formulation that encourages generators to sample unbiasedly from the Julesz texture ensemble, which is the equivalence class of all images characterized by certain filter responses. Together, these two improvements take feed forward texture synthesis and image stylization much closer to the quality of generation-via-optimization, while retaining the speed advantage.

    Original languageEnglish
    Title of host publicationProceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages4105-4113
    Number of pages9
    ISBN (Electronic)9781538604571
    DOIs
    Publication statusPublished - 6 Nov 2017
    Event30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 - Honolulu, United States
    Duration: 21 Jul 201726 Jul 2017

    Publication series

    NameProceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
    Volume2017-January

    Conference

    Conference30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
    Country/TerritoryUnited States
    CityHonolulu
    Period21/07/1726/07/17

    Fingerprint

    Dive into the research topics of 'Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis'. Together they form a unique fingerprint.

    Cite this