Parsing images of overlapping organisms with deep singling-out networks

Victor Yurchenko, Victor Lempitsky

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    8 Citations (Scopus)

    Abstract

    This work is motivated by the mostly unsolved task of parsing biological images with multiple overlapping articulated model organisms (such as worms or larvae). We present a general approach that separates the two main challenges associated with such data, individual object shape estimation and object groups disentangling. At the core of the approach is a deep feed-forward singling-out network (SON) that is trained to map each local patch to a vectorial descriptor that is sensitive to the characteristics (e.g. shape) of a central object, while being invariant to the variability of all other surrounding elements. Given a SON, a local image patch can be matched to a gallery of isolated elements using their SON-descriptors, thus producing a hypothesis about the shape of the central element in that patch. The image-level optimization based on integer programming can then pick a subset of the hypotheses to explain (parse) the whole image and disentangle groups of organisms. While sharing many similarities with existing "analysisby- synthesis" approaches, our method avoids the need for stochastic search in the high-dimensional configuration space and numerous rendering operations at test-time. We show that our approach can parse microscopy images of three popular model organisms (the C.Elegans roundworms, the Drosophila larvae, and the E. Coli bacteria) even under significant crowding and overlaps between organisms. We speculate that the overall approach is applicable to a wider class of image parsing problems concerned with crowded articulated objects, for which rendering training images is possible.

    Original languageEnglish
    Title of host publicationProceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages4752-4760
    Number of pages9
    ISBN (Electronic)9781538604571
    DOIs
    Publication statusPublished - 6 Nov 2017
    Event30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 - Honolulu, United States
    Duration: 21 Jul 201726 Jul 2017

    Publication series

    NameProceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
    Volume2017-January

    Conference

    Conference30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017
    Country/TerritoryUnited States
    CityHonolulu
    Period21/07/1726/07/17

    Fingerprint

    Dive into the research topics of 'Parsing images of overlapping organisms with deep singling-out networks'. Together they form a unique fingerprint.

    Cite this