TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer

Maria Kolos, Artem Sevastopolsky, Victor Lempitsky

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

5 Citations (Scopus)

Abstract

We propose and evaluate a neural point-based graphics method that can model semi-transparent scene parts. Similarly to its predecessor pipeline, ours uses point clouds to model proxy geometry, and augments each point with a neural descriptor. Additionally, a learnable transparency value is introduced in our approach for each point.Our neural rendering procedure consists of two steps. Firstly, the point cloud is rasterized using ray marching into a multi-channel image. This is followed by the neural rendering step that 'translates' the rasterized image into an RGB output using a learnable convolutional network. New scenes can be modeled using gradient-based optimization of neural descriptors and of the rendering network.We show that novel views of semi-transparent point cloud scenes can be generated after training with our approach. Our experiments demonstrate the benefit of introducing semi-transparency into the neural point-based modeling for a range of scenes with semi-transparent parts. The project materials and the code are available at http://saic-violet.github.io/transpr.

Original languageEnglish
Title of host publicationProceedings - 2020 International Conference on 3D Vision, 3DV 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1167-1175
Number of pages9
ISBN (Electronic)9781728181288
DOIs
Publication statusPublished - Nov 2020
Event8th International Conference on 3D Vision, 3DV 2020 - Virtual, Fukuoka, Japan
Duration: 25 Nov 202028 Nov 2020

Publication series

NameProceedings - 2020 International Conference on 3D Vision, 3DV 2020

Conference

Conference8th International Conference on 3D Vision, 3DV 2020
Country/TerritoryJapan
CityVirtual, Fukuoka
Period25/11/2028/11/20

Keywords

  • computer vision
  • neural rendering
  • photogrammetry
  • point based rendering
  • point cloud

Fingerprint

Dive into the research topics of 'TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer'. Together they form a unique fingerprint.

Cite this