Manifold Modeling in Embedded Space: An Interpretable Alternative to Deep Image Prior

Tatsuya Yokota, Hidekata Hontani, Qibin Zhao, Andrzej Cichocki

Research output: Contribution to journalArticlepeer-review

6 Citations (Scopus)

Abstract

Deep image prior (DIP), which uses a deep convolutional network (ConvNet) structure as an image prior, has attracted wide attention in computer vision and machine learning. DIP empirically shows the effectiveness of the ConvNet structures for various image restoration applications. However, why the DIP works so well is still unknown. In addition, the reason why the convolution operation is useful in image reconstruction, or image enhancement is not very clear. This study tackles this ambiguity of ConvNet/DIP by proposing an interpretable approach that divides the convolution into ``delay embedding'' and ``transformation'' (i.e., encoder-decoder). Our approach is a simple, but essential, image/tensor modeling method that is closely related to self-similarity. The proposed method is called manifold modeling in embedded space (MMES) since it is implemented using a denoising autoencoder in combination with a multiway delay-embedding transform. In spite of its simplicity, MMES can obtain quite similar results to DIP on image/tensor completion, super-resolution, deconvolution, and denoising. In addition, MMES is proven to be competitive with DIP, as shown in our experiments. These results can also facilitate interpretation/characterization of DIP from the perspective of a ``low-dimensional patch-manifold prior.''

Original languageEnglish
JournalIEEE Transactions on Neural Networks and Learning Systems
DOIs
Publication statusPublished - 2020
Externally publishedYes

Keywords

  • Autoencoder (AE)
  • convolutional neural network (CNN)
  • deblurring
  • deconvolution
  • delay embedding
  • denoising AE (DAE)
  • Hankelization
  • image inpainting
  • manifold model
  • super-resolution
  • tensor completion.

Fingerprint

Dive into the research topics of 'Manifold Modeling in Embedded Space: An Interpretable Alternative to Deep Image Prior'. Together they form a unique fingerprint.

Cite this