Splatter Image: Ultra-Fast Single-View 3D Reconstruction

  • 2024-04-16 18:56:19
  • Stanislaw Szymanowicz, Christian Rupprecht, Andrea Vedaldi
  • 0

Abstract

We introduce the \method, an ultra-efficient approach for monocular 3D objectreconstruction. Splatter Image is based on Gaussian Splatting, which allowsfast and high-quality reconstruction of 3D scenes from multiple images. Weapply Gaussian Splatting to monocular reconstruction by learning a neuralnetwork that, at test time, performs reconstruction in a feed-forward manner,at 38 FPS. Our main innovation is the surprisingly straightforward design ofthis network, which, using 2D operators, maps the input image to one 3DGaussian per pixel. The resulting set of Gaussians thus has the form an image,the Splatter Image. We further extend the method take several images as inputvia cross-view attention. Owning to the speed of the renderer (588 FPS), we usea single GPU for training while generating entire images at each iteration tooptimize perceptual metrics like LPIPS. On several synthetic, real,multi-category and large-scale benchmark datasets, we achieve better results interms of PSNR, LPIPS, and other metrics while training and evaluating muchfaster than prior works. Code, models, demo and more results are available athttps://szymanowiczs.github.io/splatter-image.

 

Quick Read (beta)

loading the full paper ...