Neural Poisson Surface Reconstruction: Resolution-Agnostic Shape Reconstruction from Point Clouds

Anonymous
Paper arXiv Code
Image of results from nPSR compared to different baselines

Resolution agnostic deep learning to solve the Poisson equation for 3D shape reconstruction.

Abstract

We introduce nPSR, an architecture for shape reconstruction that addresses the challenge of recovering 3D shapes from points. Traditional deep neural networks face challenges with common 3D shape discretization techniques due to their computational complexity at higher resolutions. To overcome this, we leverage Fourier Neural Operators (FNOs) to solve the Poisson equation and reconstruct a mesh from oriented point cloud measurements. nPSR exhibits two main advantages. First, it enables efficient training on low-resolution data while achieving comparable performance at high-resolution evaluation, thanks to the resolution-agnostic nature of FNOs. This feature allows for one-shot super-resolution. Second, our method surpasses existing approaches in reconstruction quality while being differentiable and robust with respect to point sampling rates. Overall, our proposed method not only improves upon the limitations of classical deep neural networks in shape reconstruction but also achieves superior results in terms of reconstruction quality, running time, and resolution agnosticism.

Context

For 3D shape reconstruction is about reconstructing, a common subproblem is to convert a sparse and noisy point cloud of a shape into a mesh. There are a variety of methods for doing this. Perhaps the most common one is called (screened) Poisson Surface Reconstruction. It works by solving a partial differential equation, the Poisson equation. The solution of the PDE ends up approximately being the indicator function of the object:

Visualisation of Poisson Surface Reconstruction by Kazhdan et al.

In this paper, we propose a neural network based on the Fourier Neural Operator to learn the solution operator of the PDE. This neural operator allows us to learn a network that maps the input (oriented) point cloud directly to the solution. The neural operator learns mappings between function spaces and works with arbitrary resolution. This allows us to train the network on low-resolution data and evaluate it on high-resolution data.

Fourier Neural Operator architecture

nPSR is not trained directly on the PDE, but on pairs of noisy point clouds and ground truth objects. This allows our model to learn a model of the noise and perform better than a general PDE solver. Our method is also differentiable and can be used as part of a differentiable pipeline.

BibTeX

@article{anonymous2023nPSR,
      title={Neural Poisson Surface Reconstruction:
        Resolution-Agnostic Shape Reconstruction from Point Clouds},
      author={Anonymous},
      journal={arXiv preprint arXiv:2308.01766},
      year={2023}
    }