ETH Zurich - D-INFK - IVC - CVG - Research - Multi-Body Non-Intersection Constraints

Multi-Body Depth-Map Fusion with Non-Intersection Constraints


Bastien Jacquet
ETH Zürich
Switzerland
Christian Häne
ETH Zürich
Switzerland
Roland Angst
Stanford
USA
Marc Pollefeys
ETH Zürich
Switzerland


Screen No Constraints
Screen With Constraints
Without constraints With constraints
No Constraints
With Constraints
Without constraints With constraints
Background | Both | Box
Drawer
Segmented Drawer

Abstract

Depthmap fusion is the problem of computing dense 3D reconstructions from a set of depthmaps. Whereas this problem has received a lot of attention for purely rigid scenes, there is remarkably little prior work for dense reconstructions of scenes consisting of several moving rigid bodies or parts. This paper therefore explores this multibody depthmap fusion problem. A first observation in the multi-body setting is that when treated naively, ghosting artifacts will emerge, ie. the same part will be reconstructed multiple times at different positions. We therefore introduce non-intersection constraints which resolve these issues: at any point in time, a point in space can only be occupied by at most one part. Interestingly enough, these constraints can be expressed as linear inequalities and as such define a convex set. We therefore propose to phrase the multi-body depthmap fusion problem in a convex voxel labeling framework. Experimental evaluation shows that our approach succeeds in computing artifact-free dense reconstructions of the individual parts with a minimal overhead due to the non-intersection constraints.

Publication

Dataset

3D results

  • Laptop
  • Drawer
  • Box

Acknowledgments

This work has been supported by the Max Planck Center for Visual Computing and Communication, by the 4DVideo ERC Starting Grant Nr. 210806 and by the Swiss National Science Foundation under Project Nr. 143422.


© CVG, ETH Zürich lm@inf.ethz.ch