Neural Procedural Reconstruction for Residential Buildings

Huayi Zeng, Jiaye Wu, Yasutaka Furukawa; The European Conference on Computer Vision (ECCV), 2018, pp. 737-753

This paper proposes a novel 3D reconstruction approach, dubbed Neural Procedural Reconstruction (NPR), which trains deep neural networks to procedurally apply shape grammar rules and reconstruct CAD-quality models from 3D points. In contrast to Procedural Modeling (PM), which randomly applies shape grammar rules to synthesize 3D models, NPR classifies a rule branch to explore and regresses geometric parameters at each rule application. We demonstrate the proposed system for residential buildings with aerial LiDAR as the input. Our 3D models boast extremely compact geometry and semantically segmented architectural components. Qualitative and quantitative evaluations on hundreds of houses show that our approach robustly generates CAD-quality 3D models from raw sensor data, making significant improvements over the existing state-of-the-art.