GroundUp: Rapid Sketch-Based 3D City Massing

Gizem Esra Unlu*, Mohamed Sayed, Yulia Gryaditskaya, Gabriel Brostow ;

"We propose GroundUp, the first sketch-based ideation tool for 3D city massing of urban areas. We focus on early-stage urban design, where sketching is a common tool and the design starts from balancing planned building volumes (masses) and open spaces. With Human-Centered AI in mind, we aim to help architects quickly revise their ideas by easily switching between 2D sketches and 3D models, allowing for smoother iteration and sharing of ideas. Inspired by feedback from architects and existing workflows, our system takes as a first input a user sketch of multiple buildings in a top-down view. The user then draws a perspective sketch of the envisioned site. Our method is designed to exploit the complementarity of information in the two sketches and allows users to quickly preview and adjust the inferred 3D shapes. Our model, driving the proposed urban massing system, has two main components. First, we propose a novel sketch-to-depth prediction network for perspective sketches that exploits top-down sketch shapes. Second, we amalgamate the complimentary but sparse 2D signals to condition a customarily trained latent diffusion model. Second, we use depth cues derived from the perspective sketch as a condition to our diffusion model, which ultimately completes the geometry in a top-down view. The diffusion model works in the domain of a heightfield, Thus, our final 3D geometry is represented as a heightfield, allowing users to construct the city “from the ground up”. We will release the code, datasets, and interface. The code, datasets, and interface are available at visual.cs.ucl.ac.uk/pubs/groundup."