NeuMan: Neural Human Radiance Field from a Single Video

Wei Jiang^1,2, Kwang Moo Yi^1,2, Golnoosh Samei¹, Oncel Tuzel¹, Anurag Ranjan^1
1Apple, ²UBC

Novel Human Pose and Viewpoints Renderings

We show the original video(left upper), the reconstructed canonical human model(right upper), the reconstructed scene model(left bottom), and the animated reposed human together with the scene(right bottom). All models are trained with the proposed method.

Seattle

Citron

Parking

Bike

Jogging

Lab

Telegathering

Our method is able to provide telegathering for multiple persons. It only requires a single video of the human, which makes telegathering accessible to anyone with a cellphone.

Handshake

Dance

Comparison with NeuralBody

We apply NeuralBody to our dataset in a monocular setting. NeuralBody overfits to the training observations, and produce poor rendering on the back of the subject, while ours generalize better and can faithfully render the back.

Ablations

We ablate the impacts of error correction network, and background model conditioning. Both help reconstructing a sharper and cleaner canonical NeRF model.