MMVR: Millimeter-wave Multi-View Radar Dataset and Benchmark for Indoor Perception

Mohammad Mahbubur Rahman, Ryoma Yataka, Sorachi Kato, Pu Wang*, Peizhao Li, Adriano Cardace, Petros Boufounos ;

"∗ : Equal contribution. † : The work of M. Rahman (Univ. of Alabama, USA), S. Kato (Osaka Univ., Japan), P. Li (Brandeis Univ., USA), and A. Cardace (Univ. of Bologna, Italy) was done during their internship at MERL. ♯ : The work was done as a visiting scientist from Mitsubishi Electric Corporation, Japan. ‡ : Project Lead. Compared with an extensive list of automotive radar datasets that support autonomous driving, indoor radar datasets are scarce at a smaller scale in the format of low-resolution radar point clouds and usually under an open-space single-room setting. In this paper, we scale up indoor radar data collection using multi-view high-resolution radar heatmap in a multi-day, multi-room, and multi-subject setting, with an emphasis on the diversity of environment and subjects. Referred to as the millimeter-wave multi-view radar (MMVR) dataset, it consists of 345K multi-view radar frames collected from 25 human subjects over 6 different rooms, 446K annotated bounding boxes/segmentation instances, and 7.59 million annotated keypoints to support three major perception tasks of object detection, pose estimation, and instance segmentation, respectively. For each task, we report performance benchmarks under two protocols: a single subject in an open space and multiple subjects in several cluttered rooms with two data splits: random split and cross-environment split over 395 1-min data segments. We anticipate that MMVR facilitates indoor radar perception development for indoor vehicle (robot/humanoid) navigation, building energy management, and elderly care for better efficiency, user experience, and safety. The MMVR dataset is available at https://doi.org/10.5281/zenodo.12611978."