Hetecooper: Feature Collaboration Graph for Heterogeneous Collaborative Perception

Congzhang Shao, Guiyang Luo*, Quan Yuan*, Yifu Chen, Yilin Liu, Gong Kexin, Jinglin Li ;

"Collaborative perception effectively expands the perception range of agents by sharing perceptual information, and it addresses the occlusion problem in single-vehicle perception. Most of the existing works are based on the assumption of perception model homogeneity. However, in actual collaboration scenarios, agents use different perception model architectures, which leads to differences in the size, number of channels and semantic space of intermediate features shared among collaborators, bringing challenges to collaboration. We introduce Hetecooper, a collaborative perception framework for scenarios with heterogeneous perception models. To model the correlation between heterogeneous features, we construct the feature collaboration graph, which completely preserves the semantic information and spatial information of features. Furthermore, a message passing mechanism based on graph transformer is designed to transfer feature messages in the feature collaboration graph. Firstly, the number of node channels and the semantic space are unified by the semantic mapper. Then, the feature information is aggregated by the edge weight guided attention, and finally the fusion of heterogeneous features is realized. Test results demonstrate that our method achieves superior performance in both model homogeneity and heterogeneity scenarios, and also has good scalability to the change of feature size."