Unleashing the Power of Prompt-driven Nucleus Instance Segmentation

Zhongyi Shui*, Yunlong Zhang, Kai Yao, Chenglu Zhu, Sunyi Zheng, Jingxiong Li, Honglin Li, YUXUAN SUN, Ruizhe Guo, Lin Yang* ;

"Nucleus instance segmentation in histology images is crucial for a broad spectrum of clinical applications. Current dominant algorithms rely on regression of nuclear proxy maps. Distinguishing nucleus instances from the estimated maps requires carefully curated post-processing, which is error-prone and parameter-sensitive. Recently, the Segment Anything Model (SAM) has earned huge attention in medical image segmentation, owing to its impressive generalization ability and promptable property. Nevertheless, its potential on nucleus instance segmentation remains largely underexplored. In this paper, we present a novel prompt-driven framework that consists of a nucleus prompter and SAM for automatic nucleus instance segmentation. Specifically, the prompter is developed to generate a unique point prompt for each nucleus, while SAM is fine-tuned to produce its corresponding mask. Furthermore, we propose to integrate adjacent nuclei as negative prompts to enhance model’s capability to identify overlapping nuclei. Without complicated post-processing, our proposed method sets a new state-of-the-art performance on three challenging benchmarks. Code available at https:// github.com/windygoo/PromptNucSeg."