Abstract
Broad-scale marine surveys performed by underwater vehicles significantlyincrease the availability of coral reef imagery, however it is costly andtime-consuming for domain experts to label images. Point label propagation isan approach used to leverage existing image data labeled with sparse pointlabels. The resulting augmented ground truth generated is then used to train asemantic segmentation model. Here, we first demonstrate that recent advances infoundation models enable generation of multi-species coral augmented groundtruth masks using denoised DINOv2 features and K-Nearest Neighbors (KNN),without the need for any pre-training or custom-designed algorithms. Forextremely sparsely labeled images, we propose a labeling regime based onhuman-in-the-loop principles, resulting in significant improvement inannotation efficiency: If only 5 point labels per image are available, ourproposed human-in-the-loop approach improves on the state-of-the-art by 17.3%for pixel accuracy and 22.6% for mIoU; and by 10.6% and 19.1% when 10 pointlabels per image are available. Even if the human-in-the-loop labeling regimeis not used, the denoised DINOv2 features with a KNN outperforms the priorstate-of-the-art by 3.5% for pixel accuracy and 5.7% for mIoU (5 grid points).We also provide a detailed analysis of how point labeling style and thequantity of points per image affects the point label propagation quality andprovide general recommendations on maximizing point label efficiency.