Abstract
Estimating the 6D pose of objects unseen during training is highly desirableyet challenging. Zero-shot object 6D pose estimation methods address thischallenge by leveraging additional task-specific supervision provided bylarge-scale, photo-realistic synthetic datasets. However, their performanceheavily depends on the quality and diversity of rendered data and they requireextensive training. In this work, we show how to tackle the same task butwithout training on specific data. We propose FreeZe, a novel solution thatharnesses the capabilities of pre-trained geometric and vision foundationmodels. FreeZe leverages 3D geometric descriptors learned from unrelated 3Dpoint clouds and 2D visual features learned from web-scale 2D images togenerate discriminative 3D point-level descriptors. We then estimate the 6Dpose of unseen objects by 3D registration based on RANSAC. We also introduce anovel algorithm to solve ambiguous cases due to geometrically symmetric objectsthat is based on visual features. We comprehensively evaluate FreeZe across theseven core datasets of the BOP Benchmark, which include over a hundred 3Dobjects and 20,000 images captured in various scenarios. FreeZe consistentlyoutperforms all state-of-the-art approaches, including competitors extensivelytrained on synthetic 6D pose estimation data. Code will be publicly availableat https://andreacaraffa.github.io/freeze.