Abstract
We present a zero-shot pose optimization method that enforces accuratephysical contact constraints when estimating the 3D pose of humans. Our centralinsight is that since language is often used to describe physical interaction,large pretrained text-based models can act as priors on pose estimation. We can thus leverage this insight to improve pose estimation by convertingnatural language descriptors, generated by a large multimodal model (LMM), intotractable losses to constrain the 3D pose optimization. Despite its simplicity,our method produces surprisingly compelling pose reconstructions of people inclose contact, correctly capturing the semantics of the social and physicalinteractions. We demonstrate that our method rivals more complexstate-of-the-art approaches that require expensive human annotation of contactpoints and training specialized models. Moreover, unlike previous approaches,our method provides a unified framework for resolving self-contact andperson-to-person contact.