Abstract
Camera-based remote photoplethysmography (rPPG) enables contactlessmeasurement of important physiological signals such as pulse rate (PR).However, dynamic and unconstrained subject motion introduces significantvariability into the facial appearance in video, confounding the ability ofvideo-based methods to accurately extract the rPPG signal. In this study, weleverage the 3D facial surface to construct a novel orientation-conditionedfacial texture video representation which improves the motion robustness ofexisting video-based facial rPPG estimation methods. Our proposed methodachieves a significant 18.2% performance improvement in cross-dataset testingon MMPD over our baseline using the PhysNet model trained on PURE, highlightingthe efficacy and generalization benefits of our designed video representation.We demonstrate significant performance improvements of up to 29.6% in alltested motion scenarios in cross-dataset testing on MMPD, even in the presenceof dynamic and unconstrained subject motion, emphasizing the benefits ofdisentangling motion through modeling the 3D facial surface for motion robustfacial rPPG estimation. We validate the efficacy of our design decisions andthe impact of different video processing steps through an ablation study. Ourfindings illustrate the potential strengths of exploiting the 3D facial surfaceas a general strategy for addressing dynamic and unconstrained subject motionin videos. The code is available athttps://samcantrill.github.io/orientation-uv-rppg/.