Abstract
Uncertainty estimation is a significant issue for current large languagemodels (LLMs) that are generally poorly calibrated and over-confident,especially with reinforcement learning from human feedback (RLHF). Unlikehumans, whose decisions and confidences not only stem from intrinsic beliefsbut can also be adjusted through daily observations, existing calibrationmethods for LLMs focus on estimating or eliciting individual confidence withouttaking full advantage of the "Collective Wisdom": the interaction amongmultiple LLMs that can collectively improve both accuracy and calibration. Inthis work, we propose Collaborative Calibration, a post-hoc training-freecalibration strategy that leverages the collaborative and expressivecapabilities of multiple tool-augmented LLM agents in a simulated groupdeliberation process. We demonstrate the effectiveness of CollaborativeCalibration on generative QA tasks across various domains, showing itspotential in harnessing the rationalization of collectively calibratedconfidence assessments and improving the reliability of model predictions.