Abstract
Socioeconomic bias in society exacerbates disparities, influencing access toopportunities and resources based on individuals' economic and socialbackgrounds. This pervasive issue perpetuates systemic inequalities, hinderingthe pursuit of inclusive progress as a society. In this paper, we investigatethe presence of socioeconomic bias, if any, in large language models. To thisend, we introduce a novel dataset SilverSpoon, consisting of 3000 samples thatillustrate hypothetical scenarios that involve underprivileged peopleperforming ethically ambiguous actions due to their circumstances, and askwhether the action is ethically justified. Further, this dataset has adual-labeling scheme and has been annotated by people belonging to both ends ofthe socioeconomic spectrum. Using SilverSpoon, we evaluate the degree ofsocioeconomic bias expressed in large language models and the variation of thisdegree as a function of model size. We also perform qualitative analysis toanalyze the nature of this bias. Our analysis reveals that while humansdisagree on which situations require empathy toward the underprivileged, mostlarge language models are unable to empathize with the socioeconomicallyunderprivileged regardless of the situation. To foster further research in thisdomain, we make SilverSpoon and our evaluation harness publicly available.