Abstract
Neural language models, particularly large-scale ones, have been consistentlyproven to be most effective in predicting brain neural activity across a rangeof studies. However, previous research overlooked the comparison of thesemodels with psychologically plausible ones. Moreover, evaluations were relianton limited, single-modality, and English cognitive datasets. To address thesequestions, we conducted an analysis comparing encoding performance of variousneural language models and psychologically plausible models. Our study utilizedextensive multi-modal cognitive datasets, examining bilingual word anddiscourse levels. Surprisingly, our findings revealed that psychologicallyplausible models outperformed neural language models across diverse contexts,encompassing different modalities such as fMRI and eye-tracking, and spanninglanguages from English to Chinese. Among psychologically plausible models, theone incorporating embodied information emerged as particularly exceptional.This model demonstrated superior performance at both word and discourse levels,exhibiting robust prediction of brain activation across numerous regions inboth English and Chinese.