Abstract
The aim of SemEval-2024 Task 1, "Semantic Textual Relatedness for African andAsian Languages" is to develop models for identifying semantic textualrelatedness (STR) between two sentences using multiple languages (14 Africanand Asian languages) and settings (supervised, unsupervised, andcross-lingual). Large language models (LLMs) have shown impressive performanceon several natural language understanding tasks such as multilingual machinetranslation (MMT), semantic similarity (STS), and encoding sentence embeddings.Using a combination of LLMs that perform well on these tasks, we developed twoSTR models, $\textit{TranSem}$ and $\textit{FineSem}$, for the supervised andcross-lingual settings. We explore the effectiveness of several trainingmethods and the usefulness of machine translation. We find that directfine-tuning on the task is comparable to using sentence embeddings andtranslating to English leads to better performance for some languages. In thesupervised setting, our model performance is better than the official baselinefor 3 languages with the remaining 4 performing on par. In the cross-lingualsetting, our model performance is better than the baseline for 3 languages(leading to $1^{st}$ place for Africaans and $2^{nd}$ place for Indonesian), ison par for 2 languages and performs poorly on the remaining 7 languages. Ourcode is publicly available at https://github.com/dipta007/SemEval24-Task8.