Introducing cosmosGPT: Monolingual Training for Turkish Language Models

  • 2024-04-26 12:34:11
  • H. Toprak Kesgin, M. Kaan Yuce, Eren Dogan, M. Egemen Uzun, Atahan Uz, H. Emre Seyrek, Ahmed Zeer, M. Fatih Amasyali
  • 0

Abstract

The number of open source language models that can produce Turkish isincreasing day by day, as in other languages. In order to create the basicversions of such models, the training of multilingual models is usuallycontinued with Turkish corpora. The alternative is to train the model with onlyTurkish corpora. In this study, we first introduce the cosmosGPT models that wecreated with this alternative method. Then, we introduce new finetune datasetsfor basic language models to fulfill user requests and new evaluation datasetsfor measuring the capabilities of Turkish language models. Finally, acomprehensive comparison of the adapted Turkish language models on differentcapabilities is presented. The results show that the language models we builtwith the monolingual corpus have promising performance despite being about 10times smaller than the others.

 

Quick Read (beta)

loading the full paper ...