While LLM-based chatbots such as ChatGPT produce understandable and usually grammatically correct Norwegian, there are still artefacts from the much larger corpora of English that make their way into the generated text. For example, research has shown that Norwegian text generation from ChatGPT shows clear signs of being influenced by English in how word senses are distributed in translated text (Ebeling, 2024). However, as the general language generation capabilities are improving, it becomes increasingly difficult to evaluate these, and traditional methods
When evaluating text that is translated from or influenced by English, the term Anglicism is often used. An anglicism can be a recent loan from English, or an English-sounding phrasing in Norwegian or other languages. Sometimes, a direct translation from English might work, but in some cases, a direct translation might sound off.
This reliance on English becomes especially clear in the case of idioms. Han kalte det en dag is not an appropriate translation of 'He called it a day' (in the sense of leaving work, for example), while others like krokodilletårer `crocodile tears' are less informative.
This project would explore idioms as an aid in investigating the text generation capabilities of text generation tasks such as translation for Norwegian, and would contribute to an important set of benchmarks for evaluating the quality of generated Norwegian.
The student could create a parallel Norwegian–English dataset of idioms that could be used to assess these capabilities, but it is also possible to explore other types of anglicisms.
A related project at ILOS that focuses on corpus linguistics could form some basis of the project, but the focus would be on dataset creation, benchmarking, and (L)LM capabilities in the context of machine translation. There are also related efforts in the ongoing CAALLM project, where LTG is a partner.