Trial lecture
Autoregressive vs masked language models
Time and place: April 10, 2026 12:00 PM. Kristen Nygaards sal (5370), Ole Johan Dahls hus
Main research findings
Modern language models – the technology behind tools like ChatGPT – keep improving by training on more text, but the supply of human-written text is running out. This "data wall" already limits most of the world's roughly 7,000 languages and will eventually affect even the most resource-rich language, English.
This thesis develops methods that help language models learn more from less data. We present new training techniques that can produce competitive models using a fraction of the data that standard approaches require. The results span the full life cycle of language models: from initial training on limited text to the final step of turning a trained model into a useful assistant. Norwegian serves as the primary case study, but the methods apply broadly to any language-based data constraints.
These findings matter because most people do not speak English. When language technology only works well for a few data-rich languages, it deepens existing inequalities and may push speakers toward dominant languages. Our work helps bridge this gap by making high-quality language models feasible for smaller languages.
Adjudication committee
- Senior Research Scientist David Salinas, ELLIS institute Tübingen and University of Freiburg, Germany
- Associate Professor Anna Rogers, IT University of Copenhagen, Denmark
- Associate Professor Ali Ramezani-Kebrya,University of Oslo, Norway
Supervisors
Associate Professor Andrey Kutuzov, University of Oslo, Norway
Professor Erik Velldal, University of Oslo, Norway
Professor Lilja Øvrelid, University of Oslo, Norway
Chair of defence
- Professor Ole Christian Lingjærde , Department of Informatics, University of Oslo, Norway
Contact information at Department: Pernille Adine Nordby