David Samuel

• Samuel, David; Øvrelid, Lilja; Velldal, Erik & Kutuzov, Andrey (2026). Fluent Alignment with Disfluent Judges: Post-training for lower-resource languages. I Vondrick, C.; Hariharan, B.; Raffel, C.; Pinto, L.; Yang, D. & Faust, A. (Red.), International Conference on Learning Representations (ICLR) 2026. International Conference on Learning Representations (ICLR). ISSN 9798331339678. s. 94000–94027. doi: https:/proceedings.iclr.cc/paper_files/paper/2026/hash/97a057ab1235218027dc07c50eedee0d-Abstract-Conference.html. Fulltekst i vitenarkiv Vis sammendrag

  We propose a post-training method for lower-resource languages that preserves the fluency of language models even when aligned by disfluent reward models. Preference optimization is now a well-researched topic, but previous work has mostly addressed models for English and Chinese. Lower-resource languages lack both datasets written by native speakers and instruction-tuned language models capable of generating fluent synthetic data. To address this, we focus on developing a fluent preference-aligned language model without any instruction-tuning data in the target language. Our approach uses an on-policy training method, which we compare with two common alternatives: supervised finetuning on machine-translated data and multilingual finetuning. We conduct a case study on Norwegian Bokmål and evaluate fluency through native-speaker assessments. The results show that the on-policy aspect is crucial and outperforms the alternatives without relying on any hard-to-obtain data.

• Samuel, David; Mikhailov, Vladislav; Velldal, Erik; Øvrelid, Lilja; Charpentier, Lucas Georges Gabriel & Kutuzov, Andrey [Vis alle 7 forfattere av denne artikkelen] (2025). Small Languages, Big Models: A Study of Continual Training on Languages of Norway. I Johansson, Richard & Stymne, Sara (Red.), Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). University of Tartu Library. ISSN 9789908531090. s. 573–608. doi: https:/aclanthology.org/2025.nodalida-1.61/. Fulltekst i vitenarkiv Vis sammendrag

  Training large language models requires vast amounts of data, posing a challenge for less widely spoken languages like Norwegian and even more so for truly low-resource languages like Northern S{\'a}mi. To address this issue, we present a novel three-stage continual training approach that substantially improves the downstream performance together with the inference efficiency for the target languages. Based on our findings, we train, evaluate, and openly release a new generative language model for Norwegian Bokm{\r{a}}l, Nynorsk, and Northern S{\'a}mi with 11.4 billion parameters: NorMistral-11B.

• Fedorova, Mariia; Frydenberg, Jonas Sebulon; Handford, Victoria; Langø, Victoria Ovedie Chruickshank; Willoch, Solveig Helene & Midtgaard, Marthe Løken [Vis alle 9 forfattere av denne artikkelen] (2025). Multi-label Scandinavian Language Identification (SLIDE). I Holdt, Špela Arhar; Ilinykh, Nikolai; Scalvini, Barbara; Bruton, Micaella; Debess, Iben Nyholm & Tudor, Crina Madalina (Red.), Proceedings of the Third Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2025). Tartu Ülikool Kirjastus. ISSN 9789908531212. s. 179–189. Fulltekst i vitenarkiv Vis sammendrag

  Identifying closely related languages at sentence level is difficult, in particular because it is often impossible to assign a sentence to a single language. In this paper, we focus on multi-label sentence-level Scandinavian language identification (LID) for Danish, Norwegian Bokmål, Norwegian Nynorsk, and Swedish. We present the Scandinavian Language Identification and Evaluation, SLIDE, a manually curated multi-label evaluation dataset and a suite of LID models with varying speed–accuracy tradeoffs. We demonstrate that the ability to identify multiple languages simultaneously is necessary for any accurate LID method, and present a novel approach to training such multi-label LID models.

• Mikhailov, Vladislav; Enstad, Tita Ranveig; Samuel, David; Farsethås, Hans Christian; Velldal, Erik & Øvrelid, Lilja [Vis alle 7 forfattere av denne artikkelen] (2025). NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark. I Che, Wanxiang; Nabende, Joyce; Shutova, Ekaterina & Pilehvar, Mohammad (Red.), Findings of the Association for Computational Linguistics: ACL 2025. Association for Computational Linguistics (ACL). ISSN 9798891762565. s. 3495–3541. doi: 10.18653/v1/2025.findings-acl.181. Fulltekst i vitenarkiv Vis sammendrag

  This paper introduces NorEval, a new and comprehensive evaluation suite for large-scale standardized benchmarking of Norwegian generative language models (LMs). NorEval consists of 24 high-quality human-created datasets – of which five are created from scratch. In contrast to existing benchmarks for Norwegian, NorEval covers a broad spectrum of task categories targeting Norwegian language understanding and generation, establishes human baselines, and focuses on both of the official written standards of the Norwegian language: Bokmål and Nynorsk. All our datasets and a collection of over 100 human-created prompts are integrated into LM Evaluation Harness, ensuring flexible and reproducible evaluation. We describe the NorEval design and present the results of benchmarking 19 open-source pretrained and instruction-tuned LMs for Norwegian in various scenarios. Our benchmark, evaluation framework, and annotation materials are publicly available.

• Burchell, Laurie; Gibert, Ona de; Arefev, Nikolay; Aulamo, Mikko; Bañón, Marta & Chen, Pinzhen [Vis alle 35 forfattere av denne artikkelen] (2025). An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT). I Che, Wanxiang; Nabende, Joyce; Shutova, Ekaterina & Pilehvar, Mohammad (Red.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics (ACL). ISSN 9798891762510. s. 17452–17485. doi: 10.18653/v1/2025.acl-long.854. Fulltekst i vitenarkiv Vis sammendrag

  Training state-of-the-art large language models requires vast amounts of clean and diverse textual data. However, building suitable multilingual datasets remains a challenge. In this work, we present HPLT v2, a collection of high-quality multilingual monolingual and parallel corpora, extending prior work of the HPLT project. The monolingual portion of the data contains 8T tokens covering 193 languages, while the parallel data contains 380M sentence pairs covering 51 languages. We document the entire data pipeline and release the code to reproduce it. We provide extensive analysis of the quality and characteristics of our data. Finally, we evaluate the performance of language models and machine translation systems trained on HPLT v2, demonstrating its value.

• Rosa, Javier De La; Mikhailov, Vladislav; Zhang, Lemei; Wetjen, Freddy; Samuel, David & Liu, Peng [Vis alle 19 forfattere av denne artikkelen] (2025). The Impact of Copyrighted Material on Large Language Models: A Norwegian Perspective. I Johansson, Richard & Stymne, Sara (Red.), Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). University of Tartu Library. ISSN 9789908531090. s. 544–560. Fulltekst i vitenarkiv
• Charpentier, Lucas Georges Gabriel & Samuel, David (2024). GPT or BERT: why not both? I Hu, Michael Y.; Mueller, Aaron; Ross, Candace; Williams, Adina; Linzen, Tal; Zhuang, Chengxu; Choshen, Leshem; Cotterell, Ryan; Warstadt, Alex & Wilcox, Ethan Gotlieb (Red.), The 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning. Association for Computational Linguistics (ACL). ISSN 9798891762220. s. 262–283. Fulltekst i vitenarkiv
• Samuel, David (2024). BERTs are Generative In-Context Learners. I Globerson, Amir; Mackey, Lester; Belgrave, Danielle; Fan, Angela; Paquet, Ulrich; Tomczak, Jakub & Zhang, Cheng (Red.), Advances in Neural Information Processing Systems 37 (NeurIPS 2024). Curran Associates, Inc.. ISSN 9781713899921. s. 2558–2589. doi: 10.52202/079017-0084. Fulltekst i vitenarkiv Vis sammendrag

  While in-context learning is commonly associated with causal language models, such as GPT, we demonstrate that this capability also 'emerges' in masked language models. Through an embarrassingly simple inference technique, we enable an existing masked model, DeBERTa, to perform generative tasks without additional training or architectural changes. Our evaluation reveals that the masked and causal language models behave very differently, as they clearly outperform each other on different categories of tasks. These complementary strengths suggest that the field's focus on causal models for in-context learning may be limiting – both architectures can develop these capabilities, but with distinct advantages; pointing toward promising hybrid approaches that combine the strengths of both objectives.

• Mæhlum, Petter; Samuel, David; Norman, Rebecka Maria; Jelin, Elma; Bjertnæs, Øyvind Andresen & Øvrelid, Lilja [Vis alle 7 forfattere av denne artikkelen] (2024). It’s Difficult to be Neutral – Human and LLM-based Sentiment Annotation of Patient Comments, Proceedings of the First Workshop on Patient-Oriented Language Processing (CL4Health). ELRA Language Resources Association. ISSN 9782493814258. s. 8–19. Fulltekst i vitenarkiv
• Samuel, David; Charpentier, Lucas Georges Gabriel & Wold, Sondre (2024). More room for language: Investigating the effect of retrieval on language models. I Duh, Kevin; Gomez, Helena & Bethard, Steven (Red.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers). Association for Computational Linguistics (ACL). ISSN 9798891761155. s. 282–305. Fulltekst i vitenarkiv
• Samuel, David & Øvrelid, Lilja (2023). Tokenization with Factorized Subword Encoding, Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics (ACL). ISSN 9781959429623. Fulltekst i vitenarkiv
• Samuel, David; Kutuzov, Andrei; Øvrelid, Lilja & Velldal, Erik (2023). Trained on 100 million words and still in shape: BERT meets British National Corpus. I Vlachos, Andreas & Augenstein, Isabelle (Red.), Findings of the Association for Computational Linguistics: EACL 2023. Association for Computational Linguistics (ACL). ISSN 9781959429470. s. 1954–1974. doi: 10.18653/v1/2023.findings-eacl.146. Fulltekst i vitenarkiv Vis sammendrag

  While modern masked language models (LMs) are trained on ever larger corpora, we here explore the effects of down-scaling training to a modestly-sized but representative, well-balanced, and publicly available English text source – the British National Corpus. We show that pre-training on this carefully curated corpus can reach better performance than the original BERT model. We argue that this type of corpora has great potential as a language modeling benchmark. To showcase this potential, we present fair, reproducible and data-efficient comparative studies of LMs, in which we evaluate several training objectives and model architectures and replicate previous empirical results in a systematic way. We propose an optimized LM architecture called LTG-BERT.

• Charpentier, Lucas Georges Gabriel; Wold, Sondre; Samuel, David & Rønningstad, Egil (2023). BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer. I Alumäe, Tanel & Fishel, Mark (Red.), Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa). University of Tartu. ISSN 9789916219997. Fulltekst i vitenarkiv
• Charpentier, Lucas Georges Gabriel & Samuel, David (2023). Not all layers are equally as important: Every Layer Counts BERT. I Warstadt, Alex; Mueller, Aaron; Choshen, Leshem; Wilcox, Ethan; Zhuang, Chengxu; Ciro, Juan; Mosquera, Rafael; Paranjabe, Bhargavi; Williams, Adina; Linzen, Tal & Cotterell, Ryan (Red.), Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning. Association for Computational Linguistics (ACL). ISSN 9781952148026. s. 238–252. doi: 10.18653/v1/2023.conll-babylm.20. Fulltekst i vitenarkiv
• Samuel, David (2023). Mean BERTs make erratic language teachers: the effectiveness of latent bootstrapping in low-resource settings. I Warstadt, Alex; Mueller, Aaron; Choshen, Leshem; Wilcox, Ethan; Zhuang, Chengxu; Ciro, Juan; Mosquera, Rafael; Paranjabe, Bhargavi; Williams, Adina; Linzen, Tal & Cotterell, Ryan (Red.), Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning. Association for Computational Linguistics (ACL). ISSN 9781952148026. s. 221–237. doi: 10.18653/v1/2023.conll-babylm.19. Fulltekst i vitenarkiv
• Samuel, David; Kutuzov, Andrei; Touileb, Samia; Velldal, Erik; Øvrelid, Lilja & Rønningstad, Egil [Vis alle 8 forfattere av denne artikkelen] (2023). NorBench – A Benchmark for Norwegian Language Models. I Alumäe, Tanel & Fishel, Mark (Red.), Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa). University of Tartu. ISSN 9789916219997. s. 618–633. Fulltekst i vitenarkiv Vis sammendrag

  We present NorBench: a streamlined suite of NLP tasks and probes for evaluating Norwegian language models (LMs) on standardized data splits and evaluation metrics. We also introduce a range of new Norwegian language models (both encoder and encoder-decoder based). Finally, we compare and analyze their performance, along with other existing LMs, across the different benchmark tests of NorBench.

• You, Huiling; Samuel, David; Touileb, Samia & Øvrelid, Lilja (2022). EventGraph: Event Extraction as Semantic Graph Parsing, The 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text. Association for Computational Linguistics (ACL). ISSN 9781959429050. s. 7–15. Fulltekst i vitenarkiv
• You, Huiling; Samuel, David; Touileb, Samia & Øvrelid, Lilja (2022). EventGraph at CASE 2021 Task 1: A General Graph-based Approach to Protest Event Extraction, The 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text. Association for Computational Linguistics (ACL). ISSN 9781959429050. s. 155–160. Fulltekst i vitenarkiv
• Samuel, David; Barnes, Jeremy; Kurtz, Robin; Oepen, Stephan; Øvrelid, Lilja & Velldal, Erik (2022). Direct parsing to sentiment graphs. Association for Computational Linguistics (ACL). Annual Meeting Conference Proceedings. ISSN 0736-587X. 2(60), s. 470–478. doi: 10.18653/v1/2022.acl-short.51. Fulltekst i vitenarkiv
• Samuel, David & Straka, Milan (2021). ÚFAL at MultiLexNorm 2021: Improving Multilingual Lexical Normalization by Fine-tuning ByT5, Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021). Association for Computational Linguistics (ACL). ISSN 9781954085909. s. 483–492. Fulltekst i vitenarkiv Vis sammendrag

  We present the winning entry to the Multilingual Lexical Normalization (MultiLexNorm) shared task at W-NUT 2021 (van der Goot et al., 2021a), which evaluates lexical-normalization systems on 12 social media datasets in 11 languages. We base our solution on a pre-trained byte-level language model, ByT5 (Xue et al., 2021a), which we further pre-train on synthetic data and then fine-tune on authentic normalization data. Our system achieves the best performance by a wide margin in intrinsic evaluation, and also the best performance in extrinsic evaluation through dependency parsing. The source code is released at https://github.com/ufal/multilexnorm2021 and the fine-tuned models at https://huggingface.co/ufal.

• Samuel, David & Straka, Milan (2020). ÚFAL at MRP 2020: Permutation-invariant Semantic Parsing in PERIN. I Oepen, Stephan; Abend, Omri; Abzianidze, Lasha; Bos, Johan; Hajic, Jan; Hershcovich, Daniel; Li, Bin; O'Gorman, Tim & Zeman, Daniel (Red.), Proceedings of the CoNLL 2020 Shared Task:Cross-Framework Meaning Representation Parsing. Association for Computational Linguistics (ACL). ISSN 9781952148644. s. 53–64. doi: 10.18653/v1/2020.conll-shared.5. Fulltekst i vitenarkiv Vis sammendrag

  We present PERIN, a novel permutation-invariant approach to sentence-to-graph semantic parsing. PERIN is a versatile, cross-framework and language independent architecture for universal modeling of semantic structures. Our system participated in the CoNLL 2020 shared task, Cross-Framework Meaning Representation Parsing (MRP 2020), where it was evaluated on five different frameworks (AMR, DRG, EDS, PTG and UCCA) across four languages. PERIN was one of the winners of the shared task. The source code and pretrained models are available at http://www.github.com/ufal/perin.

Se alle arbeider i NVA

• Samuel, David; Kutuzov, Andrey; Øvrelid, Lilja & Velldal, Erik (2026). Data-Constrained Language Modeling. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Language model performance scales with training data, yet the reservoir of human-written text is finite– a constraint that already limits most languages and will eventually affect even English as models continue to scale. This thesis addresses data constraints across the entire life cycle of language models: we develop architectural improvements and training methods that maximize learning from limited and repeated data, and we propose post-training methods that enable fluency-preserving preference optimization without native instruction-tuning datasets.

Se alle arbeider i NVA