Vladislav Mikhailov

• Oepen, Stephan; Arefyev, Nikolay; Aulamo, Mikko; Bañón, Marta; Buljan, Maja & Burchell, Laurie V. [Vis alle 29 forfattere av denne artikkelen] (2026). HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models. I Piperidis,, Stelios; Bel, Núria; Heuvel, Henk van den; Ide, Nancy; Krek, Simon & Toral, Antonio (Red.), Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026). European Language Resources Association. ISSN 9782493814494. s. 1409–1434. doi: 10.63317/25xbdofco9od. Fulltekst i vitenarkiv Vis sammendrag

  We present an ongoing initiative to provide open, very large, high-quality, and richly annotated textual datasets for almost 200 languages. At 30 trillion tokens, this is likely the largest generally available multilingual collection of LLM pre-training data. These datasets are derived from web crawls from different sources and accompanied with a complete, open-source pipeline for document selection from web archives, text extraction from HTML, language identification for noisy texts, exact and near-deduplication, annotation with, among others, register labels, text quality estimates, and personally identifiable information; and final selection and filtering. We report on data quality probes through contrastive and analytical statistics, through manual inspection of samples for some 20 languages, and through end-to-end evaluation of various language model architectures trained on this data. For multilingual LLM evaluation, we provide a comprehensive collection of benchmarks for nine European languages, with special emphasis on natively created tasks, mechanisms to mitigate prompt sensitivity, and refined normalization and aggregation of scores. Additionally, we train and evaluate a family of 57 monolingual encoder–decoder models, as well as about 30 “smallish” monolingual GPT-like reference models. Besides the monolingual data and models, we also present a very large collection of parallel texts automatically mined from this data, together with a novel parallel corpus synthesized via machine translation.

• Samuel, David; Mikhailov, Vladislav; Velldal, Erik; Øvrelid, Lilja; Charpentier, Lucas Georges Gabriel & Kutuzov, Andrey [Vis alle 7 forfattere av denne artikkelen] (2025). Small Languages, Big Models: A Study of Continual Training on Languages of Norway. I Johansson, Richard & Stymne, Sara (Red.), Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). University of Tartu Library. ISSN 9789908531090. s. 573–608. doi: https:/aclanthology.org/2025.nodalida-1.61/. Fulltekst i vitenarkiv Vis sammendrag

  Training large language models requires vast amounts of data, posing a challenge for less widely spoken languages like Norwegian and even more so for truly low-resource languages like Northern S{\'a}mi. To address this issue, we present a novel three-stage continual training approach that substantially improves the downstream performance together with the inference efficiency for the target languages. Based on our findings, we train, evaluate, and openly release a new generative language model for Norwegian Bokm{\r{a}}l, Nynorsk, and Northern S{\'a}mi with 11.4 billion parameters: NorMistral-11B.

• Touileb, Samia; Mikhailov, Vladislav; Kroka, Marie Ingeborg; Øvrelid, Lilja & Velldal, Erik (2025). Benchmarking Abstractive Summarisation: A Dataset of Human-authored Summaries of Norwegian News Articles. I Johansson, Richard & Stymne, Sara (Red.), Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). University of Tartu Library. ISSN 9789908531090. s. 729–738. doi: https:/aclanthology.org/2025.nodalida-1.73/. Fulltekst i vitenarkiv Vis sammendrag

  We introduce a dataset of high-quality human-authored summaries of news articles in Norwegian. The dataset is intended for benchmarking of the abstractive summarisation capabilities of generative language models. Each document in the dataset is provided with three different candidate gold-standard summaries written by native Norwegian speakers and all summaries are provided in both of the written variants of Norwegian – Bokmål and Nynorsk. The paper describes details on the data creation effort as well as an evaluation of existing open LLMs for Norwegian on the dataset. We also provide insights from a manual human evaluation, comparing human-authored to model generated summaries. Our results indicate that the dataset provides a challenging LLM benchmark for Norwegian summarisation capabilities.

• Mikhailov, Vladislav; Artemova, Ekaterina; Butenko, Zoia; Øvrelid, Lilja & Velldal, Erik (2025). Overview of the Preference Prediction Task at the ELOQUENT 2025 lab for evaluating generative language model quality. CEUR Workshop Proceedings. 4038, s. 1320–1329. doi: https:/ceur-ws.org/Vol-4038/paper_105.pdf. Fulltekst i vitenarkiv Vis sammendrag

  The Preference Prediction shared task, part of the ELOQUENT Lab at CLEF-2025, challenges participants to build systems that simulate human judgment in evaluating pairs of LLM-generated responses. This novel task includes two sub-tasks. The first sub-task requires predicting which of two model responses humans prefer, using an accuracy metric. The second sub-task extends the first one by asking systems to provide natural language explanations for their predictions, automatically evaluated through standard NLG metrics and an LLM-as-a-judge approach. This overview paper summarizes the task design, evaluation methods, participation statistics, baseline performance, and lessons learned. We discuss the system performance in both sub-tasks. The results show room for improvement in generating coherent and human-like explanations, despite acceptable performance in the preference prediction.

• Karlgren, Jussi; Artemova, Ekaterina; Bojar, Ondřej; Engels, Marie Isabel; Mikhailov, Vladislav & Šindelář, Pavel [Vis alle 8 forfattere av denne artikkelen] (2025). Overview of ELOQUENT 2025: Shared Tasks for Evaluating Generative Language Model Quality. I Carrillo-de-Albornoz, Jorge (Red.), Experimental IR Meets Multilinguality, Multimodality, and Interaction (CLEF 2025).. Springer Publishing Company. ISSN 9783032043542. doi: 10.1007/978-3-032-04354-2_14. Fulltekst i vitenarkiv
• Mikhailov, Vladislav; Enstad, Tita Ranveig; Samuel, David; Farsethås, Hans Christian; Velldal, Erik & Øvrelid, Lilja [Vis alle 7 forfattere av denne artikkelen] (2025). NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark. I Che, Wanxiang; Nabende, Joyce; Shutova, Ekaterina & Pilehvar, Mohammad (Red.), Findings of the Association for Computational Linguistics: ACL 2025. Association for Computational Linguistics (ACL). ISSN 9798891762565. s. 3495–3541. doi: 10.18653/v1/2025.findings-acl.181. Fulltekst i vitenarkiv Vis sammendrag

  This paper introduces NorEval, a new and comprehensive evaluation suite for large-scale standardized benchmarking of Norwegian generative language models (LMs). NorEval consists of 24 high-quality human-created datasets – of which five are created from scratch. In contrast to existing benchmarks for Norwegian, NorEval covers a broad spectrum of task categories targeting Norwegian language understanding and generation, establishes human baselines, and focuses on both of the official written standards of the Norwegian language: Bokmål and Nynorsk. All our datasets and a collection of over 100 human-created prompts are integrated into LM Evaluation Harness, ensuring flexible and reproducible evaluation. We describe the NorEval design and present the results of benchmarking 19 open-source pretrained and instruction-tuned LMs for Norwegian in various scenarios. Our benchmark, evaluation framework, and annotation materials are publicly available.

• Mikhailov, Vladislav; Mæhlum, Petter; Langø, Victoria; Velldal, Erik & Øvrelid, Lilja (2025). A Collection of Question Answering Datasets for Norwegian. I Johansson, Richard & Stymne, Sara (Red.), Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). University of Tartu Library. ISSN 9789908531090. s. 397–407. doi: https:/aclanthology.org/2025.nodalida-1.43/. Fulltekst i vitenarkiv Vis sammendrag

  This paper introduces a new suite of question answering datasets for Norwegian; NorOpenBookQA, NorCommonSenseQA, NorTruthfulQA, and NRK-Quiz-QA. The data covers a wide range of skills and knowledge domains, including world knowledge, commonsense reasoning, truthfulness, and knowledge about Norway. Covering both of the written standards of Norwegian – Bokmål and Nynorsk – our datasets comprise over 10k question-answer pairs, created by native speakers. We detail our dataset creation approach and present the results of evaluating 11 language models (LMs) in zero- and few-shot regimes. Most LMs perform better in Bokmål than Nynorsk, struggle most with commonsense reasoning, and are often untruthful in generating answers to questions. All our datasets and annotation materials are publicly available.

• Burchell, Laurie; Gibert, Ona de; Arefev, Nikolay; Aulamo, Mikko; Bañón, Marta & Chen, Pinzhen [Vis alle 35 forfattere av denne artikkelen] (2025). An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT). I Che, Wanxiang; Nabende, Joyce; Shutova, Ekaterina & Pilehvar, Mohammad (Red.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics (ACL). ISSN 9798891762510. s. 17452–17485. doi: 10.18653/v1/2025.acl-long.854. Fulltekst i vitenarkiv Vis sammendrag

  Training state-of-the-art large language models requires vast amounts of clean and diverse textual data. However, building suitable multilingual datasets remains a challenge. In this work, we present HPLT v2, a collection of high-quality multilingual monolingual and parallel corpora, extending prior work of the HPLT project. The monolingual portion of the data contains 8T tokens covering 193 languages, while the parallel data contains 380M sentence pairs covering 51 languages. We document the entire data pipeline and release the code to reproduce it. We provide extensive analysis of the quality and characteristics of our data. Finally, we evaluate the performance of language models and machine translation systems trained on HPLT v2, demonstrating its value.

• Rosa, Javier De La; Mikhailov, Vladislav; Zhang, Lemei; Wetjen, Freddy; Samuel, David & Liu, Peng [Vis alle 19 forfattere av denne artikkelen] (2025). The Impact of Copyrighted Material on Large Language Models: A Norwegian Perspective. I Johansson, Richard & Stymne, Sara (Red.), Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). University of Tartu Library. ISSN 9789908531090. s. 544–560. Fulltekst i vitenarkiv

Se alle arbeider i NVA