Petter Mæhlum

• Norman, Rebecka Maria; Storset, Lilja Charlotte; Mæhlum, Petter; Iversen, Hilde Karin Hestad; Jelin, Elma & Velldal, Erik [Show all 8 contributors for this article] (2026). Exploiting survey and online patient experience comments in general practice: validating a fine-tuned language model for automatic sentiment analysis. Family Practice. ISSN 0263-2136. 43(5). doi: 10.1093/fampra/cmag071. Full text in Research Archive
• Mæhlum, Petter; Evensen, Anders Næss & Scherrer, Yves (2025). Improved Norwegian Bokmål Translations for FLORES. In Haddow, Barry; Kocmi, Tom; Koehn, Philipp & Monz, Christof (Ed.), Proceedings of the Tenth Conference on Machine Translation. Association for Computational Linguistics (ACL). ISSN 9798891763418. p. 1124–1132. doi: 10.18653/v1/2025.wmt-1.86. Full text in Research Archive Show summary

  FLORES+ is a collection of parallel datasets obtained by translation from originally English source texts. FLORES+ contains Norwegian translations for the two official written variants of Norwegian: Norwegian Bokmål and Norwegian Nynorsk. However, the earliest Bokmål version contained non-native-like mistakes, and even after a later revision, the dataset contained grammatical and lexical errors. This paper aims at correcting unambiguous mistakes, and thus creating a new version of the Bokmål dataset. At the same time, we provide a translation into Radical Bokmål, a sub-variety of Norwegian which is closer to Nynorsk in some aspects, while still being within the official norms for Bokmål. We discuss existing errors and differences in the various translations and the corrections that we provide.

• Jelin, Elma; Storset, Lilja Charlotte; Norman, Rebecka Maria; Iversen, Hilde Karin Hestad; Ellingsen-Dalskau, Lina Harvold & Mæhlum, Petter [Show all 9 contributors for this article] (2025). From words to action? A scoping review on automatic sentiment analysis of patient experience comments from online sources and surveys. BMJ Health & Care Informatics. 32(1). doi: 10.1136/bmjhci-2025-101631. Full text in Research Archive Show summary

  Background Automatic analysis of free-text patient comments enables the efficient processing of large feedback volumes, reducing reliance on manual review. A 2021 review examined natural language processing (NLP) and sentiment analysis (SA) in patient experience research; however, recent advances in deep learning and generative artificial intelligence (AI) call for an updated synthesis. Objectives This scoping review aims to map and summarise recent studies applying SA to unstructured patient experience data related to healthcare services. Methods Following Joanna Briggs Institute methodology and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines, we conducted a comprehensive search across Medline, CINAHL, Web of Science, Cochrane, Embase and APA PsycINFO. We included studies published from January 2020 to March 2024 in English or a Scandinavian language. Eligible studies analysed patient feedback using NLP techniques and described the development or validation of SA models. Two reviewers independently screened the studies and extracted data, which were presented in tables both tabular and narrative forms. Results 30 studies were included, primarily from the USA, Europe and Asia. Patient comments were mostly sourced from online platforms such as social media. Feedback largely concerned hospital care. 18 studies employed rule-based SA approaches, while 12 applied supervised machine learning (ML) and only 4 studies used deep learning models. Few addressed the visualisation or practical applications in healthcare. Conclusion Despite significant progress, modern methods like deep learning and generative AI remain underused in SA of patient-experience data. Limited focus on implementation restricts SA’s role in quality improvement. Future research should assess advanced methods and their cost-effectiveness versus traditional ML.

• Scherrer, Yves; Goot, Rob van der & Mæhlum, Petter (2025). Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification. In Scherrer, Yves; Jauhiainen, Tommi; Ljubešić, Nikola; Nakov, Preslav; Tiedemann, Jörg & Zampieri, Marcos (Ed.), Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects. Association for Computational Linguistics (ACL). ISSN 9798891762084. p. 1–8. Full text in Research Archive
• Fedorova, Mariia; Frydenberg, Jonas Sebulon; Handford, Victoria; Langø, Victoria Ovedie Chruickshank; Willoch, Solveig Helene & Midtgaard, Marthe Løken [Show all 9 contributors for this article] (2025). Multi-label Scandinavian Language Identification (SLIDE). In Holdt, Špela Arhar; Ilinykh, Nikolai; Scalvini, Barbara; Bruton, Micaella; Debess, Iben Nyholm & Tudor, Crina Madalina (Ed.), Proceedings of the Third Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2025). Tartu Ülikool Kirjastus. ISSN 9789908531212. p. 179–189. Full text in Research Archive Show summary

  Identifying closely related languages at sentence level is difficult, in particular because it is often impossible to assign a sentence to a single language. In this paper, we focus on multi-label sentence-level Scandinavian language identification (LID) for Danish, Norwegian Bokmål, Norwegian Nynorsk, and Swedish. We present the Scandinavian Language Identification and Evaluation, SLIDE, a manually curated multi-label evaluation dataset and a suite of LID models with varying speed–accuracy tradeoffs. We demonstrate that the ability to identify multiple languages simultaneously is necessary for any accurate LID method, and present a novel approach to training such multi-label LID models.

• Midtgaard, Marthe Løken; Mæhlum, Petter & Scherrer, Yves (2025). LTG at VarDial 2025 NorSID: More and Better Training Data for Slot and Intent Detection. In Scherrer, Yves; Jauhiainen, Tommi; Ljubešić, Nikola; Nakov, Preslav; Tiedemann, Jörg & Zampieri, Marcos (Ed.), Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects. Association for Computational Linguistics (ACL). ISSN 9798891762084. p. 200–208. Full text in Research Archive Show summary

  This paper describes the LTG submission to the VarDial 2025 shared task, where we participate in the Norwegian slot and intent detection subtasks. The shared task focuses on Norwegian dialects, which present challenges due to their low-resource nature and variation. We test a variety of neural models and training data configurations, with the focus on improving and extending the available Norwegian training data. This includes automatically re-aligning slot spans in Norwegian Bokmål, as well as re-translating the original English training data into both Bokmål and Nynorsk. % to address dialectal diversity. We also re-annotate an external Norwegian dataset to augment the training data. Our best models achieve first place in both subtasks, achieving an span F1 score of 0.893 for slot filling and an accuracy of 0.980 for intent detection. Our results indicate that while translation quality is less critical, improving the slot labels has a notable impact on slot performance. Moreover, adding more standard Norwegian data improves performance, but incorporating even small amounts of dialectal data leads to greater gains.

• Storset, Lilja Charlotte; Rønningstad, Egil; Mæhlum, Petter; Velldal, Erik & Øvrelid, Lilja (2025). Mixed Feelings: Cross-Domain Sentiment Classification of Patient Feedback. In Johansson, Richard & Stymne, Sara (Ed.), Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). University of Tartu Library. ISSN 9789908531090. p. 537–543. doi: https:/aclanthology.org/2025.nodalida-1.58/. Full text in Research Archive Show summary

  Sentiment analysis of patient feedback from the public health domain can aid decision makers in evaluating the provided services. The current paper focuses on free-text comments in patient surveys about general practitioners and psychiatric healthcare, annotated with four sentence-level polarity classes - positive, negative, mixed and neutral - while also attempting to alleviate data scarcity by leveraging general-domain sources in the form of reviews. For several different architectures, we compare in-domain and out-of-domain effects, as well as the effects of training joint multi-domain models.

• Mikhailov, Vladislav; Mæhlum, Petter; Langø, Victoria; Velldal, Erik & Øvrelid, Lilja (2025). A Collection of Question Answering Datasets for Norwegian. In Johansson, Richard & Stymne, Sara (Ed.), Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). University of Tartu Library. ISSN 9789908531090. p. 397–407. doi: https:/aclanthology.org/2025.nodalida-1.43/. Full text in Research Archive Show summary

  This paper introduces a new suite of question answering datasets for Norwegian; NorOpenBookQA, NorCommonSenseQA, NorTruthfulQA, and NRK-Quiz-QA. The data covers a wide range of skills and knowledge domains, including world knowledge, commonsense reasoning, truthfulness, and knowledge about Norway. Covering both of the written standards of Norwegian – Bokmål and Nynorsk – our datasets comprise over 10k question-answer pairs, created by native speakers. We detail our dataset creation approach and present the results of evaluating 11 language models (LMs) in zero- and few-shot regimes. Most LMs perform better in Bokmål than Nynorsk, struggle most with commonsense reasoning, and are often untruthful in generating answers to questions. All our datasets and annotation materials are publicly available.

• Burchell, Laurie; Gibert, Ona de; Arefev, Nikolay; Aulamo, Mikko; Bañón, Marta & Chen, Pinzhen [Show all 35 contributors for this article] (2025). An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT). In Che, Wanxiang; Nabende, Joyce; Shutova, Ekaterina & Pilehvar, Mohammad (Ed.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics (ACL). ISSN 9798891762510. p. 17452–17485. doi: 10.18653/v1/2025.acl-long.854. Full text in Research Archive Show summary

  Training state-of-the-art large language models requires vast amounts of clean and diverse textual data. However, building suitable multilingual datasets remains a challenge. In this work, we present HPLT v2, a collection of high-quality multilingual monolingual and parallel corpora, extending prior work of the HPLT project. The monolingual portion of the data contains 8T tokens covering 193 languages, while the parallel data contains 380M sentence pairs covering 51 languages. We document the entire data pipeline and release the code to reproduce it. We provide extensive analysis of the quality and characteristics of our data. Finally, we evaluate the performance of language models and machine translation systems trained on HPLT v2, demonstrating its value.

• Rosa, Javier De La; Mikhailov, Vladislav; Zhang, Lemei; Wetjen, Freddy; Samuel, David & Liu, Peng [Show all 19 contributors for this article] (2025). The Impact of Copyrighted Material on Large Language Models: A Norwegian Perspective. In Johansson, Richard & Stymne, Sara (Ed.), Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). University of Tartu Library. ISSN 9789908531090. p. 544–560. Full text in Research Archive
• Mæhlum, Petter; Samuel, David; Norman, Rebecka Maria; Jelin, Elma; Bjertnæs, Øyvind Andresen & Øvrelid, Lilja [Show all 7 contributors for this article] (2024). It’s Difficult to be Neutral – Human and LLM-based Sentiment Annotation of Patient Comments, Proceedings of the First Workshop on Patient-Oriented Language Processing (CL4Health). ELRA Language Resources Association. ISSN 9782493814258. p. 8–19. Full text in Research Archive
• Mæhlum, Petter & Scherrer, Yves (2024). NoMusic - The Norwegian Multi-Dialectal Slot and Intent Detection Corpus. In Scherrer, Yves; Jauhiainen, Tommi; Ljubešić, Nikola; Zampieri, Marcos; Nakov, Preslav & Tiedemann, Jörg (Ed.), Proceedings of the Eleventh Workshop on NLP for Similar Languages, Varieties, and Dialects (VarDial 2024). Association for Computational Linguistics (ACL). ISSN 9798891761049. doi: 10.18653/v1/2024.vardial-1.9. Full text in Research Archive
• Wold, Sondre; Mæhlum, Petter & Hove, Oddbjørn (2024). Estimating Lexical Complexity from Document-Level Distributions. In Calzolari, Nicoletta; Kan, Min-Yen; Hoste, Veronique; Lenci, Alessandro; Sakti, Sakriani & Xue, Nianwen (Ed.), Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). European Language Resources Association. ISSN 9782493814104. p. 6309–6318. Full text in Research Archive
• Touileb, Samia; Murstad, Jeanett; Mæhlum, Petter; Steskal, Lubos; Storset, Lilja Charlotte & You, Huiling [Show all 7 contributors for this article] (2024). EDEN: A Dataset for Event Detection in Norwegian News. In Calzolari, Nicoletta; Kan, Min-Yen; Hoste, Veronique; Lenci, Alessandro; Sakti, Sakriani & Xue, Nianwen (Ed.), Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). European Language Resources Association. ISSN 9782493814104. p. 5495–5506. Full text in Research Archive Show summary

  We present EDEN, the first Norwegian dataset annotated with event information at the sentence level, adapting the widely used ACE event schema to Norwegian. The paper describes the manual annotation of Norwegian text as well as transcribed speech in the news domain, together with inter-annotator agreement and discussions of relevant dataset statistics. We also present preliminary modeling results using a graph-based event parser. The resulting dataset will be made freely available for download and use.

• Mæhlum, Petter & Ivanova, Sardana (2023). Phonotactics as an Aid in Low Resource Loan Word Detection and Morphological Analysis in Sakha. In Ilinykh, Nikolai; Morger, Felix; Dannélls, Dana; Dobnik, Simon; Megyesi, Beáta & Nivre, Joakim (Ed.), Proceedings of the Second Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2023). Association for Computational Linguistics (ACL). ISSN 9781959429739. p. 111–120. Full text in Research Archive Show summary

  Obtaining information about loan words and irregular morphological patterns can be difficult for low-resource languages. Using Sakha as an example, we show that it is possible to exploit known phonemic regularities such as vowel harmony and consonant distributions to identify loan words and irregular patterns, which can be helpful in rule-based downstream tasks such as parsing and POS-tagging. We evaluate phonemically inspired methods for loanword detection, combined with bi-gram vowel transition probabilities to inspect irregularities in the morphology of loanwords. We show that both these techniques can be useful for the detection of such patterns. Finally, we inspect the plural suffix -ЛАр [-LAr] to observe some of the variation in morphology between native and foreign words.

• Barnes, Jeremy Claude; Touileb, Samia; Mæhlum, Petter & Lison, Pierre (2023). Identifying Token-Level Dialectal Features in Social Media. In Alumäe, Tanel & Fishel, Mark (Ed.), Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa). University of Tartu. ISSN 9789916219997. Full text in Research Archive Show summary

  Dialectal variation is present in many human languages and is attracting a growing interest in NLP. Most previous work concentrated on either (1) classifying dialectal varieties at the document or sentence level or (2) performing standard NLP tasks on dialectal data. In this paper, we propose the novel task of token-level dialectal feature prediction. We present a set of fine-grained annotation guidelines for Norwegian dialects, expand a corpus of dialectal tweets, and manually annotate them using the introduced guidelines. Furthermore, to evaluate the learnability of our task, we conduct labeling experiments using a collection of baselines, weakly supervised and supervised sequence labeling models. The obtained results show that, despite the difficulty of the task and the scarcity of training data, many dialectal features can be predicted with reasonably high accuracy.

• Mæhlum, Petter; Velldal, Erik & Øvrelid, Lilja (2023). A Diagnostic Dataset for Sentiment and Negation Modeling for Norwegian. In Ilinykh, Nikolai; Morger, Felix; Dannélls, Dana; Dobnik, Simon; Megyesi, Beáta & Nivre, Joakim (Ed.), Proceedings of the Second Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2023). Association for Computational Linguistics (ACL). ISSN 9781959429739. Full text in Research Archive Show summary

  Negation constitutes a challenging phenomenon for many natural language processing tasks, such as sentiment analysis (SA). In this paper we investigate the relationship between negation and sentiment in the context of Norwegian professional reviews. The first part of this paper includes a corpus study which investigates how negation is tied to sentiment in this domain, based on existing annotations. In the second part, we introduce NoReC-NegSynt, a synthetically augmented test set for negation and sentiment, to allow for a more detailed analysis of the role of negation in current neural SA models. This diagnostic test set, containing both clausal and non-clausal negation, allows for analyzing and comparing models’ abilities to treat several different types of negation. We also present a case-study, applying several neural SA models to the diagnostic data.

• Mæhlum, Petter; Kåsen, Andre; Touileb, Samia & Barnes, Jeremy (2022). Annotating Norwegian language varieties on Twitter for Part-of-speech. In Scherrer, Yves; Jauhiainen, Tommi; ́c, Nikola Ljubeši; Nakov, Preslav; Tiedemann, Jörg & Zampieri, Marcos (Ed.), Proceedings of the Ninth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2022). Association for Computational Linguistics (ACL). p. 64–69. Full text in Research Archive
• Mæhlum, Petter; Haug, Dag Trygve Truslew; Jørgensen, Tollef Emil; Kåsen, Andre; Nøklestad, Anders & Rønningstad, Egil [Show all 9 contributors for this article] (2022). NARC – Norwegian Anaphora Resolution Corpus. International Conference on Computational Linguistics (ICCL) (COLING). ISSN 1525-2477. 29(7), p. 48–60. Full text in Research Archive Show summary

  Published in: Proceedings of the Fifth Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC): https://aclanthology.org/venues/coling/. We present the Norwegian Anaphora Resolution Corpus (NARC), the first publicly available corpus annotated with anaphoric relations between noun phrases for Norwegian. The paper describes the annotated data for 326 documents in Norwegian Bokmål, together with inter-annotator agreement and discussions of relevant statistics. We also present preliminary modelling results which are comparable to existing corpora for other languages, and discuss relevant problems in relation to both modelling and the annotations themselves.

• Kutuzov, Andrei; Touileb, Samia; Mæhlum, Petter; Enstad, Tita & Witteman, Alexandra (2022). NorDiaChange: Diachronic Semantic Change Dataset for Norwegian, Proceedings of the Language Resources and Evaluation Conference. European Language Resources Association. ISSN 9791095546726. p. 2563–2572. Full text in Research Archive Show summary

  We describe NorDiaChange: the first diachronic semantic change dataset for Norwegian. NorDiaChange comprises two novel subsets, covering about 80 Norwegian nouns manually annotated with graded semantic change over time. Both datasets follow the same annotation procedure and can be used interchangeably as train and test splits for each other. NorDiaChange covers the time periods related to pre- and post-war events, oil and gas discovery in Norway, and technological developments. The annotation was done using the DURel framework and two large historical Norwegian corpora. NorDiaChange is published in full under a permissive licence, complete with raw annotation data and inferred diachronic word usage graphs (DWUGs).

• Barnes, Jeremy; Mæhlum, Petter & Touileb, Samia (2021). NorDial: A Preliminary Corpus of Written Norwegian Dialect Use. In Dobnik, Simon & Øvrelid, Lilja (Ed.), Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa). Linköping University Electronic Press. ISSN 9789179296148. p. 445–451. Full text in Research Archive Show summary

  Norway has a large amount of dialectal variation, as well as a general tolerance to its use in the public sphere. There are, however, few available resources to study this variation and its change over time and in more informal areas, on social media. In this paper, we propose a first step to creating a corpus of dialectal variation of written Norwegian. We collect a small corpus of tweets and manually annotate them as Bokmål, Nynorsk, any dialect, or a mix. We further perform preliminary experiments with state-of-the-art models, as well as an analysis of the data to expand this corpus in the future. Finally, we make the annotations available for future work.

• Mæhlum, Petter; Barnes, Jeremy; Kurtz, Robin; Øvrelid, Lilja & Velldal, Erik (2021). Negation in Norwegian: an annotated dataset. In Dobnik, Simon & Øvrelid, Lilja (Ed.), Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa). Linköping University Electronic Press. ISSN 9789179296148. p. 299–308. Full text in Research Archive
• Øvrelid, Lilja; Mæhlum, Petter; Barnes, Jeremy & Velldal, Erik (2020). A Fine-Grained Sentiment Dataset for Norwegian. In Calzolari, Nicoletta; Béchet, Frédéric; Blache, Philippe; Choukri, Khalid; Cieri, Christopher; Declerck, Thierry; Goggi, Sara; Isahara, Hitoshi; Maegaard, Bente; Mariani, Joseph; Mazo, Hélène; Moreno, Asuncion; Odijk, Jan & Piperidis, Stelios (Ed.), Proceedings of The 12th Language Resources and Evaluation Conference. European Language Resources Association. ISSN 9791095546344. Full text in Research Archive
• Mæhlum, Petter; Barnes, Jeremy Claude; Øvrelid, Lilja & Velldal, Erik (2019). Annotating evaluative sentences for sentiment analysis: a dataset for Norwegian. In Hartmann, Mareike & Plank, Barbara (Ed.), Proceedings of the 22nd Nordic Conference on Computational Linguistics (NoDaLiDa). Linköping University Electronic Press. ISSN 9789179299958. p. 121–130. Full text in Research Archive Show summary

  This paper documents the creation of a large-scale dataset of evaluative sentences – i.e. both subjective and objective sentences that are found to be sentiment-bearing – based on mixed-domain professional reviews from various news-sources. We present both the annotation scheme and first results for classification experiments. The effort represents a step toward creating a Norwegian dataset for fine-grained sentiment analysis.

View all works in NVA

• Jelin, Elma; Bjertnæs, Øyvind Andresen; Norman, Rebecka Maria; Iversen, Hilde Karin Hestad & Mæhlum, Petter (2024). Mapping the Landscape: A Scoping Review on Advancing Patient Experience Feedback Through Natural Language Processing (NLP) and Machine Learning (ML), including sentiment Analysis (SA). Full text in Research Archive
• Norman, Rebecka Maria; Jelin, Elma; Mæhlum, Petter; Velldal, Erik; Øvrelid, Lilja & Bjertnæs, Øyvind Andresen (2024). From words to action: development of a machine learning model using natural language processing to automatically turn free-text patient comments into insights for quality improvement. Full text in Research Archive
• Wold, Sondre & Mæhlum, Petter (2024). Nesten 60 år etter LIX -- Lesbarhet på ordnivå. Full text in Research Archive
• Mæhlum, Petter & Scherrer, Yves (2024). NoMusic – Et parallellkorpus for skriftlig dialekt. Full text in Research Archive
• Mæhlum, Petter & Eivind, Torgersen (2022). Det er mange måter å si nei på – her er noen av de fineste. [Internet]. titan.no. Full text in Research Archive
• Eivind, Torgersen & Mæhlum, Petter (2022). Ordet «ikke» skaper trøbbel for kunstig intelligens. [Internet]. forskning.no. Full text in Research Archive

View all works in NVA