Egil Rønningstad

• Rønningstad, Egil; Klinger, Roman; Øvrelid, Lilja & Velldal, Erik (2026). Entity-Level Sentiment Analysis with Sentence Relevance Detection. In Piperidis,, Stelios; Bel, Núria; Heuvel, Henk van den; Ide, Nancy; Krek, Simon & Toral, Antonio (Ed.), Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026). European Language Resources Association. ISSN 9782493814494. p. 8040–8055. doi: 10.63317/35ideqx4jk89. Full text in Research Archive Show summary

  The task of entity-level sentiment analysis (Elsa) is to extract sentiment scores for a given entity (such as person names or organization names) from a text. Elsa is a challenging task and involves processing of longer documents, where several entities may be mentioned with varying importance for the final score aggregation. Fine-tuning encoder-based Transformers (such as BERT) constitutes the state of the art for sentiment predictions, however, these models are still limited by their restricted input lengths. Decoder-only models so far still underperform on the task. We approach the context limitation by learning to extract segments that are relevant for the sentiment prediction for a given entity, without preprocessing by chunking and aggregation. For decoder models, we explore fine-tuning these through supervised fine-tuning and pairwise comparison, a method borrowed from reward modeling for preference optimization. Both methods perform well and set a new standard for the Elsa task. We further show that pairwise classification is faster, simpler, and shows less variance than the more common direct supervision for this task.

• Fedorova, Mariia; Arefyev, Nikolay; Buljan, Maja; Helcl, Jindřich; Oepen, Stephan & Rønningstad, Egil [Show all 7 contributors for this article] (2026). OpenLID-v3: Improving the Precision of Closely Related Language Identification – An Experience Report. In Scherrer, Yves; Aepli, Noëmi; Blaschke, Verena; Jauhiainen, Tommi; Ljubešić, Nikola; Nakov, Preslav; Tiedemann, Jörg & Zampieri, Marcos (Ed.), Proceedings of the 13th Workshop on NLP for Similar Languages, Varieties and Dialects. Association for Computational Linguistics (ACL). ISSN 9798891763722. p. 275–292. doi: 10.18653/v1/2026.vardial-1.23. Full text in Research Archive Show summary

  Language identification (LID) is an essential step in building high-quality multilingual datasets from web data. Existing LID tools (such as OpenLID or GlotLID) often struggle to identify closely related languages and to distinguish valid natural language from noise, which contaminates language-specific subsets, especially for low-resource languages. In this work we extend the OpenLID classifier by adding more training data, merging problematic language variant clusters, and introducing a special label for marking noise. We call this extended system OpenLID-v3 and evaluate it against GlotLID on multiple benchmarks. During the development we focus on three groups of closely related languages (Bosnian, Croatian, and Serbian; Romance varieties of Northern Italy and Southern France; and Scandinavian languages) and contribute new evaluation datasets where existing ones are inadequate. We find that ensemble approaches improve precision but also substantially reduce coverage for low-resource languages.

• Storset, Lilja Charlotte; Rønningstad, Egil; Mæhlum, Petter; Velldal, Erik & Øvrelid, Lilja (2025). Mixed Feelings: Cross-Domain Sentiment Classification of Patient Feedback. In Johansson, Richard & Stymne, Sara (Ed.), Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025). University of Tartu Library. ISSN 9789908531090. p. 537–543. doi: https:/aclanthology.org/2025.nodalida-1.58/. Full text in Research Archive Show summary

  Sentiment analysis of patient feedback from the public health domain can aid decision makers in evaluating the provided services. The current paper focuses on free-text comments in patient surveys about general practitioners and psychiatric healthcare, annotated with four sentence-level polarity classes - positive, negative, mixed and neutral - while also attempting to alleviate data scarcity by leveraging general-domain sources in the form of reviews. For several different architectures, we compare in-domain and out-of-domain effects, as well as the effects of training joint multi-domain models.

• Rønningstad, Egil; Velldal, Erik & Øvrelid, Lilja (2024). A GPT among Annotators: LLM-based Entity-Level Sentiment Annotation. In Henning, Sophie & Stede, Manfred (Ed.), Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII). Association for Computational Linguistics (ACL). ISSN 9798891760738. p. 133–139. Full text in Research Archive Show summary

  We investigate annotator variation for the novel task of Entity-Level Sentiment Analysis (ELSA) which annotates the aggregated sentiment directed towards volitional entities in a text. More specifically, we analyze the annotations of a newly constructed Norwegian ELSA dataset and release additional data with each annotator’s labels for the 247 entities in the dataset’s test split. We also perform a number of experiments prompting ChatGPT for these sentiment labels regarding each entity in the text and compare the generated annotations with the human labels. Cohen’s Kappa for agreement between the best LLM-generated labels and curated gold was 0.425, which indicates that these labels would not have high quality. Our analyses further investigate the errors that ChatGPT outputs, and compare them with the variations that we find among the 5 trained annotators that all annotated the same test data.

• Rønningstad, Egil; Klinger, Roman; Øvrelid, Lilja & Velldal, Erik (2024). Entity-Level Sentiment: More than the Sum of Its Parts, The 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis Proceedings of the Workshop. Association for Computational Linguistics (ACL). ISSN 9798891761568. p. 84–96. Full text in Research Archive Show summary

  In sentiment analysis of longer texts, there may be a variety of topics discussed, of entities mentioned, and of sentiments expressed regarding each entity. We find a lack of studies exploring how such texts express their sentiment towards each entity of interest, and how these sentiments can be modelled. In order to better understand how sentiment regarding persons and organizations (each entity in our scope) is expressed in longer texts, we have collected a dataset of expert annotations where the overall sentiment regarding each entity is identified, together with the sentence-level sentiment for these entities separately. We show that the reader's perceived sentiment regarding an entity often differs from an arithmetic aggregation of sentiments at the sentence level. Only 70% of the positive and 55% of the negative entities receive a correct overall sentiment label when we aggregate the (human-annotated) sentiment labels for the sentences where the entity is mentioned. Our dataset reveals the complexity of entity-specific sentiment in longer texts, and allows for more precise modelling and evaluation of such sentiment expressions.

• Rønningstad, Egil (2023). UIO at SemEval-2023 Task 12: Multilingual fine-tuning for sentiment classification in low-resource languages. Association for Computational Linguistics (ACL). Annual Meeting Conference Proceedings. ISSN 0736-587X. doi: 10.18653/v1/2023.semeval-1.144. Full text in Research Archive Show summary

  Our contribution to the 2023 AfriSenti-SemEval shared task 12: Sentiment Analysis for African Languages, provides insight into how a multilingual large language model can be a resource for sentiment analysis in languages not seen during pretraining. The shared task provides datasets of a variety of African languages from different language families. The languages are to various degrees related to languages used during pretraining, and the language data contain various degrees of code-switching. We experiment with both monolingual and multilingual datasets for the final fine-tuning, and find that with the provided datasets that contain samples in the thousands, monolingual fine-tuning yields the best results.

• Charpentier, Lucas Georges Gabriel; Wold, Sondre; Samuel, David & Rønningstad, Egil (2023). BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer. In Alumäe, Tanel & Fishel, Mark (Ed.), Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa). University of Tartu. ISSN 9789916219997. Full text in Research Archive
• Samuel, David; Kutuzov, Andrei; Touileb, Samia; Velldal, Erik; Øvrelid, Lilja & Rønningstad, Egil [Show all 8 contributors for this article] (2023). NorBench – A Benchmark for Norwegian Language Models. In Alumäe, Tanel & Fishel, Mark (Ed.), Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa). University of Tartu. ISSN 9789916219997. p. 618–633. Full text in Research Archive Show summary

  We present NorBench: a streamlined suite of NLP tasks and probes for evaluating Norwegian language models (LMs) on standardized data splits and evaluation metrics. We also introduce a range of new Norwegian language models (both encoder and encoder-decoder based). Finally, we compare and analyze their performance, along with other existing LMs, across the different benchmark tests of NorBench.

• Rønningstad, Egil; Øvrelid, Lilja & Velldal, Erik (2022). Entity-Level Sentiment Analysis (ELSA): An Exploratory Task Survey. International Conference on Computational Linguistics (ICCL) (COLING). ISSN 1525-2477. p. 6773–6783. Full text in Research Archive Show summary

  This paper explores the task of identifying the overall sentiment expressed towards volitional entities (persons and organizations) in a document - what we refer to as Entity-Level Sentiment Analysis (ELSA). While identifying sentiment conveyed towards an entity is well researched for shorter texts like tweets, we find little to no research on this specific task for longer texts with multiple mentions and opinions towards the same entity. This lack of research would be understandable if ELSA can be derived from existing tasks and models. To assess this, we annotate a set of professional reviews for their overall sentiment towards each volitional entity in the text. We sample from data already annotated for document-level, sentence-level, and target-level sentiment in a multi-domain review corpus, and our results indicate that there is no single proxy task that provides this overall sentiment we seek for the entities at a satisfactory level of performance. We present a suite of experiments aiming to assess the contribution towards ELSA provided by document-, sentence-, and target-level sentiment analysis, and provide a discussion of their shortcomings. We show that sentiment in our dataset is expressed not only with an entity mention as target, but also towards targets with a sentiment-relevant relation to a volitional entity. In our data, these relations extend beyond anaphoric coreference resolution, and our findings call for further research of the topic. Finally, we also present a survey of previous relevant work.

• Mæhlum, Petter; Haug, Dag Trygve Truslew; Jørgensen, Tollef Emil; Kåsen, Andre; Nøklestad, Anders & Rønningstad, Egil [Show all 9 contributors for this article] (2022). NARC – Norwegian Anaphora Resolution Corpus. International Conference on Computational Linguistics (ICCL) (COLING). ISSN 1525-2477. 29(7), p. 48–60. Full text in Research Archive Show summary

  Published in: Proceedings of the Fifth Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC): https://aclanthology.org/venues/coling/. We present the Norwegian Anaphora Resolution Corpus (NARC), the first publicly available corpus annotated with anaphoric relations between noun phrases for Norwegian. The paper describes the annotated data for 326 documents in Norwegian Bokmål, together with inter-annotator agreement and discussions of relevant statistics. We also present preliminary modelling results which are comparable to existing corpora for other languages, and discuss relevant problems in relation to both modelling and the annotations themselves.

View all works in NVA

• Rønningstad, Egil & Øvrelid, Lilja (2020). Targeted Sentiment Analysis for Norwegian text. Universitetet i Oslo. Full text in Research Archive Show summary

  Targeted Sentiment Analysis attempts to extract sentiment targets and the sentiment polarity towards these targets, as explicitly expressed in text. Targeted Sentiment Analysis is a difficult task where there may be multiple sentiment targets in one sentence, and there may be conflicting sentiments towards one target. Sentiment may be expressed through nuances and combinations of words at different positions in the sentence. State-of-the-art models for Targeted Sentiment Analysis therefore require large amount of data. In our thesis we explore approaches to Targeted Sentiment Analysis in scenarios where a) we have a large annotated dataset, b) we have a very limited amount of annotated data, and c) we have no annotated data for the target language and domain. Given a large monolingual dataset, we provide a state-of-the art model through the multilingual BERT (M-BERT) pretrained language model. Given more limited data we show how bilingual training data allows for noteworthy improvements over monolingual training. Given a scenario with no labeled data for the target domain and language, we demonstrate the cross-lingual performance of M-BERT for the Norwegian and English language pair. We isolate and compare the effect of domain and language differences, and demonstrate the option of machine-translating text for Targeted Sentiment Analysis.

View all works in NVA