Bakgrunn
Professor ved språkteknologigruppen (LTG) i seksjon for maskinlæring ved IFI. Har over 25 års erfaring fra forskning og undervisning i bruk av maskinlæring for analyse av naturlige (altså `menneskelige') språk. Leder for øyeblikket SANT-prosjektet som fokuserer på sentimentanalyse. Er også del av SFI'ene MediaFutures og NorwAI, samt SFF'et Integreat – Norsk senter for kunnskapsdrevet maskinlæring. Sitter i fagrådet for UiOs Senter for data- og beregningsvitenskap dScience.
Undervisning
Har undervist bl.a. følgende emner ved IFI:
Personlig hjemmeside
velldal.net/erik/
Emneord:
NLP,
språkteknologi,
språkmodeller,
datalingvistikk,
maskinlæring,
kunstig intelligens,
AI,
Machine Learning,
Natural Language Processing,
sentimentanalyse,
nevrale modeler,
dyp læring,
Kunstig intelligens,
language models,
language modeling
Publikasjoner
-
Olsen, Helene Bøsei; Touileb, Samia & Velldal, Erik
(2023).
Arabic dialect identification: An in-depth error analysis on the MADAR parallel corpus.
I Sawaf, Hassan; El-Beltagy, Samhaa; Zaghouani, Wajdi; Magdy, Walid; Abdelali, Ahmed; Tomeh, Nadi; Abu Farha, Ibrahim; Habash, Nizar; Khalifa, Salam; Keleg, Amr; Haddad, Hatem; Zitouni, Imed; Mrini, Khalil & Almatham, Rawan (Red.),
Proceedings of ArabicNLP 2023.
Association for Computational Linguistics.
ISSN 978-1-959429-27-2.
s. 370–384.
doi:
10.18653/v1/2023.arabicnlp-1.30.
Fulltekst i vitenarkiv
-
-
Mæhlum, Petter; Velldal, Erik & Øvrelid, Lilja
(2023).
A Diagnostic Dataset for Sentiment and Negation Modeling for Norwegian.
I Ilinykh, Nikolai; Morger, Felix; Dannélls, Dana; Dobnik, Simon; Megyesi, Beáta & Nivre, Joakim (Red.),
Proceedings of the Second Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2023).
Association for Computational Linguistics.
ISSN 978-1-959429-73-9.
Fulltekst i vitenarkiv
-
Kolesnichenko, Larisa; Velldal, Erik & Øvrelid, Lilja
(2023).
Word Substitution with Masked Language Models as Data Augmentation for Sentiment Analysis.
I Ilinykh, Nikolai; Morger, Felix; Dannélls, Dana; Dobnik, Simon; Megyesi, Beáta & Nivre, Joakim (Red.),
Proceedings of the Second Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2023).
Association for Computational Linguistics.
ISSN 978-1-959429-73-9.
Fulltekst i vitenarkiv
-
-
Touileb, Samia; Øvrelid, Lilja & Velldal, Erik
(2023).
Measuring normative and descriptive biases in language models using census data.
I Vlachos, Andreas & Augenstein, Isabelle (Red.),
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics.
Association for Computational Linguistics.
ISSN 978-1-959429-44-9.
Fulltekst i vitenarkiv
-
Samuel, David; Kutuzov, Andrei; Øvrelid, Lilja & Velldal, Erik
(2023).
Trained on 100 million words and still in shape: BERT meets British National Corpus.
I Vlachos, Andreas & Augenstein, Isabelle (Red.),
Findings of the Association for Computational Linguistics: EACL 2023.
Association for Computational Linguistics.
ISSN 978-1-959429-47-0.
s. 1954–1974.
Vis sammendrag
While modern masked language models (LMs) are trained on ever larger corpora, we here explore the effects of down-scaling training to a modestly-sized but representative, well-balanced, and publicly available English text source – the British National Corpus. We show that pre-training on this carefully curated corpus can reach better performance than the original BERT model. We argue that this type of corpora has great potential as a language modeling benchmark. To showcase this potential, we present fair, reproducible and data-efficient comparative studies of LMs, in which we evaluate several training objectives and model architectures and replicate previous empirical results in a systematic way. We propose an optimized LM architecture called LTG-BERT.
-
Mæhlum, Petter; Haug, Dag Trygve Truslew; Jørgensen, Tollef Emil; Kåsen, Andre; Nøklestad, Anders & Rønningstad, Egil
[Vis alle 9 forfattere av denne artikkelen]
(2022).
NARC – Norwegian Anaphora Resolution Corpus.
International Conference on Computational Linguistics (ICCL) (COLING).
ISSN 1525-2477.
29(7),
s. 48–60.
Fulltekst i vitenarkiv
Vis sammendrag
Published in: Proceedings of the Fifth Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC): https://aclanthology.org/venues/coling/.
We present the Norwegian Anaphora Resolution Corpus (NARC), the first publicly available corpus annotated with anaphoric relations between noun phrases for Norwegian. The paper describes the annotated data for 326 documents in Norwegian Bokmål, together with inter-annotator agreement and discussions of relevant statistics. We also present preliminary modelling results which are comparable to existing corpora for other languages, and discuss relevant problems in relation to both modelling and the annotations themselves.
-
Rønningstad, Egil; Øvrelid, Lilja & Velldal, Erik
(2022).
Entity-Level Sentiment Analysis (ELSA): An Exploratory Task Survey.
International Conference on Computational Linguistics (ICCL) (COLING).
ISSN 1525-2477.
s. 6773–6783.
Fulltekst i vitenarkiv
Vis sammendrag
This paper explores the task of identifying the overall sentiment expressed towards volitional entities (persons and organizations) in a document - what we refer to as Entity-Level Sentiment Analysis (ELSA). While identifying sentiment conveyed towards an entity is well researched for shorter texts like tweets, we find little to no research on this specific task for longer texts with multiple mentions and opinions towards the same entity. This lack of research would be understandable if ELSA can be derived from existing tasks and models. To assess this, we annotate a set of professional reviews for their overall sentiment towards each volitional entity in the text. We sample from data already annotated for document-level, sentence-level, and target-level sentiment in a multi-domain review corpus, and our results indicate that there is no single proxy task that provides this overall sentiment we seek for the entities at a satisfactory level of performance. We present a suite of experiments aiming to assess the contribution towards ELSA provided by document-, sentence-, and target-level sentiment analysis, and provide a discussion of their shortcomings. We show that sentiment in our dataset is expressed not only with an entity mention as target, but also towards targets with a sentiment-relevant relation to a volitional entity. In our data, these relations extend beyond anaphoric coreference resolution, and our findings call for further research of the topic. Finally, we also present a survey of previous relevant work.
-
Touileb, Samia; Øvrelid, Lilja & Velldal, Erik
(2022).
Occupational Biases in Norwegian and Multilingual Language Models.
I Christine, Basta; Marta R., Costa-jussà; Hila, Gonen; Christian, Hardmeier & Gabriel, Stanovsky (Red.),
Proceedings of The 4th Workshop on Gender Bias in Natural Language Processing.
Association for Computational Linguistics.
ISSN 978-1-955917-68-1.
s. 200–211.
doi:
10.18653/v1/2022.gebnlp-1.21.
Vis sammendrag
In this paper we explore how a demographic distribution of occupations, along gender dimensions, is reflected in pre-trained language models. We give a descriptive assessment of the distribution of occupations, and investigate to what extent these are reflected in four Norwegian and two multilingual models. To this end, we introduce a set of simple bias probes, and perform five different tasks combining gendered pronouns, first names, and a set of occupations from the Norwegian statistics bureau. We show that language specific models obtain more accurate results, and are much closer to the real-world distribution of clearly gendered occupations. However, we see that none of the models have correct representations of the occupations that are demo-graphically balanced between genders. We also discuss the importance of the training data on which the models were trained on, and argue that template-based bias probes can some-times be fragile, and a simple alteration in a template can change a model’s behavior.
-
Kutuzov, Andrei; Velldal, Erik & Øvrelid, Lilja
(2022).
Contextualized embeddings for semantic change detection: Lessons learned .
Northern European Journal of Language Technology (NEJLT).
ISSN 2000-1533.
8(1).
doi:
10.3384/nejlt.2000-1533.2022.3478.
Vis sammendrag
We present a qualitative analysis of the (potentially erroneous) outputs of contextualized embedding-based methods for detecting diachronic semantic change. First, we introduce an ensemble method outperforming previously described contextualized approaches. This method is used as a basis for an in-depth analysis of the degrees of semantic change predicted for English words across 5 decades. Our findings show that contextualized methods can often predict high change scores for words which are not undergoing any real diachronic semantic shift in the lexicographic sense of the term (or at least the status of these shifts is questionable). Such challenging cases are discussed in detail with examples, and their linguistic categorization is proposed. Our conclusion is that pre-trained contextualized language models are prone to confound changes in lexicographic senses and changes in contextual variance, which naturally stem from their distributional nature, but is different from the types of issues observed in methods based on static embeddings. Additionally, they often merge together syntactic and semantic aspects of lexical entities. We propose a range of possible future solutions to these issues.
-
Barnes, Jeremy; Oberlaender, Laura; Troiano, Enrica; Kutuzov, Andrei; Buchmann, Jan & Agerri, Rodrigo
[Vis alle 8 forfattere av denne artikkelen]
(2022).
SemEval 2022 Task 10: Structured Sentiment Analysis.
I Emerson, Guy; Schluter, Natalie; Stanovsky, Gabriel; Kumar, Ritesh; Palmer, Alexis; Schneider, Nathan; Singh, Siddarth & Ratan, Shyam (Red.),
Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022).
Association for Computational Linguistics.
ISSN 978-1-955917-80-3.
s. 1280–1295.
-
-
-
Barnes, Jeremy; Kurtz, Robin; Oepen, Stephan; Øvrelid, Lilja & Velldal, Erik
(2021).
Structured Sentiment Analysis as Dependency Graph Parsing.
I Zong, Chengqing; Xia, Fei; Li, Wenjie & Navigli, Roberto (Red.),
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).
Association for Computational Linguistics.
ISSN 978-1-954085-52-7.
s. 3387–3402.
doi:
http:/dx.doi.org/10.18653/v1/2021.acl-long.263.
-
-
-
-
Kutuzov, Andrei; Barnes, Jeremy; Velldal, Erik; Øvrelid, Lilja & Oepen, Stephan
(2021).
Large-Scale Contextualised Language Modelling for Norwegian.
I Dobnik, Simon & Øvrelid, Lilja (Red.),
Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa).
Linköping University Electronic Press.
ISSN 978-91-7929-614-8.
s. 30–40.
Vis sammendrag
We present the ongoing NorLM initiative to support the creation and use of very large contextualised language models for Norwegian (and in principle other Nordic languages), including a ready-to-use software environment, as well as an experience report for data preparation and training. This paper introduces the first large-scale monolingual language models for Norwegian, based on both the ELMo and BERT frameworks. In addition to detailing the training process, we present contrastive benchmark results on a suite of NLP tasks for Norwegian. For additional background and access to the data, models, and software, please see: http://norlm.nlpl.eu
-
Touileb, Samia; Øvrelid, Lilja & Velldal, Erik
(2020).
Gender and sentiment, critics and authors: a dataset of Norwegian book reviews.
I Costa-jussà, Marta R.; Hardmeier, Christian; Radford, Will & Webster, Kellie (Red.),
Proceedings of the Second Workshop on Gender Bias in Natural Language Processing.
Association for Computational Linguistics.
ISSN 978-1-952148-43-9.
s. 125–138.
Vis sammendrag
Gender bias in models and datasets is widely studied in NLP. The focus has usually been on analysing how females and males express themselves, or how females and males are described. However, a less studied aspect is the combination of these two perspectives, how female and male describe the same or opposite gender. In this paper, we present a new gender annotated sentiment dataset of critics reviewing the works of female and male authors. We investigate if this newly annotated dataset contains differences in how the works of male and female authors are critiqued, in particular in terms of positive and negative sentiment. We also explore the differences in how this is done by male and female critics. We show that there are differences in how critics assess the works of authors of the same or opposite gender. For example, male critics rate crime novels written by females, and romantic and sentimental works written by males, more negatively.
-
Øvrelid, Lilja; Mæhlum, Petter; Barnes, Jeremy & Velldal, Erik
(2020).
A Fine-Grained Sentiment Dataset for Norwegian.
I Calzolari, Nicoletta; Béchet, Frédéric; Blache, Philippe; Choukri, Khalid; Cieri, Christopher; Declerck, Thierry; Goggi, Sara; Isahara, Hitoshi; Maegaard, Bente; Mariani, Joseph; Mazo, Hélène; Moreno, Asuncion; Odijk, Jan & Piperidis, Stelios (Red.),
Proceedings of The 12th Language Resources and Evaluation Conference.
European Language Resources Association.
ISSN 979-10-95546-34-4.
-
Jørgensen, Fredrik; Aasmoe, Tobias; Husevåg, Anne-Stine Ruud; Øvrelid, Lilja & Velldal, Erik
(2020).
NorNE: Annotating Named Entities for Norwegian.
I Calzolari, Nicoletta; Béchet, Frédéric; Blache, Philippe; Choukri, Khalid; Cieri, Christopher; Declerck, Thierry; Goggi, Sara; Isahara, Hitoshi; Maegaard, Bente; Mariani, Joseph; Mazo, Hélène; Moreno, Asuncion; Odijk, Jan & Piperidis, Stelios (Red.),
Proceedings of The 12th Language Resources and Evaluation Conference.
European Language Resources Association.
ISSN 979-10-95546-34-4.
s. 4547–4556.
-
Barnes, Jeremy Claude; Velldal, Erik & Øvrelid, Lilja
(2020).
Improving Sentiment Analysis with Multi-task Learning of Negation.
Natural Language Engineering.
ISSN 1351-3249.
1(21).
doi:
10.1017/S1351324920000510.
Vis sammendrag
Sentiment analysis is directly affected by compositional phenomena in language that act on the prior polarity of the words and phrases found in the text. Negation is the most prevalent of these phenomena and in order to correctly predict sentiment, a classifier must be able to identify negation and disentangle the effect that its scope has on the final polarity of a text. This paper proposes a multi-task approach to explicitly incorporate information about negation in sentiment analysis, which we show outperforms learning negation implicitly in a data-driven manner. We describe our approach, a cascading neural architecture with selective sharing of LSTM layers, and show that explicitly training the model with negation as an auxiliary task helps improve the main task of sentiment analysis. The effect is demonstrated across several different standard English-language data sets for both tasks and we analyze several aspects of our system related to its performance, varying types and amounts of input data and different multi-task setups.
-
Hammer, Hugo Lewi; Riegler, Michael Alexander; Øvrelid, Lilja & Velldal, Erik
(2019).
THREAT: A Large Annotated Corpus for Detection of Violent Threats,
Proceedings of Content Based Multimedia Information (CBMI 2019).
IEEE conference proceedings.
ISSN 978-1-7281-4673-7.
s. 1–5.
doi:
10.1109/CBMI.2019.8877435.
-
Ravishankar, Vinit; Gökirmak, Memduh; Øvrelid, Lilja & Velldal, Erik
(2019).
Multilingual Probing of Deep Pre-Trained Contextual Encoders.
I Nivre, Joakim; Derczynski, Leon; Ginter, Filip; Lindi, Bjørn; Oepen, Stephan; Søgaard, Anders & Tidemann, Jorg (Red.),
Proceedings of the First NLPL Workshop on Deep Learning for Natural Language Processing.
Linköping University Electronic Press.
ISSN 978-91-7929-999-6.
s. 37–47.
-
Mæhlum, Petter; Barnes, Jeremy Claude; Øvrelid, Lilja & Velldal, Erik
(2019).
Annotating evaluative sentences for sentiment analysis: a dataset for Norwegian.
I Hartmann, Mareike & Plank, Barbara (Red.),
Proceedings of the 22nd Nordic Conference on Computational Linguistics (NoDaLiDa).
Linköping University Electronic Press.
ISSN 978-91-7929-995-8.
s. 121–130.
Fulltekst i vitenarkiv
Vis sammendrag
This paper documents the creation of a large-scale dataset of evaluative sentences – i.e. both subjective and objective sentences that are found to be sentiment-bearing – based on mixed-domain professional reviews from various news-sources. We present both the annotation scheme and first results for classification experiments. The effort represents a step toward creating a Norwegian dataset for fine-grained sentiment analysis.
-
Barnes, Jeremy Claude; Touileb, Samia; Øvrelid, Lilja & Velldal, Erik
(2019).
Lexicon information in neural sentiment analysis: a multi-task learning approach.
I Hartmann, Mareike & Plank, Barbara (Red.),
Proceedings of the 22nd Nordic Conference on Computational Linguistics (NoDaLiDa).
Linköping University Electronic Press.
ISSN 978-91-7929-995-8.
s. 175–186.
Fulltekst i vitenarkiv
Vis sammendrag
This paper explores the use of multi-task learning (MTL) for incorporating external knowledge in neural models. Specifically, we show how MTL can enable a BiLSTM sentiment classifier to incorporate information from sentiment lexicons. Our MTL set-up is shown to improve model performance (compared to a single-task set-up) on both English and Norwegian sentence-level sentiment datasets. The paper also introduces a new sentiment lexicon for Norwegian.
-
Ravishankar, Vinit; Øvrelid, Lilja & Velldal, Erik
(2019).
Probing Multilingual Sentence Representations With X-Probe.
I Augenstein, Isabelle; Gella, Spandana; Ruder, Sebastian; Can, Burcu; Welbl, Johannes; Ren, Xiang & Rei, Marek (Red.),
The 4th Workshop on Representation Learning for NLP (RepL4NLP-2019) : Proceedings of the Workshop.
Association for Computational Linguistics.
ISSN 978-1-950737-35-2.
s. 156–168.
doi:
10.18653/v1/W19-4318.
Vis sammendrag
This paper extends the task of probing sentence representations for linguistic insight in a multilingual domain. In doing so, we make two contributions: first, we provide datasets for multilingual probing, derived from Wikipedia, in five languages, viz. English, French, German, Spanish and Russian. Second, we evaluate six sentence encoders for each language, each trained by mapping sentence representations to English sentence representations, using sentences in a parallel corpus. We discover that cross-lingually mapped representations are often better at retaining certain linguistic information than representations derived from English encoders trained on natural language inference (NLI) as a downstream task.
-
Barnes, Jeremy Claude; Øvrelid, Lilja & Velldal, Erik
(2019).
Sentiment analysis is not solved! Assessing and probing sentiment classification.
I Linzen, Tal; Chrupała, Grzegorz; Belinkov, Yonatan & Hupkes, Dieuwke (Red.),
The BlackboxNLP Workshop on Analyzing and Interpreting
Neural Networks for NLP at ACL 2019: Proceedings of the Second Workshop.
Association for Computational Linguistics.
ISSN 978-1-950737-30-7.
s. 12–23.
doi:
10.18653/v1/w19-4802.
Fulltekst i vitenarkiv
Vis sammendrag
Neural methods for sentiment analysis have led to quantitative improvements over previous approaches, but these advances are not always accompanied with a thorough analysis of the qualitative differences. Therefore, it is not clear what outstanding conceptual challenges for sentiment analysis remain. In this work, we attempt to discover what challenges still prove a problem for sentiment classifiers for English and to provide a challenging dataset. We collect the subset of sentences that an (oracle) ensemble of state-of-the-art sentiment classifiers misclassify and then annotate them for 18 linguistic and paralinguistic phenomena, such as negation, sarcasm, modality, etc. Finally, we provide a case study that demonstrates the usefulness of the dataset to probe the performance of a given sentiment classifier with respect to linguistic phenomena.
-
Rodina, Julia; Bakshandaeva, Daria; Fomin, Vadim; Kutuzov, Andrei; Touileb, Samia & Velldal, Erik
(2019).
Measuring Diachronic Evolution of Evaluative Adjectives with Word Embeddings: the Case for English, Norwegian, and Russian.
I Tahmasebi, Nina; Borin, Lars; Jatowt, Adam & Xu, Yang (Red.),
Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change.
Association for Computational Linguistics.
ISSN 978-1-950737-31-4.
s. 202–209.
doi:
10.18653/v1/W19-4725.
Fulltekst i vitenarkiv
Vis sammendrag
We measure the intensity of diachronic semantic shifts in adjectives in English, Norwegian and Russian across 5 decades. This is done in order to test the hypothesis that evaluative adjectives are more prone to temporal semantic change. To this end, 6 different methods of quantifying semantic change are used. Frequency-controlled experimental results show that, depending on the particular method, evaluative adjectives either do not differ from other types of adjectives in terms of semantic change or appear to actually be less prone to shifting (particularly, to ‘jitter’-type shifting). Thus, in spite of many well-known examples of semantically changing evaluative adjectives (like ‘terrific’ or ‘incredible’), it seems that such cases are not specific to this particular type of words.
-
Kutuzov, Andrei; Velldal, Erik & Øvrelid, Lilja
(2019).
One-to-X Analogical Reasoning on Word Embeddings: a Case for Diachronic Armed Conflict Prediction from News Texts.
I Tahmasebi, Nina; Borin, Lars; Jatowt, Adam & Xu, Yang (Red.),
Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change.
Association for Computational Linguistics.
ISSN 978-1-950737-31-4.
s. 196–201.
doi:
10.18653/v1/W19-4724.
Fulltekst i vitenarkiv
Vis sammendrag
We extend the well-known word analogy task to a one-to-X formulation, including one-to-none cases, when no correct answer exists. The task is cast as a relation discovery problem and applied to historical armed conflicts datasets, attempting to predict new relations of type ‘location:armed-group’ based on data about past events. As the source of semantic information, we use diachronic word embedding models trained on English news texts. A simple technique to improve diachronic performance in such task is demonstrated, using a threshold based on a function of cosine distance to decrease the number of false positives; this approach is shown to be beneficial on two different corpora. Finally, we publish a ready-to-use test set for one-to-X analogy evaluation on historical armed conflicts data.
-
Fares, Murhaf; Oepen, Stephan & Velldal, Erik
(2018).
Transfer and Multi-Task Learning for Noun–Noun Compound Interpretation.
I Riloff, Ellen (Red.),
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.
Association for Computational Linguistics.
ISSN 978-1-948087-84-1.
s. 1488–1498.
doi:
10.18653/v1/d18-1178.
Fulltekst i vitenarkiv
Vis sammendrag
In this paper, we empirically evaluate the utility of transfer and multi-task learning on a challenging semantic classification task: semantic interpretation of noun--noun compounds. Through a comprehensive series of experiments and in-depth error analysis, we show that transfer learning via parameter initialization and multi-task learning via parameter sharing can help a neural classification model generalize over a highly skewed distribution of relations. Further, we demonstrate how dual annotation with two distinct sets of relations over the same set of compounds can be exploited to improve the overall accuracy of a neural classifier and its F1 scores on the less frequent, but more difficult relations.
-
Stadsnes, Cathrine; Øvrelid, Lilja & Velldal, Erik
(2018).
Evaluating Semantic Vectors for Norwegian.
NIKT: Norsk IKT-konferanse for forskning og utdanning.
ISSN 1892-0713.
Vis sammendrag
In this article, we present two benchmark data sets for evaluating models of semantic word similarity for Norwegian. While such resources are available for English, they did not exist for Norwegian prior to this work. Furthermore, we produce large-coverage semantic vectors trained on the Norwegian Newspaper Corpus using several popular word embedding frameworks. Finally, we demonstrate the usefulness of the created resources for evaluating performance of different word embedding models on the tasks of analogical reasoning and synonym detection. The benchmark data sets and word embeddings are all made freely available.
-
Kutuzov, Andrei; Øvrelid, Lilja; Szymanski, Terrence & Velldal, Erik
(2018).
Diachronic word embeddings and semantic shifts: a survey,
Proceedings of the 27th International Conference on Computational Linguistics.
Association for Computational Linguistics.
ISSN 978-1-948087-50-6.
s. 1384–1397.
Fulltekst i vitenarkiv
Vis sammendrag
Recent years have witnessed a surge of publications aimed at tracing temporal changes in lexical semantics using distributional methods, particularly prediction-based word embedding models. However, this vein of research lacks the cohesion, common terminology and shared practices of more established areas of natural language processing. In this paper, we survey the current state of academic research related to diachronic word embeddings and semantic shifts detection. We start with discussing the notion of semantic shifts, and then continue with an overview of the existing methods for tracing such time-related shifts with word embedding models. We propose several axes along which these methods can be compared, and outline the main challenges before this emerging subfield of NLP, as well as prospects and possible applications.
-
Velldal, Erik; Øvrelid, Lilja; Bergem, Eivind Alexander; Stadsnes, Cathrine; Touileb, Samia & Jørgensen, Fredrik
(2018).
NoReC: The Norwegian Review Corpus.
I Calzolari, Nicoletta; Choukri, Khalid; Cieri, Christopher; Declerck, Thierry; Goggi, Sara; Hasida, Koiti; Isahara, Hitoshi; Maegaard, Bente; Mariani, Joseph; Mazo, Hélène; Moreno, Asuncion; Odijk, Jan; Piperidis, Stelios & Tokunaga, Takenobu (Red.),
Proceedings of the Eleventh International Conference on Language Resources and Evaluation.
European Language Resources Association.
ISSN 979-10-95546-00-9.
s. 4186–4191.
Fulltekst i vitenarkiv
Vis sammendrag
This paper presents the Norwegian Review Corpus (NoReC), created for training and evaluating models for document-level sentiment analysis. The full-text reviews have been collected from major Norwegian news sources and cover a range of different domains, including literature, movies, video games, restaurants, music and theater, in addition to product reviews across a range of categories. Each review is labeled with a manually assigned score of 1–6, as provided by the rating of the original author. This first release of the corpus comprises more than 35,000 reviews. It is distributed using the CoNLL-U format, pre-processed using UDPipe, along with a rich set of metadata. The work reported in this paper forms part of the SANT initiative (Sentiment Analysis for Norwegian Text), a project seeking to provide open resources and tools for sentiment analysis and opinion mining for Norwegian.
-
Lapponi, Emanuele; Søyland, Martin G.; Velldal, Erik & Oepen, Stephan
(2018).
The Talk of Norway: A Richly Annotated Corpus of the Norwegian Parliament, 1998–2016.
Language Resources and Evaluation.
ISSN 1574-020X.
52(3),
s. 873–893.
doi:
10.1007/s10579-018-9411-5.
Fulltekst i vitenarkiv
Vis sammendrag
In this work we present the Talk of Norway (ToN) data set, a collection of Norwegian Parliament speeches from 1998 to 2016. Every speech is richly annotated with metadata harvested from different sources, and augmented with language type, sentence, token, lemma, part-of-speech, and morphological feature annotations. We also present a pilot study on party classification in the Norwegian Parliament, carried out in the context of a cross-faculty collaboration involving researchers from both Political Science and Computer Science. Our initial experiments demonstrate how the linguistic and institutional annotations in ToN can be used to gather insights on how different aspects of the political process affect classification.
-
Eckart de Castilho, Richard; Ide, Nancy; Lapponi, Emanuele; Oepen, Stephan; Suderman, Keith & Velldal, Erik
[Vis alle 7 forfattere av denne artikkelen]
(2017).
Representation and Interchange of Linguistic Annotation. An In-Depth, Side-by-Side Comparison of Three Designs.
I Schneider, Nathan & Xue, Nianwen (Red.),
Proceedings of the 11th Linguistic Annotation Workshop.
Association for Computational Linguistics.
ISSN 978-1-945626-39-5.
s. 67–75.
doi:
10.18653/v1/w17-0808.
-
Oepen, Stephan; Øvrelid, Lilja; Björne, Jari; Johansson, Richard; Lapponi, Emanuele & Ginter, Filip
[Vis alle 7 forfattere av denne artikkelen]
(2017).
The 2017 Shared Task on Extrinsic Parser Evaluation: Towards a Reusable Community Infrastructure.
I Oepen, Stephan (Red.),
Proceedings of the 2017 Shared Task on Extrinsic Parser Evaluation at the Fourth International Conference on Dependency Linguistics and the 15th International Conference on Parsing Technologies.
Association for Computational Linguistics.
ISSN 978-1-945626-74-6.
s. 1–16.
doi:
10.18653/v1/k18-2002.
-
Kutuzov, Andrei; Velldal, Erik & Øvrelid, Lilja
(2017).
Temporal dynamics of semantic relations in word embeddings: an application to predicting armed conflict participants,
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.
Association for Computational Linguistics.
ISSN 978-1-945626-83-8.
s. 1825–1830.
doi:
10.18653/v1/D17-1194.
Fulltekst i vitenarkiv
Vis sammendrag
This paper deals with using word embedding models to trace the temporal dynamics of semantic relations between pairs of words. The set-up is similar to the well-known analogies task, but expanded with a time dimension. To this end, we apply incremental updating of the models with new training texts, including incremental vocabulary expansion, coupled with learned transformation matrices that let us map between members of the relation.
The proposed approach is evaluated on the task of predicting insurgent armed groups based on geographical locations. The gold standard data for the time span 1994--2010 is extracted from the UCDP Armed Conflicts dataset. The results show that the method is feasible and outperforms the baselines, but also that important work still remains to be done.
-
Kutuzov, Andrei; Velldal, Erik & Øvrelid, Lilja
(2017).
Tracing armed conflicts with diachronic word embedding models.
I Caselli, Tommaso (Red.),
Proceedings of the Events and Stories in the News Workshop.
Association for Computational Linguistics.
ISSN 978-1-945626-63-0.
s. 31–36.
doi:
10.18653/v1/W17-2705.
Fulltekst i vitenarkiv
Vis sammendrag
Recent studies have shown that word embedding models can be used to trace time-related (diachronic) semantic shifts for particular words. In this paper, we evaluate some of these approaches on the new task of predicting the dynamics of global armed conflicts on a year-to-year basis, using a dataset from the field of conflict research as the gold standard and the Gigaword news corpus as the training data. The results show that much work still remains in extracting `cultural' semantic shifts from diachronic word embedding models. At the same time, we present a new task complete with an evaluation set and introduce the `anchor words' method which outperforms previous approaches on this data.
-
Enger, Martine; Velldal, Erik & Øvrelid, Lilja
(2017).
An open-source tool for negation detection: a maximum-margin approach.
I Blanco, Eduardo; Morante, Roser & Saurí, Roser (Red.),
Proceedings of the Workshop on Computational Semantics Beyond Events and Roles.
Association for Computational Linguistics.
ISSN 978-1-945626-49-4.
s. 64–69.
doi:
10.18653/v1/w17-1810.
Vis sammendrag
This paper presents an open-source toolkit for negation detection. It identifies negation cues and their corresponding scope in either raw or parsed text using maximum-margin classification. The system design draws on best practice from the existing literature on negation detection, aiming for a simple and portable system that still achieves competitive performance. Pre-trained models and experimental results are provided for English.
-
Sand, Heidi Marion; Velldal, Erik & Øvrelid, Lilja
(2017).
Wordnet extension via word embeddings: Experiments on the Norwegian Wordnet.
I Tiedemann, Jörg (Red.),
Proceedings of the 21st Nordic Conference on Computational Linguistics (NoDaLiDa).
Linköping University Electronic Press.
ISSN 978-91-7685-601-7.
s. 298–302.
Fulltekst i vitenarkiv
Vis sammendrag
This paper describes the process of automatically adding synsets and hypernymy relations to an existing wordnet based on word embeddings computed for POStagged lemmas in a large news corpus, achieving exact match attachment accuracy of over 80%. The reported experiments are based on the Norwegian Wordnet, but the method is language independent and also applicable to other wordnets. Moreover, this study also represents the first documented experiments of the Norwegian Wordnet.
-
Hohle, Petter; Øvrelid, Lilja & Velldal, Erik
(2017).
Optimizing a PoS Tagset for Norwegian Dependency Parsing.
I Tiedemann, Jörg (Red.),
Proceedings of the 21st Nordic Conference on Computational Linguistics (NoDaLiDa).
Linköping University Electronic Press.
ISSN 978-91-7685-601-7.
s. 142–151.
Fulltekst i vitenarkiv
Vis sammendrag
This paper reports on a suite of experiments that evaluates how the linguistic granularity of part-of-speech tagsets impacts the performance of tagging and syntactic dependency parsing. Our results show that parsing accuracy can be significantly improved by introducing more finegrained morphological information in the tagset, even if tagger accuracy is compromised. Our taggers and parsers are trained and tested using the annotations of the Norwegian Dependency Treebank.
-
Velldal, Erik; Øvrelid, Lilja & Hohle, Petter
(2017).
Joint UD Parsing of Norwegian Bokmål and Nynorsk.
I Tiedemann, Jörg (Red.),
Proceedings of the 21st Nordic Conference on Computational Linguistics (NoDaLiDa).
Linköping University Electronic Press.
ISSN 978-91-7685-601-7.
s. 1–10.
Fulltekst i vitenarkiv
Vis sammendrag
This paper investigates interactions in parser performance for the two official standards for written Norwegian: Bokmål and Nynorsk. We demonstrate that while applying models across standards yields poor performance, combining the training data for both standards yields better results than previously achieved for each of them in isolation. This has immediate practical value for processing Norwegian, as it means that a single parsing pipeline is sufficient to cover both varieties, with no loss in accuracy. Based on the Norwegian Universal Dependencies treebank we present results for multiple taggers and parsers, experimenting with different ways of varying the training data given to the learners, including the use of machine translation.
-
Kutuzov, Andrei; Fares, Murhaf; Oepen, Stephan & Velldal, Erik
(2017).
Word vectors, reuse, and replicability: Towards a community repository of large-text resources.
I Tiedemann, Jörg (Red.),
Proceedings of the 21st Nordic Conference on Computational Linguistics (NoDaLiDa).
Linköping University Electronic Press.
ISSN 978-91-7685-601-7.
s. 271–276.
Fulltekst i vitenarkiv
Vis sammendrag
This paper describes an emerging shared repository of large-text resources for creating word vectors, including pre-processed corpora and pre-trained vectors for a range of frameworks and configurations. This will facilitate reuse, rapid experimentation, and replicability of results.
-
-
Oepen, Stephan; Read, Jonathon; Scheffler, Tatjana; Sidarenka, Uladzimir; Stede, Manfred & Velldal, Erik
[Vis alle 7 forfattere av denne artikkelen]
(2016).
OPT: Oslo–Potsdam–Teesside. Pipelining Rules, Rankers, and Classifier Ensembles for Shallow Discourse Parsing.
I Xue, Nianwen (Red.),
Proceedings of the CoNLL-16 Shared Task.
Association for Computational Linguistics.
ISSN 978-1-932432-66-4.
s. 20–26.
doi:
10.18653/v1/k16-2002.
-
-
-
Fares, Murhaf; Oepen, Stephan & Velldal, Erik
(2015).
Identifying Compounds: On The Role of Syntax.
I Dickinson, Markus; Hinrichs, Erhard; Patejuk, Agnieszka & Przepiórkowski, Adam (Red.),
Proceedings of the Fourteenth Workshop on Treebanks and Linguistic Theories (TLT14).
Institute of Computer Science, Polish Academy of Sciences (IPI-PAN).
ISSN 978-83-63159-18-4.
-
Lien, Jostein; Velldal, Erik & Øvrelid, Lilja
(2015).
Improving cross-domain dependency parsing with dependency-derived clusters.
I Megyesi, Beáta (Red.),
Proceedings of the 20th Nordic Conference of Computational Linguistics.
Linköping University Electronic Press.
ISSN 978-91-7519-098-3.
s. 117–126.
Fulltekst i vitenarkiv
-
Lapponi, Emanuele; Velldal, Erik; Oepen, Stephan & Knudsen, Rune Lain
(2014).
Off-Road LAF: Encoding and Processing Annotations in NLP Workflows.
I Calzolari, Nicoletta; Choukri, Khalid; Declerck, Thierry; Loftsson, Hrafn; Maegaard, Bente; Mariani, Joseph; Moreno, Asuncion; Odijk, Jan & Piperidis, Stelios (Red.),
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14).
European Language Resources Association.
ISSN 978-2-9517408-8-4.
s. 4578–4584.
-
-
Lapponi, Emanuele; Velldal, Erik; Vazov, Nikolay Aleksandrov & Oepen, Stephan
(2013).
Towards large-scale language analysis in the cloud.
I De Smedt, Koenraad; Borin, Lars; Lindén, Krister; Maegaard, Bente; Rögnvaldsson, Eiríkur & Vider, Kadri (Red.),
Proceedings of the workshop on Nordic language research infrastructure at NODALIDA 2013.
Linköping University Electronic Press.
ISSN 978-91-7519-585-8.
s. 1–10.
Fulltekst i vitenarkiv
-
Lapponi, Emanuele; Velldal, Erik; Vazov, Nikolay Aleksandrov & Oepen, Stephan
(2013).
HPC-ready Language Analysis for Human Beings.
I Oepen, Stephan; Hagen, Kristin & Johannessen, Janne Bondi (Red.),
Proceedings of the 19th Nordic Conference of Computational Linguistics (NODALIDA 2013).
Linköping University Electronic Press.
ISSN 978-91-7519-589-6.
s. 447–452.
Fulltekst i vitenarkiv
-
-
Read, Jonathon; Velldal, Erik & Øvrelid, Lilja
(2012).
Topic classification for Suicidology.
Journal of Computing Science and Engineering.
ISSN 1976-4677.
6(2),
s. 143–150.
doi:
10.5626/JCSE.2012.6.2.143.
-
Velldal, Erik; Øvrelid, Lilja; Read, Jonathon & Oepen, Stephan
(2012).
Speculation and Negation: Rules, Rankers, and the Role of Syntax.
Computational Linguistics.
ISSN 0891-2017.
38(2),
s. 369–410.
doi:
10.1162/COLI_a_00126.
Vis sammendrag
This article explores a combination of deep and shallow approaches to the problem of resolving
the scope of speculation and negation within a sentence, specifically in the domain of biomedical
research literature. The first part of the article focuses on speculation. After first showing how
speculation cues can be accurately identified using a very simple classifier informed only by
local lexical context, we go on to explore two different syntactic approaches to resolving the
in-sentence scopes of these cues. Whereas one uses manually crafted rules operating over dependency
structures, the other automatically learns a discriminative ranking function over nodes
in constituent trees. We provide an in-depth error analysis and discussion of various linguistic
properties characterizing the problem, and show that although both approaches perform well
in isolation, even better results can be obtained by combining them, yielding the best published
results to date on the CoNLL-2010 Shared Task data. The last part of the article describes how our
speculation system is ported to also resolve the scope of negation.With only modest modifications
to the initial design, the system obtains state-of-the-art results on this task also.
-
Lapponi, Emanuele; Velldal, Erik; Øvrelid, Lilja & Read, Jonathon
(2012).
UiO2: Sequence-labeling Negation Using Dependency Features.
I Agirre, Eneko; Bos, Johan & Diab, Mona (Red.),
*SEM 2012: The First Joint Conference on Lexical and Computational Semantics --- Volume 1: Proceedings of the main conference and the shared task.
Association for Computational Linguistics.
ISSN 978-1-937284-21-3.
s. 319–327.
-
-
-
-
Read, Jonathon; Velldal, Erik; Oepen, Stephan & Øvrelid, Lilja
(2011).
Resolving speculation and negation scope in biomedical articles with a syntactic constituent ranker.
I Gao, Helena Hong & Dong, Minghui (Red.),
Proceedings of the 25th Pacific Asia Conference on Language, Information and Computation, and the Fourth International Symposium on Languages in Biology and Medicine.
Pacific Asia Conference on Language, Information and Computation.
ISSN 978-4-905166-02-3.
-
-
-
-
Øvrelid, Lilja; Velldal, Erik & Oepen, Stephan
(2010).
Syntactic Scope Resolution in Uncertainty Analysis.
I Jurafsky, Dan (Red.),
Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010).
Daheng Electronic Press.
ISSN 978-7-900268-00-6.
s. 1379–1387.
-
-
Oepen, Stephan; Velldal, Erik; Lønning, Jan Tore; Meurer, Paul; Rosén, Victoria & Flickinger, Dan
(2007).
Towards Hybrid Quality-Oriented Machine Translation. On Linguistics and Probabilities in MT.
I Way, Andy & Gawronska, Barbara (Red.),
Proceedings of the 11th International Conference on Theoretical Issues in Machine Translation.
Skövde University Studies in Informatics.
ISSN 978-91-977095-0-7.
s. 144–153.
-
Velldal, Erik & Oepen, Stephan
(2006).
Statistical Ranking in Tactical Generation.
I Jurafsky, Dan & Gaussier, Eric (Red.),
Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing.
Association for Computational Linguistics.
ISSN 1-932432-73-6.
s. 517–525.
-
Velldal, Erik & Oepen, Stephan
(2005).
Maximum Entropy Models for Realization Ranking.
I Tsujii, Jun-ichi (Red.),
Proceedings of the 10th Machine Translation Summit.
Asia-Pacific Association for Machine Translation.
ISSN 974-7431-26-2.
Se alle arbeider i Cristin
-
Touileb, Samia; Øvrelid, Lilja & Velldal, Erik
(2021).
Using Gender- and Polarity-informed Models to Investigate Bias.
-
-
-
Lapponi, Emanuele; Oepen, Stephan; Skjærholt, Arne & Velldal, Erik
(2015).
LAP: The CLARINO Language Analysis Portal.
-
-
Read, Jonathon & Velldal, Erik
(2011).
Labeling emotions in suicide notes: Cost-sensitive learning with heterogeneous features.
-
Velldal, Erik
(2011).
Random Indexing Re-Hashed.
-
Velldal, Erik
(2010).
Detecting Uncertainty in Biomedical Literature: A Simple Disambiguation Approach Using Sparse Random Indexing.
-
-
-
-
Oepen, Stephan; Dyvik, Helge; Lønning, Jan Tore; Velldal, Erik; Beermann, Dorothee & Carroll, John
[Vis alle 12 forfattere av denne artikkelen]
(2004).
Som å kappete med trollet? Towards MRS-Based Norwegian-English Machine Translation.
-
Strand, Håvard; Velldal, Erik & Landsverk, Peder
(2019).
Partial Automation of the Data-collection Process.
Universitetet i Oslo.
-
Velldal, Erik
(2008).
Empirical Realization Ranking.
Unipub forlag.
ISSN 0806-3222.
Vis sammendrag
This thesis develops a new approach to the problem of indeterminacy in grammar-based natural language generation (NLG). The problem of indeterminacy concerns the fact that, for a given input semantic representation, the grammar might allow for several (i.e. thousands) alternative surface realizations. While the traditional approach to dealing with this problem is to rank the generated strings using a surface-oriented n-gram language model (LM), this thesis develops a linguistically informed approach based on features that are keyed to the internal structure of the realizations. The approach extends on the methodology previously used for statistical parsing and statistical unification-based grammars, and adapts it to the context of generation. This allows us to train treebank-based discriminative realization rankers based on modeling frameworks such as Maximum Entropy (MaxEnt) and Support Vector Machines (SVMs). The training data is based on the novel notion of a generation treebank, which we show how to automatically create on the basis of an existing parse-oriented treebank.
For reference, we also develop an n-gram-based LM trained on a large corpus of raw text. Our experimental results show that the use of a discriminative model trained on just a few thousand items in a generation treebank, gives significantly better ranking performance than the use of a traditional surface-oriented LM. Moreover, we show that even better results can be obtained by combining the two modeling approaches. This is done by including the LM as an additional feature in the discriminative model. Evaluation scores are reported for several data sets and using a range of different automated metrics. We also include results for a manual evaluation carried out by a panel of external anonymous judges.
The hybrid system for surface realization described in this thesis is currently integrated for target language generation in the Norwegian‒English machine translation (MT) system LOGON. We also show how the realization ranker is used together with a global end-to-end reranking model for selecting the final output of the MT system.
Se alle arbeider i Cristin
Publisert
4. nov. 2010 14:24
- Sist endret
13. sep. 2023 15:15