Jeremy Barnes

• Barnes, Jeremy Claude; Touileb, Samia; Mæhlum, Petter & Lison, Pierre (2023). Identifying Token-Level Dialectal Features in Social Media. In Alumäe, Tanel & Fishel, Mark (Ed.), Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa). University of Tartu. ISSN 978-99-1621-999-7. Full text in Research Archive Show summary

  Dialectal variation is present in many human languages and is attracting a growing interest in NLP. Most previous work concentrated on either (1) classifying dialectal varieties at the document or sentence level or (2) performing standard NLP tasks on dialectal data. In this paper, we propose the novel task of token-level dialectal feature prediction. We present a set of fine-grained annotation guidelines for Norwegian dialects, expand a corpus of dialectal tweets, and manually annotate them using the introduced guidelines. Furthermore, to evaluate the learnability of our task, we conduct labeling experiments using a collection of baselines, weakly supervised and supervised sequence labeling models. The obtained results show that, despite the difficulty of the task and the scarcity of training data, many dialectal features can be predicted with reasonably high accuracy.

• Samuel, David; Barnes, Jeremy; Kurtz, Robin; Oepen, Stephan; Øvrelid, Lilja & Velldal, Erik (2022). Direct parsing to sentiment graphs. Association for Computational Linguistics (ACL). Annual Meeting Conference Proceedings. ISSN 0736-587X. 2(60), p. 470–478. doi: 10.18653/v1/2022.acl-short.51.
• Moore, Andrew & Barnes, Jeremy Claude (2021). Multi-task Learning of Negation and Speculation for Targeted Sentiment Classification, Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics. ISSN 978-1-954085-46-6. p. 2838–2869.
• Lison, Pierre; Barnes, Jeremy & Hubin, Aliaksandr (2021). skweak: Weak Supervision Made Easy for NLP. In Ji, Heng & Park, Jong (Ed.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations. Association for Computational Linguistics. ISSN 978-1-954085-56-5. p. 337–346. doi: 10.18653/v1/2021.acl-demo.40. Full text in Research Archive Show summary

  We present skweak, a versatile, Python-based software toolkit enabling NLP developers to apply weak supervision to a wide range of NLP tasks. Weak supervision is an emerging machine learning paradigm based on a simple idea: instead of labelling data points by hand, we use labelling functions derived from domain knowledge to automatically obtain annotations for a given dataset. The resulting labels are then aggregated with a generative model that estimates the accuracy (and possible confusions) of each labelling function. The skweak toolkit makes it easy to implement a large spectrum of labelling functions (such as heuristics, gazetteers, neural models or linguistic constraints) on text data, apply them on a corpus, and aggregate their results in a fully unsupervised fashion. skweak is especially designed to facilitate the use of weak supervision for NLP tasks such as text classification and sequence labelling. We illustrate the use of skweak for NER and sentiment analysis. skweak is released under an open-source license and is available at https://github.com/NorskRegnesentral/skweak

• Touileb, Samia & Barnes, Jeremy (2021). The interplay between language similarity and script on a novel multi-layer Algerian dialect corpus. In Zong, Chengqing; Xia, Fei; Li, Wenjie & Navigli, Roberto (Ed.), Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Association for Computational Linguistics. ISSN 978-1-954085-54-1. p. 3700–3712. doi: 10.18653/v1/2021.findings-acl.324. Show summary

  Recent years have seen a rise in interest for cross-lingual transfer between languages with similar typology, and between languages of various scripts.However, the interplay be-tween language similarity and difference in script on cross-lingual transfer is a less studied problem. We explore this interplay on cross-lingual transfer for two supervised tasks,namely part-of-speech tagging and sentiment analysis.We introduce a newly annotated corpus of Algerian user-generated comments comprising parallel annotations of Algerian written in Latin, Arabic, and code-switched scripts, as well as annotations for sentiment and topic categories.We perform baseline experiments by fine-tuning multi-lingual language models. We further explore the effect of script vs. language similarity in cross-lingual transfer by fine-tuning multi-lingual models on languages which are a) typologically distinct, but use the same script, b) typologically similar, but use a distinct script, or c) are typo-logically similar and use the same script. We find there is a delicate relationship between script and typology for part-of-speech, while sentiment analysis is less sensitive.

• Barnes, Jeremy; Kurtz, Robin; Oepen, Stephan; Øvrelid, Lilja & Velldal, Erik (2021). Structured Sentiment Analysis as Dependency Graph Parsing. In Zong, Chengqing; Xia, Fei; Li, Wenjie & Navigli, Roberto (Ed.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics. ISSN 978-1-954085-52-7. p. 3387–3402. doi: http:/dx.doi.org/10.18653/v1/2021.acl-long.263.
• Barnes, Jeremy; Mæhlum, Petter & Touileb, Samia (2021). NorDial: A Preliminary Corpus of Written Norwegian Dialect Use. In Dobnik, Simon & Øvrelid, Lilja (Ed.), Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa). Linköping University Electronic Press. ISSN 978-91-7929-614-8. p. 445–451. Show summary

  Norway has a large amount of dialectal variation, as well as a general tolerance to its use in the public sphere. There are, however, few available resources to study this variation and its change over time and in more informal areas, on social media. In this paper, we propose a first step to creating a corpus of dialectal variation of written Norwegian. We collect a small corpus of tweets and manually annotate them as Bokmål, Nynorsk, any dialect, or a mix. We further perform preliminary experiments with state-of-the-art models, as well as an analysis of the data to expand this corpus in the future. Finally, we make the annotations available for future work.

• Barnes, Jeremy; Øvrelid, Lilja & Velldal, Erik (2021). If you've got it, flaunt it: Making the most of fine-grained sentiment annotations. In Merlo, Paola; Tiedemann, Jörg & Tsarfaty, Reut (Ed.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics. ISSN 978-1-954085-02-2. p. 49–62.
• Mæhlum, Petter; Barnes, Jeremy; Kurtz, Robin; Øvrelid, Lilja & Velldal, Erik (2021). Negation in Norwegian: an annotated dataset . In Dobnik, Simon & Øvrelid, Lilja (Ed.), Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa). Linköping University Electronic Press. ISSN 978-91-7929-614-8. p. 299–308.
• Kutuzov, Andrei; Barnes, Jeremy; Velldal, Erik; Øvrelid, Lilja & Oepen, Stephan (2021). Large-Scale Contextualised Language Modelling for Norwegian. In Dobnik, Simon & Øvrelid, Lilja (Ed.), Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa). Linköping University Electronic Press. ISSN 978-91-7929-614-8. p. 30–40. Show summary

  We present the ongoing NorLM initiative to support the creation and use of very large contextualised language models for Norwegian (and in principle other Nordic languages), including a ready-to-use software environment, as well as an experience report for data preparation and training. This paper introduces the first large-scale monolingual language models for Norwegian, based on both the ELMo and BERT frameworks. In addition to detailing the training process, we present contrastive benchmark results on a suite of NLP tasks for Norwegian. For additional background and access to the data, models, and software, please see: http://norlm.nlpl.eu

• Navas-Alejo, Irean; Badia, Toni & Barnes, Jeremy (2020). Cross-lingual Emotion Intensity Prediction. In Nissim, Malvina; Patti, Viviana & Plank, Barbara (Ed.), Proceedings of the Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media. Association for Computational Linguistics. ISSN 978-4-87974-723-5. p. 140–152.
• Øvrelid, Lilja; Mæhlum, Petter; Barnes, Jeremy & Velldal, Erik (2020). A Fine-Grained Sentiment Dataset for Norwegian. In Calzolari, Nicoletta; Béchet, Frédéric; Blache, Philippe; Choukri, Khalid; Cieri, Christopher; Declerck, Thierry; Goggi, Sara; Isahara, Hitoshi; Maegaard, Bente; Mariani, Joseph; Mazo, Hélène; Moreno, Asuncion; Odijk, Jan & Piperidis, Stelios (Ed.), Proceedings of The 12th Language Resources and Evaluation Conference. European Language Resources Association. ISSN 979-10-95546-34-4.
• Lison, Pierre; Barnes, Jeremy; Hubin, Aliaksandr & Touileb, Samia (2020). Named Entity Recognition without Labelled Data: A Weak Supervision Approach . In Jurafsky, Dan; Chai, Joyce; Schluter, Natalie & Tetreault, Joel (Ed.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics. ISSN 978-1-952148-25-5. p. 1518–1533. Full text in Research Archive Show summary

  Named Entity Recognition (NER) performance often degrades rapidly when applied to target domains that differ from the texts observed during training. When in-domain labelled data is available, transfer learning techniques can be used to adapt existing NER models to the target domain. But what should one do when there is no hand-labelled data for the target domain? This paper presents a simple but powerful approach to learn NER models in the absence of labelled data through weak supervision. The approach relies on a broad spectrum of labelling functions to automatically annotate texts from the target domain. These annotations are then merged together using a hidden Markov model which captures the varying accuracies and confusions of the labelling functions. A sequence labelling model can finally be trained on the basis of this unified annotation. We evaluate the approach on two English datasets (CoNLL 2003 and news articles from Reuters and Bloomberg) and demonstrate an improvement of about 7 percentage points in entity-level F1 scores compared to an out-of-domain neural NER model.

• Barnes, Jeremy Claude; Velldal, Erik & Øvrelid, Lilja (2020). Improving Sentiment Analysis with Multi-task Learning of Negation. Natural Language Engineering. ISSN 1351-3249. 1(21). doi: 10.1017/S1351324920000510. Show summary

  Sentiment analysis is directly affected by compositional phenomena in language that act on the prior polarity of the words and phrases found in the text. Negation is the most prevalent of these phenomena and in order to correctly predict sentiment, a classifier must be able to identify negation and disentangle the effect that its scope has on the final polarity of a text. This paper proposes a multi-task approach to explicitly incorporate information about negation in sentiment analysis, which we show outperforms learning negation implicitly in a data-driven manner. We describe our approach, a cascading neural architecture with selective sharing of LSTM layers, and show that explicitly training the model with negation as an auxiliary task helps improve the main task of sentiment analysis. The effect is demonstrated across several different standard English-language data sets for both tasks and we analyze several aspects of our system related to its performance, varying types and amounts of input data and different multi-task setups.

• Barnes, Jeremy Claude & Klinger, Roman (2019). Embedding Projection for Targeted Cross-Lingual Sentiment: Model Comparisons and a Real-World Study. The journal of artificial intelligence research. ISSN 1076-9757. 66, p. 691–742. doi: 10.1613/jair.1.11561. Full text in Research Archive
• Barnes, Jeremy Claude (2019). LTG-Oslo Hierarchical Multi-task Network: The importance of negation for document-level sentiment in Spanish. CEUR Workshop Proceedings. ISSN 1613-0073. 2421(5), p. 378–389. Show summary

  This paper details LTG-Oslo team's participation in the sentiment track of the NEGES 2019 evaluation campaign. We participated in the task with a hierarchical multi-task network, which used shared lower-layers in a deep BiLSTM to predict negation, while the higher layers were dedicated to predicting document-level sentiment. The multi-task component shows promise as a way to incorporate information on negation into deep neural sentiment classifiers, despite the fact that the absolute results on the test set were relatively low for a binary classification task.

• R. Atrio, Alex; Badia, Toni & Barnes, Jeremy Claude (2019). On the Effect of Word Order on Cross-lingual Sentiment Analysis. Revista de Procesamiento de Lenguaje Natural (SEPLN). ISSN 1135-5948. 63, p. 23–30. Full text in Research Archive
• Mæhlum, Petter; Barnes, Jeremy Claude; Øvrelid, Lilja & Velldal, Erik (2019). Annotating evaluative sentences for sentiment analysis: a dataset for Norwegian. In Hartmann, Mareike & Plank, Barbara (Ed.), Proceedings of the 22nd Nordic Conference on Computational Linguistics (NoDaLiDa). Linköping University Electronic Press. ISSN 978-91-7929-995-8. p. 121–130. Full text in Research Archive Show summary

  This paper documents the creation of a large-scale dataset of evaluative sentences – i.e. both subjective and objective sentences that are found to be sentiment-bearing – based on mixed-domain professional reviews from various news-sources. We present both the annotation scheme and first results for classification experiments. The effort represents a step toward creating a Norwegian dataset for fine-grained sentiment analysis.

• Barnes, Jeremy Claude; Touileb, Samia; Øvrelid, Lilja & Velldal, Erik (2019). Lexicon information in neural sentiment analysis: a multi-task learning approach. In Hartmann, Mareike & Plank, Barbara (Ed.), Proceedings of the 22nd Nordic Conference on Computational Linguistics (NoDaLiDa). Linköping University Electronic Press. ISSN 978-91-7929-995-8. p. 175–186. Full text in Research Archive Show summary

  This paper explores the use of multi-task learning (MTL) for incorporating external knowledge in neural models. Specifically, we show how MTL can enable a BiLSTM sentiment classifier to incorporate information from sentiment lexicons. Our MTL set-up is shown to improve model performance (compared to a single-task set-up) on both English and Norwegian sentence-level sentiment datasets. The paper also introduces a new sentiment lexicon for Norwegian.

• Barnes, Jeremy Claude; Øvrelid, Lilja & Velldal, Erik (2019). Sentiment analysis is not solved! Assessing and probing sentiment classification. In Linzen, Tal; Chrupała, Grzegorz; Belinkov, Yonatan & Hupkes, Dieuwke (Ed.), The BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP at ACL 2019: Proceedings of the Second Workshop. Association for Computational Linguistics. ISSN 978-1-950737-30-7. p. 12–23. doi: 10.18653/v1/w19-4802. Full text in Research Archive Show summary

  Neural methods for sentiment analysis have led to quantitative improvements over previous approaches, but these advances are not always accompanied with a thorough analysis of the qualitative differences. Therefore, it is not clear what outstanding conceptual challenges for sentiment analysis remain. In this work, we attempt to discover what challenges still prove a problem for sentiment classifiers for English and to provide a challenging dataset. We collect the subset of sentences that an (oracle) ensemble of state-of-the-art sentiment classifiers misclassify and then annotate them for 18 linguistic and paralinguistic phenomena, such as negation, sarcasm, modality, etc. Finally, we provide a case study that demonstrates the usefulness of the dataset to probe the performance of a given sentiment classifier with respect to linguistic phenomena.

View all works in Cristin

• Barnes, Jeremy; De Clercq, Orphee & Klinger, Roman (2023). The 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis. Association for Computational Linguistics. ISBN 978-1-959429-87-6. 616 p.
• Barnes, Jeremy (2021). Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics. ISBN 978-1-954085-46-6. 5953 p.

View all works in Cristin

• Lison, Pierre; Barnes, Jeremy Claude & Hubin, Aliaksandr (2021). skweak: weak supervision made easy for NLP. Show summary

  We present skweak, a versatile, Python-based software toolkit enabling NLP developers to apply weak supervision to a wide range of NLP tasks. Weak supervision is an emerging machine learning paradigm based on a simple idea: instead of labelling data points by hand, we use labelling functions derived from domain knowledge to automatically obtain annotations for a given dataset. The resulting labels are then aggregated with a generative model that estimates the accuracy (and possible confusions) of each labelling function. The skweak toolkit makes it easy to implement a large spectrum of labelling functions (such as heuristics, gazetteers, neural models or linguistic constraints) on text data, apply them on a corpus, and aggregate their results in a fully unsupervised fashion. skweak is especially designed to facilitate the use of weak supervision for NLP tasks such as text classification and sequence labelling. We illustrate the use of skweak for NER and sentiment analysis. skweak is released under an open-source license and is available at https://github.com/NorskRegnesentral/skweak

View all works in Cristin