Shenbin Qian

• Idahl, Maximilian; Tiedemann, Jörg; Pyysalo, Sampo; Salinas, David; Galica, Tomasz & Qian, Shenbin [Vis alle 22 forfattere av denne artikkelen] (2026). MultiSynt/MT: Trillion-Token Multi-Parallel Pretraining Data Translated Across 36 Languages. I Lee, Hung-Yi; Dev, Sunipa; Li, Junyi Jessy & Elliott, Desmond (Red.), Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL). ISSN 9798891764590. s. 49670–49694. doi: 10.18653/2026.emnlp-main.2185. Fulltekst i vitenarkiv Vis sammendrag

  Open web-scale pre-training corpora remain concentrated in English, limiting multilingual LLM development. We introduce MultiSynt/MT, an open synthetic parallel corpus with approximately 4.8 trillion target-language tokens across 36 European languages, produced by translating 100 billion high-quality Nemotron-CC tokens with Tower+ and OPUS-MT/HPLT-MT systems. For many medium- and lower-resource European languages, this is the largest openly available pre-training resource. On a broad multilingual benchmark suite, reference LLMs trained on MultiSynt/MT reach the final score of HPLT 2.0, a native-data baseline, using roughly 72% fewer pre-training tokens, and outperform it by approximately 15% relative at a matched 100B-token training budget. Our analyses also identify evaluation blind spots: standard multiple-choice benchmarks miss translation-quality differences that a fluency-sensitive LLM-as-judge evaluation cleanly recovers on the trained LLMs (with no fluency deficit in MultiSynt itself), and Norwegian idiomatic and culturally grounded tasks remain better served by native data. We release the corpus, including row-aligned translations from multiple systems, to support controlled research on multilingual pre-training data and evaluation.

• Qian, Shenbin & Scherrer, Yves (2026). Why do Large Language Models Fail in Low-resource Translation? Unraveling the Token Dynamics of Large Language Models for Machine Translation. I Shterionov, Dimitar; Vanmassenhove, Eva; Sisto, Mirella De; Blain, Fred; Pourmostafa, Javed; Sharami, Roshan; Lepp, Lisa; Manna, Chiara; Rescigno, Argentina Anna; Karakanta, Aliana; Terryn, Ayla Rigouts; Lardelli, Manuel; Resende, Natalia; Murgolo, Elena; Hackenbuchner, Janica; Zaretskaya, Anna; Esplà-Gomis, Miquel; Etchegoyhen, Thierry; Gromann, Dagmar; Bawden, Rachel; Haddow, Barry; Szoc, Sara; Forcada, Mikel & Moniz, Helena (Red.), Proceedings of the 26th Annual Conference of the European Association for Machine Translation. European Association for Machine Translation (EAMT). ISSN 9789403901404. s. 31–56. doi: 10.26116/9789403901404. Fulltekst i vitenarkiv Vis sammendrag

  Large Language Models (LLMs) have recently demonstrated strong performance in machine translation (MT). However, most prior work focuses on improving or benchmarking translation quality, offering limited insight into when and why LLM-based translation fails. In this work, we systematically analyze failure modes of LLMs in MT by evaluating 15 models, including four reasoning LLMs, across 22 language pairs (LPs) with varying resource levels. We find that non-English-centric LPs consistently yield lower COMET scores than English-centric pairs. To investigate the underlying causes, we introduce Token Activation Rate (TAR), a metric that captures how effectively a model utilizes language-specific tokens in its vocabulary during generation. We validate TAR as a proxy for language representation using models with known language distributions in the training data, and show that lower TAR is strongly associated with poorer translation performance. Furthermore, reasoning LLMs tend to generate more tokens when translating into low-TAR languages, suggesting a compensatory mechanism, although its impact on translation quality varies across models. Overall, our findings emphasize the importance of token-level dynamics in understanding MT performance of LLMs.

Se alle arbeider i NVA