Jie Bian

• Bian, Jie & Welzl, Michael (2026). Linking Rationale to Decision on Internet Standards: A Retrieval-Based Approach Using Synthetic Data. I Piperidis,, Stelios; Bel, Núria; Heuvel, Henk van den; Ide, Nancy; Krek, Simon & Toral, Antonio (Red.), Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026). European Language Resources Association. ISSN 9782493814494. s. 7149–7162. doi: 10.63317/3szh4omfcsxb. Fulltekst i vitenarkiv Vis sammendrag

  The Internet Engineering Task Force (IETF) develops Internet-Drafts (I-Ds) and Requests for Comments (RFCs) as formal specifications for Internet Protocols. While these documents capture finalized technical standards, the rich design rationales and deliberations that shape them are often buried in informal discussions across mailing lists. These discussions are rarely linked explicitly to the specifications they inform, making it difficult to trace the origins of specific design decisions. We address this gap by generating synthetic data that explicitly links discussion threads to their corresponding RFC/I‑D sections, producing roughly 350 000 such aligned instances. This data enables training a semantic embedding-based information retrieval (IR) system that, given an email discussion, retrieves the most relevant specification content. Our experiments show that this synthetic supervision helps models learn associations between informal discourse and formal documentation, though the task remains challenging due to the implicit and context-dependent nature of the links.

• Bian, Jie; Arefyev, Nikolay; Mühlhäuser, Max & Welzl, Michael (2025). Automated Insights Into GitHub Collaboration Dynamics. IEEE Access. 13, s. 85526–85542. doi: 10.1109/access.2025.3566309. Fulltekst i vitenarkiv Vis sammendrag

  Today, GitHub (Trademark) is the most widely used platform for open-source software development. Large projects may comprise hundreds of distributed collaborators and thousands of GitHub “issues” (structured discussions). It includes basic support for dealing with issues, via “Pull Requests” (PRs)—document changes that can manually be defined to “close” them (i.e., they address and thereby conclude the issue discussion). Unresolved issues can pile up. For example, at the time of writing, the Kubernetes repository has almost 2000 open issues; finding which ones a PR might close is a hard task by itself. We address this by automatically identifying issue-PR relationships using language models (LMs) and leveraging Information Retrieval (IR) techniques. To foster further research, we contribute a carefully curated novel dataset called CodeConvo, reflecting the most influential open-source repositories for code development as well as some technical document repositories. We use this dataset to benchmark several state-of-the-art (SOTA) non-proprietary models that show exceptional performance on the MTEB benchmark, as well as to train and evaluate the performance of Smart Insights into GitHub Issue-PR Relations (SIGIR), our tailored model. The best SIGIR model/data combination yields an average Mean Reciprocal Rank (MRR) above 0.7, around 20% higher than the best baseline performance. Notably, ablation studies revealed that knowledge transfer occurs not only between different programming languages but also between code and technical documents, albeit to a lesser extent. We believe these results are encouraging and can stimulate the practical application of LMs for taming the complexity of very large projects, in GitHub and beyond.

• Bian, Jie & Welzl, Michael (2025). Empowering IETF Collaboration with NLP Search Innovations and LLM-Enhanced RFC Writing. I Brunstrom, Anna & Beckett, Ryan (Red.), ANRW '25: Proceedings of the 2025 Applied Networking Research Workshop. Association for Computing Machinery (ACM). ISSN 9798400720093. s. 24–31. doi: 10.1145/3744200.3744761. Fulltekst i vitenarkiv Vis sammendrag

  The Internet Engineering Task Force (IETF) produces extensive textual data, including discussions in email archives and GitHub repositories and Internet-Drafts (I-Ds), which are preliminary versions of Requests for Comments (RFCs). The sheer volume and complexity of this material present significant workflow challenges, contributing to the duration of the standardization process (it is common to take several years from an initial draft to the final RFC). This paper explores the potential of Natural Language Processing (NLP) using Large Language Models (LLMs) to streamline IETF workflows. We use Information Retrieval (IR) to i) build a search system that helps users locate comments related to details in an I-D, and ii) partially automate RFC writing.

• Bian, Jie & Welzl, Michael (2025). Instruction Tuning TextFlow Semi-automatic RFCs Generation. I Ichise, Ryutaro (Red.), Natural Language Processing and Information Systems NLDB 2025. Springer Nature. ISSN 9783031971402. s. 350–364. doi: 10.1007/978-3-031-97141-9_24. Fulltekst i vitenarkiv Vis sammendrag

  Instruction tuning enhances language models’ (LMs) performance by refining them with specific guidelines. This paper explores the application of instruction tuning in generating and editing Internet Drafts (I-Ds), the preliminary versions of Request for Comments (RFCs). The process involves training models with detailed instructions based on earlier drafts and feedback from Working Groups (WGs) to improve the text. This approach enables the generation of drafts that adhere to established Internet Engineering Task Force (IETF) standards, significantly reducing the need for extensive manual revisions that can span years. This work marks a promising start for the future development of network protocols. By combining instruction tuning models with human expertise, we are moving towards more efficient and accurate technical documentation.

• Bian, Jie; Welzl, Michael; Kutuzov, Andrei & Arefyev, Nikolay (2024). Tell Me Why: Language Models Help Explain the Rationale Behind Internet Protocol Design. I LI, Geoffrey Ye; Liang, Le; Gündüz, Deniz & Antón-Haro, Carles (Red.), 2024 IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350343199. s. 447–453. doi: 10.1109/ICMLCN59089.2024.10624781. Fulltekst i vitenarkiv Vis sammendrag

  Request for Comments (RFCs) serve as guidebooks for the implementation of Internet protocols or network mechanisms. They reveal how these protocols and mechanisms work, but the underlying reasons for their operation are not always available in RFCs. We present an attempt to automate the discovery of these reasons in mailing list archives via natural language processing methods. Our approach leverages the known relationship between text changes and the discussions leading to these changes in the Working Group (WG) GitHub repositories of the Internet Engineering Task Force (IETF) to obtain labeled training data. We find that our model is able to generalize, and it can indeed discover emails that have led to formulations in RFCs. This is a first step towards facilitating a deeper understanding of these often complex documents, which can be helpful for developers, protocol designers, and educators alike.

Se alle arbeider i NVA

• Bian, Jie (2026). Automated Extraction of Design-Decision Rationales in Internet Protocols. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Internet standards define how systems communicate, but they rarely explain why they were designed in a certain way. The ideas and discussions behind important decisions are often buried in long email threads and are hard to find. This makes it difficult for people in education, industry, and research to understand, maintain, and improve these standards. This thesis shows that modern language models can help recover and explain this hidden reasoning. First, this work builds new datasets that link design decisions with the discussions that shaped them. Then, it develops methods to find relevant explanations and generate clear answers to questions such as: “Why was this decision made?” The results show that these models can uncover and organise the reasoning behind complex technical standards. This makes the development process more transparent, easier to follow, and more efficient. This work is important for education, industry, and research. It can help new contributors understand standards faster, support better decision-making, and reduce errors in implementation. In the long term, it can make the development of Internet technologies more open, reliable, and collaborative.

Se alle arbeider i NVA