Antonio Martini

• Shivashankar, Karthik; Capilla, Rafael; Kruke, Maren Maritsdatter; Orucevic, Mili & Martini, Antonio (2026). QualiTagger: automating software quality categorization in issue trackers. Software quality journal. ISSN 0963-9314. 34(3). doi: 10.1007/s11219-026-09775-2. Full text in Research Archive Show summary

  Managing software quality is crucial for maintainable software systems, yet understanding how quality attributes are discussed within the informal, noisy context of issue trackers remains a significant challenge. Current automatic approaches for categorizing quality concerns often falter, as they are typically validated on small domain-specific datasets and are ill-equipped to parse the conversational language of developer discourse. This hinders effective prioritization and technical debt management. This paper introduces QualiTagger, an automated approach for classifying seven distinct software quality attributes from issue tracker text, and QualiDataSet, a novel, curated dataset of over 700,000 labeled GitHub issues that underpins this work. We demonstrate that an ensemble of specialized binary classifiers, built upon the DistilRoBERTa architecture, significantly outperforms a single multiclass model and shows superior or comparable efficacy to a general-purpose Large Language Model (GPT-4o) for this task. The model’s real-world applicability is further validated through an industrial case study at Visma focusing on security issues) and a user study with software engineering students. Our evaluation confirms that QualiTagger achieves high classification accuracy and, crucially, generalizes effectively to previously unseen (Out-of-Distribution) projects, a key indicator of its practical utility. By providing both a robust classification tool and a large-scale public dataset, this research enables a more nuanced, data-driven understanding of how software quality is managed in practice, offering valuable insights for project management and future empirical software engineering research

• Orucevic, Mili; Kruke, Maren Maritsdatter & Martini, Antonio (2026). Beyond the Code: The Value of Practicing and Evaluating Technical Debt Management. In Spinola, Rodrigo (Eds.), International Conference on Technical Debt (TechDebt ’26), April 12–13, 2026, Rio de Janeiro, Brazil. Association for Computing Machinery (ACM). ISSN 9798400724855. doi: 10.1145/3794915.3795785. Full text in Research Archive Show summary

  Background: Effective management of Technical Debt (TD) is essential for improving software quality and sustainability. However, there is a lack of evidence about the benefits of TD management practices. Objective: This experience report involving a large international company, Visma, aims to assess the impact of employing TD practices on an organization’s capabilities and outcomes related to TD. Method: This experience report describes the usage of two tools, TDMA and TD Pulse, to respectively track the practices and assess TD management capabilities. Over eight years, some teams in Visma adopted recommended TD practices (TDMA), while others either lacked formal TD practices or relied on ad hoc, informal methods. The TD Pulse questionnaire was then administered to both groups to evaluate their TD management capabilities and two key performance indicators (KPIs): time wasted due to TD and missed business opportunities attributed to TD. Results: The analysis indicates that the implementation of TD practices has significantly enhanced various TD management capabilities and reported a better TD management outcome according to the KPIs, demonstrating clear benefits as shown by the results from TD Pulse. Conclusions: This experience report suggests that organizations generally experience positive outcomes from adopting TD management practices, indicating that companies stand to benefit substantially from their implementation.

• Shivashankar, Karthik & Martini, Antonio (2026). TD-Suite: All Batteries Included Framework for Technical Debt Classification. Lecture Notes in Business Information Processing. ISSN 1865-1348. 578, p. 137–151. doi: 10.1007/978-3-032-22375-3_9. Full text in Research Archive Show summary

  In Agile software development, maintaining velocity requires the continuous management of Technical Debt (TD). However, the rapid iteration cycles inherent to Agile often obscure debt accumulation, making manual identification in issue trackers prohibitively expensive. To address this, we present TD-Suite, a comprehensive framework engineered to automate the classification of technical debt. It leverages state-of-the-art transformer models to analyze textual artifacts, such as developer discussions in issue reports, where subtle indicators of debt often lie hidden. TD-Suite provides a seamless end-to-end pipeline suitable for Agile ML Engineering, managing everything from initial data ingestion and rigorous preprocessing to model training, thorough evaluation, and final inference. It supports both binary classification (debt or no debt) and granular categorization—identifying code, architecture, design, or documentation debt—enabling Agile teams to formulate targeted refactoring strategies. To ensure robustness on real-world, imbalanced datasets, TD-Suite incorporates k-fold cross-validation, early stopping, and class weighting strategies. The framework explicitly integrates the tracking and reporting of carbon emissions associated with model training. Furthermore, it features a user-friendly Gradio web interface within a Docker container, simplifying integration into DevOps pipelines and democratizing access for practitioners without deep ML expertise.

• Shivashankar, Karthik & Martini, Antonio (2025). Enhancing Python Code Maintainability Through Large Language Model-Based Approaches. In Scanniello, Giuseppe; Lenarduzzi, Valentina; Romano, Simone; Vegas, Sira & Francese, Rita (Ed.), Product-Focused Software Process Improvement: 26th International Conference, PROFES 2025, Salerno, Italy, December 1–3, 2025, Proceedings. Springer. ISSN 9783032120892. p. 137–152. doi: 10.1007/978-3-032-12089-2_9. Full text in Research Archive Show summary

  While Large Language Models (LLMs) increasingly assist in code generation, concerns persist regarding the maintainability of the code they produce–an aspect often overshadowed by functional correctness. Overlooking maintainability can contribute to technical debt and inflate long-term software costs. This research investigates whether targeted fine-tuning can enhance an LLM’s ability to generate more maintainable Python code. We developed a approach involving the curation of custom datasets (from COMMITPACKFT and Code Alpaca Python subsets) specifically annotated for maintainability using metrics like Source Lines of Code (SLOC), Halstead Effort, and Maintainability Index (MI). A weak-to-strong generalization strategy was employed, using a smaller model (Phi 4 14B) to generate maintainability-focused examples for fine-tuning a larger model (QwenCoder2.5 32B Instruct) with parameter-efficient techniques. Evaluations revealed the fine-tuned model significantly reduced code complexity (Halstead Effort) and length (SLOC) compared to the original code samples. While the model preserved high functional similarity (verified by CodeBERTScore), results for the Maintainability Index metric were inconclusive in this evaluation. Performance on standard functional correctness benchmarks (HumanEval+, MBPP+) was largely comparable to the base model. Nevertheless, expert user feedback confirmed the fine-tuned model’s utility as a practical AI companion for code refactoring to improve maintainability.

• Irgens, Marius; Orucevic, Mili; Boye, Mads; Gundelsby, Jan Henrik & Martini, Antonio (2025). DebtQuest: Discover Technical Debt Management Issues with Survey Visualization. IEEE Working Conference on Software Visualization (VISSOFT). ISSN 2379-7576. 2025, p. 25–35. doi: 10.1109/vissoft67405.2025.00011. Full text in Research Archive Show summary

  While Technical Debt (TD) manifest in systems as code-related symptoms, it is often caused by management-related issues. However, existing TD visualization tools are primarily code-based, and limited in addressing organizational factors. In this context, we introduce DebtQuest: an interactive, survey-based visualization tool designed to help assess an organization’s TD management practices. DebtQuest enables stakeholders to explore insights, pinpoint issues, perform benchmarking, and compare changes across time. Complementing the capabilities of code-based tools to discover TD symptoms, DebtQuest supports the discovery of underlying organizational causes. DebtQuest’s design is grounded in task analysis and a targeted review of relevant visualization techniques. It was refined through real-world use and evaluated by industry stakeholders. The contribution of this work is an accessible and flexible visualization design that extends recent research in TD management, revealing organizational insights that are difficult to discover using code-based tools.

• Shivashankar, Karthik; Haugerud, Kristian & Martini, Antonio (2025). Enhancing Task Prioritization in Software Development Issues Tracking System. Journal of Software: Evolution and Process (JSEP). ISSN 2047-7473. 37(12). doi: 10.1002/smr.70068. Full text in Research Archive
• Tverdal, Simeon; Nguyen, Phu Hong; Goknil, Arda; Martini, Antonio; Astekin, Merve & Orucevic, Mili [Show all 8 contributors for this article] (2025). Combining Insights from Multiple Tools to Manage Technical Debt in Industrial C# Projects. In MacDonell, Stephen & Tahir, Amjed (Ed.), 2025 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798331595876. p. 709–720. doi: 10.1109/icsme64153.2025.00071. Full text in Research Archive Show summary

  Technical Debt (TD) is a critical challenge in software development, leading to increased maintenance costs and reduced software quality over time. While considerable research has focused on identifying and managing TD in Java projects, studies on. NET (C#) projects remain limited. Additionally, existing approaches often rely on a single tool for TD detection, overlooking the benefits of combining multiple tools. In this paper, we analyze the effectiveness of Arcan, CodeScene, Designite, and DV8 on four industrial C#. NET 8 software products to address these research gaps. To validate and enrich our findings, we conducted online seminars and interviews with developers, architects, and managers involved in these projects, gathering practitioner insights on TD relevance and tool effectiveness. By leveraging complementary tools and practitioner feedback, we uncover different types of TD, including code-level, design, architectural, and knowledge debt. Our findings highlight each tool's strengths and limitations and demonstrate how integrating their outputs with expert input provides a more comprehensive and actionable TD assessment. Based on these insights, we propose a conceptual model for prioritizing and managing TD, offering guidance for practitioners.

• Martini, Antonio; Stray, Viktoria; Besker, Terese; Moe, Nils Brede & Bosch, Jan (2025). Process Debt: Definition, Risks, and Management. Journal of Software: Evolution and Process (JSEP). ISSN 2047-7473. 37(4), p. 1–23. doi: 10.1002/smr.70017. Full text in Research Archive Show summary

  Process debt, like technical debt, can be a source of short‐term benefits but often leads to harmful consequences in the long term for a software organization. Despite its impact, the phenomenon of process debt has not been thoroughly explored in current literature, leaving a gap in understanding how it affects and is managed within organizations. This paper addresses this gap by defining process debt, describing its occurrence, the risks of its mismanagement, and showing examples of mitigation strategies. Our study began with an exploratory phase involving semi‐structured interviews with sixteen practitioners across four international organizations, allowing us to gather diverse insights into the occurrence and management of process debt. Then, to deepen our understanding and validate our findings, we conducted a cross‐company focus group with ten additional practitioners and analyzed fifty‐eight observations and thirty‐five interviews from a longitudinal case study. The analysis of the research findings led to a definition of process debt and a novel framework. We also report on the causes, consequences, and occurrence patterns of process debt over time. We present mitigation strategies and discuss which ones need further attention for future research. Our results suggest that the debt metaphor may help companies understand how to manage and improve their processes and make process‐related decisions that are beneficial both in the short and long term.

• Shivashankar, Karthik & Martini, Antonio (2025). PyExamine: A Comprehensive, Un-Opinionated Smell Detection Tool for Python, 2025 IEEE/ACM 22nd International Conference on Mining Software Repositories (MSR). p. 763–774. doi: 10.1109/msr66628.2025.00114. Full text in Research Archive Show summary

  The growth of Python adoption across diverse domains has led to increasingly complex codebases, presenting challenges in maintaining code quality. While numerous tools attempt to address these challenges, they often fall short in providing comprehensive analysis capabilities or fail to consider Pythonspecific contexts. PyExamine addresses these critical limitations through an approach to code smell detection that operates across multiple levels of analysis. PyExamine architecture enables detailed examination of code quality through three distinct but interconnected layers: architectural patterns, structural relationships, and code-level implementations. This approach allows for the detection and analysis of 49 distinct metrics, providing developers with an understanding of their codebase’s health. The metrics span across all levels of code organization, from high-level architectural concerns to granular implementation details. Through evaluation on 7 diverse projects, PyExamine achieved detection accuracy rates: 91.4% for code-level smells, 89.3% for structural smells, and 80.6% for architectural smells. These results were further validated through extensive user feedback and expert evaluations, confirming PyExamine’s capability to identify potential issues across all levels of code organization with high recall accuracy. In additional to this, we have also used PyExamine to analysis the prevalence of different type of smells, across 183 diverse Python projects ranging from small utilities to large-scale enterprise applications. PyExamine’s distinctive combination of comprehensive analysis, Python-specific detection, and high customizability makes it a valuable asset for both individual developers and large teams seeking to enhance their code quality practices.

• Irgens, Marius & Martini, Antonio (2025). Visualization Usage in Technical Debt Management. In Peter, Sybille (Eds.), Agile Processes in Software Engineering and Extreme Programming: 26th International Conference on Agile Software Development, XP 2025, Brugg-Windisch, Switzerland, June 2–5, 2025, Proceedings | SpringerLink. Springer. ISSN 9783031945441. p. 186–194. doi: 10.1007/978-3-031-94544-1_13. Full text in Research Archive Show summary

  Visualization is a promising technique in TD management, but with various tools offering visualization, the question arise: is it being used effectively by practitioners? Through an analysis of survey data from 417 participants across seven international companies, we examine the current use of visualization and its perceived potential to improve TD management. Our findings reveal that while visualization tools are not widely used, there is a clear desire for enhanced visualization capabilities.

• Pandini, Gabriele; Martini, Antonio; Videsjorden, Adela Nedisan & Fontana, Francesca Arcelli (2025). An Exploratory Study on Architectural Smell Refactoring Using Large Languages Models, 2025 IEEE 22nd International Conference on Software Architecture Companion (ICSA-C). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798331533366. doi: https:/ieeexplore.ieee.org/document/11015021. Full text in Research Archive Show summary

  Architectural smells are abundant in codebases and regularly hinder the development of stable and maintainable code. Understanding and removing these elements can consume a huge amount of developers' time, who often need to prioritize implementing new features. This causes a substantial increase in Technical Debt, compromising the scalability and maintainability of the codebases, at time bringing the development to a standstill. Meanwhile, the use of Large Language Models for small error correction is constantly growing, bringing the attention of an ever-wider audience to these technologies. This study explores a first approach to use Large Language Models to suggest refactoring for architectural smells, with a focus on Cyclic Dependencies smells. We study the use of detailed prompt and Retrieval-Augmented Generation (RAG) to enhance LLMs, and we study local vs cloud LLMs. The results are promising, also validated with a series of interviews with students and developers, and highlight how additional and precise context is key to enhance the use of LLMs to propose refactoring suggestions. A multi-agent approach seems to be more suited when increasing the complexity of the smells.

• Shivashankar, Karthik & Martini, Antonio (2025). MLScent: A Tool for Anti-Pattern Detection in ML Projects. In Lewis, Grace & Muccini, Henry (Ed.), Proceedings of the 2025 IEEE/ACM 4th International Conference on AI Engineering – Software Engineering for AI (CAIN), Ottawa, Ontario, Canada, 27–28 April 2025. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798331502195. p. 267–288. doi: https:/ieeexplore.ieee.org/document/11030059. Full text in Research Archive Show summary

  Machine learning (ML) codebases face unprecedented challenges in maintaining code quality and sustainability as their complexity grows exponentially. While traditional code smell detection tools exist, they fail to address ML-specific issues that can significantly impact model performance, reproducibility, and maintainability. This paper introduces MLScent, a novel static analysis tool that leverages sophisticated Abstract Syntax Tree (AST) analysis to detect anti-patterns and code smells specific to ML projects. MLScent implements 76 distinct detectors across major ML frameworks including TensorFlow (13 detectors), PyTorch (12 detectors), Scikit-learn (9 detectors), and Hugging Face (10 detectors), along with data science libraries like Pandas and NumPy (8 detectors each). The tool's architecture also integrates general ML smell detection (16 detectors), and specialized analysis for data preprocessing and model training workflows. Our evaluation demonstrates MLScent's effectiveness through both quantitative classification metrics and qualitative assessment via user studies feedback with ML practitioners. Results show high accuracy in identifying framework-specific anti-patterns, data handling issues, and general ML code smells across real-world projects.

• Shivashankar, Karthik; Hajj, Ghadi al & Martini, Antonio (2025). Maintainability and Scalability in Machine Learning: Challenges and Solutions. ACM Computing Surveys. ISSN 0360-0300. 57(12). doi: 10.1145/3736751. Full text in Research Archive Show summary

  Rapid advancements in Machine Learning (ML) introduce unique maintainability and scalability challenges. Our research addresses the evolving challenges and identifies ML maintainability and scalability solutions by conducting a thorough literature review of over 17,000 papers, ultimately refining our focus to 124 relevant sources that meet our stringent selection criteria. We present a catalogue of 41 Maintainability and 13 Scalability challenges and solutions across Data, Model Engineering and the overall development of ML applications and systems. This study equips practitioners with insights on building robust ML applications, laying the groundwork for future research on improving ML system robustness at different workflow stages.

• Shivashankar, Karthik; Orucevic, Mili; Kruke, Maren Maritsdatter & Martini, Antonio (2025). BEACon-TD: Classifying Technical Debt and its types across diverse software projects issues using transformers. Journal of Systems and Software. ISSN 0164-1212. 226. doi: 10.1016/j.jss.2025.112435. Full text in Research Archive Show summary

  Technical Debt (TD) identification in software projects issues is crucial for maintaining code quality, reducing long-term maintenance costs, and improving overall project health. This study advances TD identification in issues tracker using transformer-based models, addressing the critical need for accurate and efficient TD identification in large-scale software development. Our methodology employs multiple binary classifiers for TD and its type, combined through ensemble learning, to enhance accuracy and robustness in detecting various forms of TD. We train and evaluate these models on a comprehensive dataset from GitHub Archive Issues (2015–2024), supplemented with industrial data validation. We demonstrate that in-project fine-tuned transformer models significantly outperform task-specific fine-tuned models in TD classification, highlighting the importance of project-specific context in accurate TD identification. Our research also reveals the superiority of specialized binary classifiers over multi-class models for TD and its type identification, enabling more targeted debt resolution strategies. A comparative analysis shows that the smaller DistilRoBERTa model is more effective than larger language models like GPTs for TD classification tasks, especially after fine-tuning, offering insights into efficient model selection for specific TD detection tasks. The study also assesses generalization capabilities using metrics such as MCC, AUC ROC, Recall, and F1 score, focusing on model effectiveness, fine-tuning impact, and relative performance. By validating our approach on out-of-distribution and real-world industrial datasets, we ensure practical applicability, addressing the diverse nature of software projects. This research significantly enhances TD detection and offers a more nuanced understanding of TD types, contributing to improved software maintenance strategies in both academic and industrial settings. The release of our curated dataset aims to stimulate further advancements in TD classification research, ultimately enhancing software project outcomes and development practices by enabling early TD identification and management.

• Kruke, Maren; Martini, Antonio; Cruzes, Daniela Soares & Iovan, Monica (2024). Defining Security Debt: A Case Study Based On Practice. In Pfahl, Dietmar; Huerta, Javier Gonzalez; Klünder, Jil & Anwar, Hina (Ed.), Product-Focused Software Process Improvement: 25th International Conference, PROFES 2024, Tartu, Estonia, December 2–4, 2024, Proceedings. Springer. ISSN 9783031783869. p. 43–59. doi: 10.1007/978-3-031-78386-9_4. Full text in Research Archive
• Haugerud, Kristian; Shivashankar, Karthik & Martini, Antonio (2024). Towards Enhancing Task Prioritization in Software Development Through Transformer-Based Issues Classification, PROFES 2024. Springer. ISSN 9781111111113. p. 255–271. doi: 10.1007/978-3-031-78386-9_17. Full text in Research Archive
• Khakpour, Alireza; Vazquez-Ingelmo, Andrea; Colomo-Palacios, Ricardo; Garcia-Penalvo, Francisco J. & Martini, Antonio (2024). ModelViz: A Model-Driven Engineering Approach for Visual Analytics System Design. IEEE Access. 12, p. 42667–42682. doi: 10.1109/ACCESS.2024.3379268. Full text in Research Archive
• Khakpour, Alireza; Colomo-Palacios, Ricardo; Martini, Antonio & Sánchez-Gordón, Mary (2023). The Use of Domain-Specific Languages for Visual Analytics: A Systematic Literature Review. Technologies. 11(2). doi: 10.3390/technologies11020037. Full text in Research Archive Show summary

  Visual Analytics (VA) is a multidisciplinary field that requires various skills including but not limited to data analytics, visualizations, and the corresponding domain knowledge. Recently, many studies proposed creating and using Domain-Specific Languages (DSLs) for VA in order to abstract complexities and assist designers in developing better VAs for different data domains. However, development methods and types of DSLs vary for different applications and objectives. In this study, we conducted a systematic literature review to overview DSL methods and their intended applications for VA systems. Moreover, the review outlines the benefits and limitations of each of these methods. The aim is to provide decision support for both the research and development communities to choose the most compatible approach for their application. We think the communication of this research delivers a broad figure of previous relevant research and assists with the transfer and adaptation of the results to other domains.

• Skryseth, Daniel; Shivashankar, Karthik; Pilán, Ildikó & Martini, Antonio (2023). Technical Debt Classification in Issue Trackers using Natural Language Processing based on Transformers, Proceedings of the 2023 ACM/IEEE International Conference on Technical Debt (TechDebt). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350311945. p. 92–101. doi: 10.1109/TechDebt59074.2023.00017. Full text in Research Archive
• Martini, Antonio; Besker, Terese; Posch, Torsten & Bosch, Jan (2023). TD Pulse: Assessing the Systematic Management of Technical Debt. IEEE Software. ISSN 0740-7459. 40(3), p. 54–62. doi: 10.1109/MS.2022.3226035. Full text in Research Archive
• Khakpour, Alireza; Colomo-Palacios, Ricardo & Martini, Antonio (2023). Towards a framework for developing visual analytics in supply chain environments. International Journal of Information Systems and Project Management. ISSN 2182-7796. 11(1), p. 52–71. doi: 10.12821/ijispm110103. Full text in Research Archive
• Besker, Terese; Martini, Antonio & Bosch, Jan (2022). The use of incentives to promote technical debt management. Information and Software Technology. ISSN 0950-5849. 142. doi: 10.1016/j.infsof.2021.106740. Full text in Research Archive Show summary

  Context When developing software, it is vitally important to keep the level of technical debt down since, based on several studies, it has been well established that technical debt can lower the development productivity, decrease the developers' morale and compromise the overall quality of the software, among others. However, even if researchers and practitioners working in today's software development industry are quite familiar with the concept of technical debt and its related negative consequences, there has been no empirical research focusing specifically on how software managers actively communicate and manage the need to keep the level of technical debt as low as possible. Objective This study aims to understand how software companies give incentives to manage technical debt. This is carried out by exploring how companies encourage and reward practitioners for actively keeping the level of technical debt down add whether the companies use any forcing or penalising initiatives when managing technical debt. Method As a first step, this paper reports the results of both an online survey providing quantitative data from 258 participants and interviews with 32 software practitioners. As a second step, this study sets out to specifically provide a detailed assessment of additional and in-depth analysis of technical debt management strategies based on an encouraging mindset and attitude from both managers and technical roles to understand how, when and by whom such strategies are adopted in practice. Results Our findings show that having a technical debt management strategy (specially based on encouragement) can significantly impact the amount of technical debt related to the software. Conclusion The result indicates that there is considerable unfulfilled potential to influence how software practitioners can further limit and reduce technical debt by adopting a strategy based explicitly on an encouraging mindset from managers where they also specifically dedicate time and resources for technical debt remediation activities.

• Toledo, Saulo Soares de; Martini, Antonio; Nguyen, Phu Hong & Sjøberg, Dag (2022). Accumulation and Prioritization of Architectural Debt in Three Companies Migrating to Microservices. IEEE Access. 10, p. 37422–37445. doi: 10.1109/ACCESS.2022.3158648. Full text in Research Archive Show summary

  Many companies migrate to microservices because they help deliver value to customers quickly and continuously. However, like any architectural style, microservices are prone to architectural technical debt (ATD), which can be costly if the debts are not timely identified, avoided, or removed. During the early stages of migration, microservice-specific ATDs (MS-ATDs) may accumulate. For example, practitioners may decide to continue using poorly defined APIs in microservices while attempting to maintain compatibility with old functionalities. The riskiest MS-ATDs must be prioritized. Nevertheless, there is limited research regarding the prioritization of MS-ATDs in companies migrating to microservices. This study aims to identify, during migration, which MS-ATDs occur, are the most severe, and are the most challenging to solve. In addition, we propose a way to prioritize these debts. We conducted a multiple exploratory case study of three large companies that were early in the migration process to microservices. We interviewed 47 practitioners with several roles to identify the debts in their contexts. We report the MS-ATDs detected during migration, the MS-ATDs that practitioners estimate to occur in the future, and the MS-ATDs that practitioners report as difficult to solve. We discuss the results in the context of the companies involved in this study. In addition, we used a risk assessment approach to propose a way for prioritizing MS-ATDs. Practitioners from other organizations and researchers may use this approach to provide rankings to help identify and prioritize which MS-ATDs should be avoided or solved in their contexts.

• Brataas, Gunnar; Martini, Antonio; Hanssen, Geir Kjetil & Ræder, Georg (2021). Agile elicitation of scalability requirements for open systems: A case study. Journal of Systems and Software. ISSN 0164-1212. 182. doi: 10.1016/j.jss.2021.111064. Full text in Research Archive Show summary

  Eliciting scalability requirements during agile software development is complicated and poorly described in previous research. This article presents a lightweight artifact for eliciting scalability requirements during agile software development: the ScrumScale model. The ScrumScale model is a simple spreadsheet. The scalability concepts underlying the ScrumScale model are clarified in this design science research, which also utilizes coordination theory. This paper describes the open banking case study, in which a legacy banking system becomes open. This challenges the scalability of this legacy system. The first step in understanding this challenge is to elicit the new scalability requirements. In the open banking case study, key stakeholders from TietoEVRY spent 55 h eliciting the scalability requirements of TietoEVRY’s open banking project. According to TietoEVRY, the ScrumScale model provided a systematic way of producing scalability requirements. For TietoEVRY, the scalability concepts behind the ScrumScale model also offered significant advantages in dialogs with other stakeholders.

• Khakpour, Alireza; Colomo-Palacios, Ricardo & Martini, Antonio (2021). Visual Analytics for Decision Support: A Supply Chain Perspective. IEEE Access. 9, p. 81326–81344. doi: 10.1109/ACCESS.2021.3085496. Full text in Research Archive Show summary

  Supply chain (SC) activities generate huge amount of data that can be used in decision making processes. However, proper data analytics techniques are required to combine, organize, and analyze data from different sources and produce required insights available for decision makers. These techniques promote analytical reasoning of the events and patterns hidden in the data using visualizations, so-called Visual Analytics (VA). Although there is a large number of VA systems to facilitate the process of analysis and decision making, there is a lack of an adequate overview of what already exists in this area for SC management. To address that need, we conducted a systematic literature review to analyze the state of the art in SC VA systems. Particularly, we focus on use cases, the type of the decisions that a VA system intended to support, the type of visualizations employed, the type of analytics used, and the data that has been used for analysis. The goal of this study is to provide SC and VA researchers with an overview of the works carried out in the field of SC VA, helping them to observe latest trends and to recognize existing gaps that need further investigation. Consequently, a mapping between decisions of various SC business processes and their reciprocal visualization techniques and tactics have been provided. Adding to that, VA applications and use cases in SC are identified based on the SC Operation Reference (SCOR) model and underlying decision areas are recognized.

• Olsson, Jesper; Risfelt, Erik; Besker, Terese; Martini, Antonio & Torkar, Richard (2021). Measuring affective states from technical debt. Empirical Software Engineering. ISSN 1382-3256. doi: 10.1007/s10664-021-09998-w. Full text in Research Archive Show summary

  Abstract Context Software engineering is a human activity. Despite this, human aspects are under-represented in technical debt research, perhaps because they are challenging to evaluate. Objective This study’s objective was to investigate the relationship between technical debt and affective states (feelings, emotions, and moods) from software practitioners. Method Forty participants ( N = 40) from twelve companies took part in a mixed-methods approach, consisting of a repeated-measures ( r = 5) experiment ( n = 200), a survey, and semi-structured interviews. From the qualitative data, it is clear that technical debt activates a substantial portion of the emotional spectrum and is psychologically taxing. Further, the practitioners’ reactions to technical debt appear to fall in different levels of maturity. Results The statistical analysis shows that different design smells (strong indicators of technical debt) negatively or positively impact affective states. Conclusions We argue that human aspects in technical debt are important factors to consider, as they may result in, e.g., procrastination, apprehension, and burnout.

• Mårtensson, Torvald; Ståhl, Daniel; Martini, Antonio & Bosch, Jan (2021). The MaLET Model — Maturity Levels for Exploratory Testing, Proceedings of 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665427050. doi: 10.1109/SEAA53835.2021.00019. Full text in Research Archive Show summary

  Based on multiple series of interviews and workshops, this paper presents the MaLET model – a representation of the typical evolution path for companies who successfully adopt exploratory testing. The model provides a step-by-step approach to systematically improve exploratory testing over time, and shows how and why capabilities may regress. The MaLET model was validated through a series of interviews with 20 interviewees from eight case study companies in separate industry segments. The interviews also revealed examples on improvement initiatives that had failed in the companies, showing that improvement initiatives tend to fail if they are not planned in an order corresponding to the maturity levels in the model. The MaLET model was well received by the interviewees during the validation, describing the model as sound, relevant and useful in practice.

• Lenarduzzi, Valentina; Martini, Antonio; Saarimaki, Nyyti & Tamburri, Damian Andrew (2021). Technical Debt Impacting Lead-Times: An Exploratory Study, Proceedings of 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665427050. doi: 10.1109/SEAA53835.2021.00032. Full text in Research Archive Show summary

  Background: Technical Debt is a consolidated notion in software engineering research and practice. However, the estimation of its impact (interest of the debt) is still imprecise and requires heavy empirical and experimental inquiry. Objective: We aim at developing a data-driven approach to calculate the interest of Technical Debt in terms of delays in resolving affected tasks.Method: We conducted a case study to estimate the Technical Debt interest by analyzing its association with the lead time variation of resolving related Jira issues.Results: Data-driven approaches could significantly change the Technical Debt estimation and improve the removing Technical Debt prioritization. Our case study shows that the presence of Code Technical Debt did not affect the lead time for resolving the issues.Conclusion: Future works include the further refinement of this approach and its application to a larger data-set and on different type of issues.

• Ciolkowski, Marcus; Lenarduzzi, Valentina & Martini, Antonio (2021). 10 Years of Technical Debt Research and Practice: Past, Present, and Future. IEEE Software. ISSN 0740-7459. doi: 10.1109/MS.2021.3105625. Full text in Research Archive
• Toledo, Saulo Soares De; Martini, Antonio & Sjøberg, Dag (2021). Identifying architectural technical debt, principal, and interest in microservices: A multiple-case study. Journal of Systems and Software. ISSN 0164-1212. 177. doi: 10.1016/j.jss.2021.110968. Full text in Research Archive Show summary

  Background: Using a microservices architecture is a popular strategy for software organizations to deliver value to their customers fast and continuously. However, scientific knowledge on how to manage architectural debt in microservices is scarce. Objectives: In the context of microservices applications, this paper aims to identify architectural technical debts (ATDs), their costs, and their most common solutions. Method: We conducted an exploratory multiple case study by conducting 25 interviews with practitioners working with microservices in seven large companies. Results: We found 16 ATD issues, their negative impact (interest), and common solutions to repay each debt together with the related costs (principal). Two examples of critical ATD issues found were the use of shared databases that, if not properly planned, leads to potential breaks on services every time the database schema changes and bad API designs, which leads to coupling among teams. We identified ATDs occurring in different domains and stages of development and created a map of the relationships among those debts. Conclusion: The findings may guide organizations in developing microservices systems that better manage and avoid architectural debts.

• Mårtensson, Torvald; Ståhl, Daniel; Martini, Antonio & Bosch, Jan (2021). Efficient and effective exploratory testing of large-scale software systems. Journal of Systems and Software. ISSN 0164-1212. 174, p. 1–15. doi: 10.1016/j.jss.2020.110890. Full text in Research Archive
• Lenarduzzi, Valentina; Daly, Jeremy; Martini, Antonio; Panichella, Sebastiano & Tamburri, Damian Andrew (2021). Toward a Technical Debt Conceptualization for Serverless Computing. IEEE Software. ISSN 0740-7459. 38(1), p. 40–47. doi: 10.1109/MS.2020.3030786. Full text in Research Archive
• Toledo, Saulo Soares de; Martini, Antonio; Sjøberg, Dag; Przybyszewska, Agata & Frandsen, Johannes Skov (2021). Reducing Incidents in Microservices by Repaying Architectural Technical Debt, Proceedings of 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665427050. doi: 10.1109/SEAA53835.2021.00033. Full text in Research Archive Show summary

  Architectural technical debt (ATD) may create a substantial extra effort in software development, which is called interest. There is little evidence about whether repaying ATD in microservices reduces such interest. Objectives: We wanted to conduct a first study on investigating the effect of removing ATD on the occurrence of incidents in a microservices architecture. Method: We conducted a quantitative and qualitative case study of a project with approximately 1000 microservices in a large, international financing services company. We measured and compared the number of software incidents of different categories before and after repaying ATD. Results: The total number of incidents was reduced by 84%, and the numbers of critical- and high-priority incidents were both reduced by approximately 90% after the architectural refactoring. The number of incidents in the architecture with the ATD was mainly constant over time, but we observed a slight increase of low priority incidents related to inaccessibility and the environment in the architecture without the ATD. Conclusion: This study shows evidence that refactoring ATDs, such as lack of communication standards, poor management of dead-letter queues, and the use of inadequate technologies in microservices, reduces the number of critical- and high-priority incidents and, thus, part of its interest, although some low priority incidents may increase.

• Cico, Orges; Besker, Terese; Martini, Antonio; Duc, Anh Nguyen; Souza, Renata & Bosch, Jan (2021). Toward a Technical Debt Relationship with the Pivoting of Growth Phase Startups. In Ardito, Luca; Jedlitschka, Andreas; Morisio, Maurizio & Torchiano, Marco (Ed.), Product-Focused Software Process Improvement. Springer. ISSN 9783030914516. p. 265–280. doi: 10.1007/978-3-030-91452-3_18. Full text in Research Archive Show summary

  Context: Pivot has been a common strategical tactic of startups by shifting course of actions to adapt to environmental changes to the companies. Among many factors influencing the decisions of pivot or preserve, technical characteristics of the product and its evolution are possible triggering factors. We have learned that technical debt is an inherent phenomenon in startups that hinders later growth. However, we do not yet know how technical debt might lead to pivoting in startups and what TD processes we observe in different pivoting scenarios. Aim: Our goal is to evaluate how technical debt influences pivoting in growth-phase startups. Methodology: We conducted an empirical study on 11 software startups in Norway and Brazil and analyzed qualitative data using thematic analysis. Results: We identified three ways that technical debt influences pivoting: (1) direct, (2) indirect, and (3) no-influence. Managing and avoiding technical debt significantly reduces the likelihood of technology pivoting and restrains indirect effects on other pivoting types. Contribution: Our study will enable practitioners to address the influence of technical debt on pivoting in growth-phase software startups. Future researchers can benefit from our findings by conducting exploratory studies and providing educated recommendations.

• Avgeriou, Paris C.; Taibi, Davide; Ampatzoglou, Apostolos; Fontana, Francesca Arcelli; Besker, Terese & Chatzigeorgiou, Alexander [Show all 14 contributors for this article] (2021). An Overview and Comparison of Technical Debt Measurement Tools. IEEE Software. ISSN 0740-7459. 38(3), p. 61–71. doi: 10.1109/MS.2020.3024958. Full text in Research Archive
• Pavlović, Milena; Scheffer, Lonneke; Motwani, Keshav; Kanduri, Chakravarthi; Kompova, Radmila & Vazov, Nikolay Aleksandrov [Show all 42 contributors for this article] (2021). The immuneML ecosystem for machine learning analysis of adaptive immune receptor repertoires. Nature Machine Intelligence. 3(11), p. 936–944. doi: 10.1038/s42256-021-00413-z. Full text in Research Archive
• Lenarduzzi, Valentina; Besker, Terese; Taibi, Davide; Martini, Antonio & Fontana, Francesca Arcelli (2021). A systematic literature review on Technical Debt prioritization: Strategies, processes, factors, and tools. Journal of Systems and Software. ISSN 0164-1212. 171, p. 1–16. doi: 10.1016/j.jss.2020.110827. Full text in Research Archive
• Besker, Terese; Martini, Antonio & Bosch, Jan (2020). Carrot and stick approaches when managing technical debt. In Clemente, Izurieta; Matthias, Galster & Felderer, Michael (Ed.), TechDebt '20: Proceedings of the 3rd International Conference on Technical Debt. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781450379601. p. 21–30. doi: 10.1145/3387906.3388619. Full text in Research Archive
• Josep, Burgaya Pujols; Bas, Pieter; Martínez-Fernández, Silverio; Martini, Antonio & trendowicz, adam (2020). Skuld: a self-learning tool for impact-driven technical debt management. In Clemente, Izurieta; Matthias, Galster & Felderer, Michael (Ed.), TechDebt '20: Proceedings of the 3rd International Conference on Technical Debt. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781450379601. p. 113–114. doi: 10.1145/3387906.3388626. Full text in Research Archive
• Toledo, Saulo Soares De; Martini, Antonio & Sjøberg, Dag I. K. (2020). Improving Agility by Managing Shared Libraries in Microservices. In Paasivaara, Maria & Kruchten, Philippe (Ed.), Lecture Notes in Business Information Processing, vol 396. Springer. ISSN 9783030588571. p. 195–202. doi: 10.1007/978-3-030-58858-8_20. Full text in Research Archive Show summary

  Using microservices is a way of supporting an agile architecture. However, if the microservices development is not properly managed, the teams’ development velocity may be affected, reducing agility and increasing architectural technical debt. This paper investigates how to manage the use of shared libraries in microservices to improve agility during development. We interviewed practitioners from four large international companies involved in microservices projects to identify problems when using shared libraries. Our results show that the participating companies had issues with shared libraries as follows: coupling among teams, delays on fixes due to overhead on libraries development teams, and need to maintain many versions of the libraries. Our results highlight that the use of shared libraries may hinder agility on microservices. Thus, their use should be restricted to situations where shared libraries cannot be replaced by a microservice and the costs of replicating the code on each service is very high.

• Martini, Antonio; Besker, Terese & Bosch, Jan (2020). Process Debt: a First Exploration. In -, - (Eds.), 2020 27TH Asia-Pacific Software Engineering Conference (APSEC). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781728195544. p. 316–325. doi: 10.1109/APSEC51365.2020.00040. Full text in Research Archive
• Mamun, Md Abdullah Al; Martini, Antonio; Staron, Miroslaw; Berger, Christian & Hansson, Jorgen (2019). Evolution of technical debt: An exploratory study. CEUR Workshop Proceedings. 2476(6), p. 87–102. Full text in Research Archive
• Besker, Terese; Martini, Antonio & Bosch, Jan (2019). Technical debt triage in backlog management. In Avgeriou, Paris & Schmid, Klaus (Ed.), Proceedings of the Second International Conference on Technical Debt. IEEE Press. ISSN 9781728133713. doi: 10.1109/TechDebt.2019.00010. Full text in Research Archive Show summary

  Remediation of technical debt through regular refactoring initiatives is considered vital for the software system's long and healthy life. However, since today's software companies face increasing pressure to deliver customer value continuously, the balance between spending developer time, effort, and resources on implementing new features or spending it on refactoring of technical debt becomes vital. The goal of this study is to explore how the prioritization of technical debt is carried out by practitioners within today's software industry. This study also investigates what factors influence the prioritization process and its related challenges. This paper reports the results of surveying 17 software practitioners, together with follow-up interviews with them. Our results show that there is no uniform way of prioritizing technical debt and that it is commonly done reactively without applying any explicit strategies. Often, technical debt issues are managed and prioritized in a shadow backlog, separate from the official sprint backlog. This study was also able to identify several different challenges related to prioritizing technical debt, such as the lack of quantitative information about the technical debt items and that the refactoring of technical debt issues competes with the implementation of customer requirements.

• Martini, Antonio; Stray, Viktoria & Moe, Nils Brede (2019). Technical-, social- and process debt in large-scale agile: An exploratory case-study. Lecture Notes in Business Information Processing. ISSN 1865-1348. 364, p. 112–119. doi: 10.1007/978-3-030-30126-2_14. Full text in Research Archive Show summary

  Large-scale agile projects bring inter-teams interaction challenges. Teams need to be autonomous, but often crosscutting concerns affect many teams. If the teams fail to collaborate on these concerns, the negative effects might hinder agility in the medium and long term. In other words, the organization and the system accumulate debt, on which the teams pay a high interest. Such debt must therefore be prioritized and “repaid” timely. We conducted a case study with interviews, observations and document analysis. Via both team- and large-scale retrospectives we investigated how teams coordinate and discuss Technical-, Social- and Process Debts.

• Toledo, Saulo Soares De; Martini, Antonio; Przybyszewska, Agata & Sjøberg, Dag (2019). Architectural Technical Debt in Microservices: A Case Study in a Large Company. In Avgeriou, Paris & Schmid, Klaus (Ed.), Proceedings of the Second International Conference on Technical Debt. IEEE Press. ISSN 9781728133713. doi: 10.1109/TechDebt.2019.00026. Full text in Research Archive Show summary

  Introduction: Software companies aim to achieve continuous delivery to constantly provide value to their customers. A popular strategy is to use microservices architecture. However, such an architecture is also subject to debt, which hinders the continuous delivery process and thus negatively affects the software released to the customers. Objectives: The aim of this study is to identify issues, solutions and risks related to Architecture Technical Debt in microservices. Method: We conducted an exploratory case study of a real life project with about 1000 services in a large, international company. Through qualitative analysis of documents and interviews, we investigated Architecture Technical Debt in the communication layer of a system with microservices architecture. Results: Our main contributions are a list of Architecture Technical Debt issues specific for the communication layer in a system with microservices architecture, as well as their associated negative impact (interest), a solution to repay the debt and the its cost (principal). Among the found Architecture Technical Debt issues were the existence of business logic in the communication layer and a high amount of point-to-point connections between services. The studied solution consists of the implementation of different canonical models specific to different domains, the removal of business logic from the communication layer, and migration from services to use the communication layer correctly. We also contributed with a list of possible risks that can affect the payment of the debt, as lack of funding and inadequate prioritization. Conclusion: We found issues, solutions and possible risks that are specific for microservices architectures not yet encountered in the current literature. Our results may be useful for practitioners that want to avoid or repay Technical Debt in their microservices architecture.

• Besker, Terese; Martini, Antonio & Bosch, Jan (2019). Software developer productivity loss due to technical debt?A replication and extension study examining developers? development work. Journal of Systems and Software. ISSN 0164-1212. 156, p. 41–61. doi: 10.1016/j.jss.2019.06.004. Full text in Research Archive
• Hanssen, Geir Kjetil; Brataas, Gunnar & Martini, Antonio (2019). Identifying Scalability Debt in Open Systems. In Avgeriou, Paris & Schmid, Klaus (Ed.), Proceedings of the Second International Conference on Technical Debt. IEEE Press. ISSN 9781728133713. p. 48–52. doi: 10.1109/TechDebt.2019.00014. Full text in Research Archive Show summary

  Architectural technical debt can be generated by changes in the business and the environment of an organization. In this paper, we emphasize the change in scalability requirements due to new regulations. Scalability is the ability of a system to handle an increased workload. For complex systems that are abruptly exposed via open interfaces and hence a greater workload, the scalability requirements may quickly increase, leading to technical debt. We term this scalability debt. This paper describes scalability triage, a light-weight, novel technique for identifying scalability threats as a form of technical debt. We illustrate this technique with an open banking case from a large software organization. Open banking is partly caused by the new European PSD2 regulative that enforce banks to open interfaces to unknown third-party actors. Banking systems are well-established, mature systems. However, with the advent of open banking and PSD2, the workload may quickly rocket. This leads to tougher scalability requirements and accumulated architectural debt, despite previously sound architectural decisions. Using scalability triage, such risks may be identified fast. It will then be possible to prevent this form of technical debt with timely reengineering.

• Lenarduzzi, Valentina; Martini, Antonio; Taibi, Davide & Tamburri, Damian Andrew (2019). Towards surgically-precise technical debt estimation: early results and research roadmap. In Fontana, Francesca Arcelli (Eds.), Proceedings of the 3rd ACM SIGSOFT International Workshop on Machine Learning Techniques for Software Quality Evaluation, MaLTeSQuE@ESEC/SIGSOFT FSE 2019. Association for Computing Machinery (ACM). ISSN 9781450368551. p. 37–42. doi: 10.1145/3340482.3342747. Full text in Research Archive Show summary

  The concept of technical debt has been explored from many perspectives but its precise estimation is still under heavy empirical and experimental inquiry. We aim to understand whether, by harnessing approximate, data-driven, machine-learning approaches it is possible to improve the current techniques for technical debt estimation, as represented by a top industry quality analysis tool such as SonarQube. For the sake of simplicity, we focus on relatively simple regression modelling techniques and apply them to modelling the additional project cost connected to the sub-optimal conditions existing in the projects under study. Our results shows that current techniques can be improved towards a more precise estimation of technical debt and the case study shows promising results towards the identification of more accurate estimation of technical debt.

• Ståhl, Daniel; Martini, Antonio & Mårtensson, Torvald (2019). Big bangs and small pops: on critical cyclomatic complexity and developer integration behavior. In Sharp, Helen & Whalen, Mike (Ed.), Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice, ICSE (SEIP) 2019. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781728117607. p. 81–90. doi: 10.1109/ICSE-SEIP.2019.00017. Full text in Research Archive Show summary

  Continuous integration is believed by many to improve software quality, including cyclomatic complexity. In this paper an exploratory study investigates the relationship between continuous integration behavior among developers and the cyclomatic complexity - particularly at critical levels - of the source code they commit, as a correct understanding of the consequences of a software development practice, particularly one as popular as continuous integration, is of crucial importance to industry professionals when deciding on improvement efforts. It is found that developer behavior differs significantly, as does their knowledge and awareness of cyclomatic complexity as a concept. We find that even though large commits ("big bangs") contribute proportionately slightly more to cyclomatic complexity, the effect is much less pronounced for critical cyclomatic complexity. It is also found that complex code tends to overlap with high levels of change activity, and that some developers deliberately change their integration behavior in the face of such hotspots, as a strategy to avoid merge conflicts.

• Besker, Terese; Martini, Antonio & Bosch, Jan (2019). How Regulations of Safety-Critical Software Affect Technical Debt. In Staron, Miroslaw; Capilla, Rafael & Skavhaug, Amund (Ed.), 2019 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781728134215. doi: 10.1109/SEAA.2019.00020. Full text in Research Archive Show summary

  In recent years in the software industry, the use of safety-critical software is increasing at a rapid rate. However, little is known about the relationship between safety-critical regulations and the management of technical debt. The research is based on interviews with 19 practitioners working in different safety-critical domains implementing software according to different safety regulation standards. The results are three-fold. First, the result shows that performing technical debt refactoring tasks in safety-critical software requires several additional activities and costs, compared to non-safety-critical software. This study has also identified several negative effects due to the impact of these regulatory requirements. Second, the results show that the safety-critical regulations strengthen the implementation of both source code and architecture and thereby initially limit the introduction of technical debt. However, at the same time, the regulations also force the software companies to perform later suboptimal work-around solutions that are counterproductive in achieving a high-quality software since the regulations constrain the possibility of performing optimal TD refactoring activities. Third, the result shows that technical debt refactoring decisions are heavily weighed on the costs associated with the application's recertification process and that these decisions seldom include the benefits of the refactoring activities in a structured way.

• Mårtensson, Torvald; Martini, Antonio; Ståhl, Daniel & Bosch, Jan (2019). Excellence in Exploratory Testing: Success Factors in Large-Scale Industry Projects. In Franch, Xavier; Männistö, Tomi & Martínez-Fernández, Silverio (Ed.), Product-Focused Software Process Improvement - 20th International Conference, PROFES 2019, Barcelona, Spain, November 27-29, 2019, Proceedings. Springer. ISSN 9783030353322. doi: 10.1007/978-3-030-35333-9_21. Full text in Research Archive Show summary

  Based on interviews with 20 interviewees from four case study companies, this paper presents a list of key factors that enable efficient and effective exploratory testing of large-scale software systems. The nine factors are grouped into four themes: “The testers’ knowledge, experience and personality”, “Purpose and scope”, “Ways of working” and “Recording and reporting”. According to the interviewees, exploratory testing is a more creative way to work for the testers, and was therefore considered to make better use of the testers. Exploratory testing was also described as a good way to test system-wide and to test large-scale systems, especially exploratory testing with an end-user perspective. The identified key factors were confirmed by a series of follow-up interviews with the 20 interviewees and a cross-company workshop with 14 participants. This strengthens the generalizability of the findings, supporting that the list of key factors can be applied to projects in a large segment of the software industry. This paper also presents the results from a systematic literature review including 129 publications related to exploratory testing. No publication were found that summarizes the key factors that enable efficient and effective exploratory testing, which supports the novelty of the findings

• Besker, Terese; Martini, Antonio & Bosch, Jan (2018). Technical debt cripples software developer productivity: A longitudinal study on developers' daily software development work. Proceedings - International Conference on Software Engineering. ISSN 0270-5257. p. 105–114. doi: 10.1145/3194164.3194178. Full text in Research Archive Show summary

  Software companies need to continuously deliver customer value, both from a short- and long-term perspective. However, software development can be impeded by what has been described as Technical Debt (TD). The aim of this study is to explore the negative consequences of TD in terms of wasted software development time. This study also investigates on which additional activities this wasted time is spent and whether different types of TD impact the wasted time differently. This study also sets out to examine the benefits of tracking and communicating the amount of wasted time, both from a developer's and manager's perspective. This paper reports the results of a longitudinal study, surveying 43 software developers, together with follow-up interviews with 16 industrial software practitioners. The analysis of the reported wasted time revealed that developers waste, on average, 23% of their development time due to TD and that they are frequently forced to introduce new TD due to already existing TD. The most common activity on which additional time is spent is performing additional testing.

• Martini, Antonio (2018). AnaConDebt: A Tool to Assess and Track Technical Debt. Proceedings - International Conference on Software Engineering. ISSN 0270-5257. p. 55–56. doi: 10.1145/3194164.3194185. Full text in Research Archive Show summary

  It is challenging to assess and manage Technical Debt. Technical Debt is avoided or refactored if the long-term benefits, such as preventing extra-costs, exceed the cost of repaying the debt. Some tools have been recently proposed for the identification of Technical Debt, but most of them do not help in assessing the cost-benefits of repaying the Debt. Besides, it is challenging to track, visualize and plan Technical Debt refactoring systematically. Although practitioners might use simple tracking tools, calculating and communicating Technical Debt is currently not supported. Based on the results of previous research, combined with several practical experiences in collaboration with large software companies, we have developed and evaluated a lightweight tool, AnaConDebt, to track and assess Technical Debt.

• Besker, Terese; Martini, Antonio; Lokuge, Rumesh Edirisooriya; Blincoe, Kelly & Bosch, Jan (2018). Embracing technical debt, from a startup company perspective. In -, - (Eds.), 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781538678701. p. 415–425. doi: 10.1109/ICSME.2018.00051. Full text in Research Archive Show summary

  Software startups are typically under extreme pressure to get to market quickly with limited resources and high uncertainty. This pressure and uncertainty is likely to cause startups to accumulate technical debt as they make decisions that are more focused on the short-term than the long-term health of the codebase. However, most research on technical debt has been focused on more mature software teams, who may have less pressure and, therefore, reason about technical debt very differently than software startups. In this study, we seek to understand the organizational factors that lead to and the benefits and challenges associated with the intentional accumulation of technical debt in software startups. We interviewed 16 professionals involved in seven different software startups. We find that the startup phase, the experience of the developers, software knowledge of the founders, and level of employee growth are some of the organizational factors that influence the intentional accumulation of technical debt. In addition, we find the software startups are typically driven to achieve a "good enough level," and this guides the amount of technical debt that they intentionally accumulate to balance the benefits of speed to market and reduced resources with the challenges of later addressing technical debt.

• Martini, Antonio; Besker, Terese & Bosch, Jan (2018). Technical Debt tracking: Current state of practice: A survey and multiple case study in 15 large organizations. Science of Computer Programming. ISSN 0167-6423. 163, p. 42–61. doi: 10.1016/j.scico.2018.03.007. Full text in Research Archive Show summary

  Large software companies need to support continuous and fast delivery of customer value both in the short and long term. However, this can be hindered if both the evolution and maintenance of existing systems are hampered by Technical Debt. Although a lot of theoretical work on Technical Debt has been produced recently, its practical management lacks empirical studies. In this paper, we investigate the state of practice in several companies to understand what the cost of managing TD is, what tools are used to track TD, and how a tracking process is introduced in practice. We combined two phases: a survey involving 226 respondents from 15 organizations and an in-depth multiple case study in three organizations including 13 interviews and 79 Technical Debt issues. We selected the organizations where Technical Debt was better tracked in order to distill best practices. We found that the development time dedicated to managing Technical Debt is substantial (an average of 25% of the overall development), but mostly not systematic: only a few participants (26%) use a tool, and only 7.2% methodically track Technical Debt. We found that the most used and effective tools are currently backlogs and static analyzers. By studying the approaches in the companies participating in the case study, we report how companies start tracking Technical Debt and what the initial benefits and challenges are. Finally, we propose a Strategic Adoption Model for the introduction of tracking Technical Debt in software organizations.

• Martini, Antonio; Fontana, Francesca Arcelli; Biaggi, Andrea & Roveda, Riccardo (2018). Identifying and Prioritizing Architectural Debt Through Architectural Smells: A Case Study in a Large Software Company. In Cuesta, Carlos E.; Garlan, David & Pérez, Jennifer (Ed.), 12th European Conference on Software Architecture, ECSA 2018, Madrid, Spain, September 24–28, 2018, Proceedings. Springer. ISSN 9783030007607. p. 320–335. doi: 10.1007/978-3-030-00761-4_21. Full text in Research Archive Show summary

  Architectural technical debt can have a huge impact on software maintainability and evolution. Hence, different architectural violations, detected as architectural smells, need to be identified and refactored. In this paper, we conducted a multiple case-study on several architectural smells detected in four industrial projects. We conducted an in-depth investigation with a questionnaire, interviews and thorough inspection of the code with the practitioners. We evaluated the negative impact of the technical debt detected by the architectural smells, their difficulty to be refactored and the usefulness of the detection tool. The results show that practitioners appreciated the help of automatic detection, and that they prioritize refactoring architectural debt that causes more negative impact despite the higher refactoring effort.

• Martini, Antonio; Sikander, Erik & Madlani, Niel (2018). A semi-automated framework for the identification and estimation of Architectural Technical Debt: A comparative case-study on the modularization of a software component. Information and Software Technology. ISSN 0950-5849. 93, p. 264–279. doi: 10.1016/j.infsof.2017.08.005. Full text in Research Archive

View all works in NVA

• Wang, Xiaofeng; Martini, Antonio; Duc, Anh Nguyen & Stray, Viktoria (2021). Software Business: 12th International Conference, ICSOB 2021, Drammen, Norway, December 2–3, 2021, Proceedings. Springer. ISBN 9783030919825. 255 p. Full text in Research Archive
• Martini, Antonio; Wimmer, Manuel & Skavhaug, Amund (2020). Proceedings of 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2020). IEEE (Institute of Electrical and Electronics Engineers). ISBN 9781728195339. 600 p. Full text in Research Archive

View all works in NVA

• Martini, Antonio (2026). From Whispers to Strategy: the Benefits of Visualizing Technical Debt Management (TechDebt 2026 - TechDebt Program) - TechDebt 2026. Full text in Research Archive Show summary

  Technical debt is pervasive, costly, and too often managed by gut feelings. Effective management requires visibility. However, existing technical debt visualization tools are too code‑centric and fragmented: they are tied to specific languages, rely on static metrics that miss context, and seldom reshape how organizations prioritize or act. Practitioners wish for more and better Technical Debt visualization. This keynote will move beyond metrics like the isolated complexity scores and code duplications to cover more holistic project- and organization-wide visualizations that give a stronger decision support to manage Technical Debt. The keynote draws on more than a decade of research in collaboration led by Prof. Martini with 20+ international companies based in North Europe such as Visma, Akva Group, Knowit, Vy and Siemens. It will showcase visualizations from tools like TD Pulse, DebtQuest and novel AI-powered solutions integrating multiple data sources in meaningful insights, that reveal gaps in organisational scaffolding and practices, inter- and cross-team misalignment, role friction, issue resolution patterns, and their ties to KPIs. The results of our latest research, an ongoing three-year Norwegian innovation project, shows that these visualizations have a strong learning effect and consistently help assessing and improving the management of technical debt in various organizations.

• Martini, Antonio & Shivashankar, Karthik (2024). Better Software for All: Using LLMs to Improve Software Development Practices. Full text in Research Archive Show summary

  The use of AI is becoming ubiquitous. In software development for example, tasks can be (semi)-automated using AI and software organizations already report benefits from using AI. In our work, we have focused on improving the quality of software by creating fine-tuned models that can aid software practitioners in some key tasks. We will show how we created such models, how they might be a better choice than using out-of-the-shelf LLMs such as ChatGPT, and when we are with their practical application in software organizations.

• Martini, Antonio & Helleve, Torstein (2024). Brukar maskinlæring for betre handtering av IT-etterslep. Digi.no. Full text in Research Archive
• Martini, Antonio (2024). Technical Debt and AI. Full text in Research Archive
• Martini, Antonio (2024). Introduction to technical debt. Full text in Research Archive
• Martini, Antonio & Helleve, Torstein (2024). Ubehandla teknisk gjeld kan ha store konsekvensar for samfunnet. https://forskning.no/. Full text in Research Archive
• Martini, Antonio & Helleve, Torstein (2024). Brukar maskinlæring for betre handtering av IT-etterslep. Titan.uio.no. Full text in Research Archive
• Martini, Antonio (2023). Do we know our technical debt better than AI? Full text in Research Archive
• Martini, Antonio (2023). Do we know our technical debt better than AI? Full text in Research Archive
• Martini, Antonio (2023). Technical Debt Trends and Impact of AI. Full text in Research Archive
• Wang, Xiaofeng; Martini, Antonio; Nguyen, Anh Duc; Stray, Viktoria & Wnuk, Kryzstof (2023). Special section on IST for ICSOB2021. Information and Software Technology. ISSN 0950-5849. 160. doi: 10.1016/j.infsof.2023.107243. Full text in Research Archive
• Martini, Antonio (2023). Introduction to Technical Debt. Full text in Research Archive
• Martini, Antonio; Seaman, Carolyn; Chatzigeorgiou, Alexander; Ciolkowski, Marcus & Lenarduzzi, Valentina (2021). Message from the Chair of TechDebt 2021. IEEE Xplore Digital Library. doi: 10.1109/TechDebt52882.2021.00005. Full text in Research Archive
• Shivashankar, Karthik; Martini, Antonio & Sandve, Geir Kjetil Ferkingstad (2025). The Dual Role of Machine Learning in Technical Debt Management. Applying Machine Learning for Identification While Analyzing Debt within ML Systems. Universitetet i Oslo. Full text in Research Archive Show summary

  Context: The management of Technical Debt (TD) is a critical challenge in software engineering, directly impacting project costs and long-term viability. The proliferation of Machine Learning (ML) systems introduces a complex duality: they present novel forms of debt while simultaneously offering powerful automated techniques for its identification and mitigation. Problem: Existing TD management practices often rely on manual, subjective processes or generic tools that fail to scale or address the specific quality challenges inherent in modern codebases, particularly those involving Python and ML. A comprehensive, automated, and context-aware approach is needed to effectively navigate this dual role of ML in TD. Methodology: This thesis addresses this gap through a multi-faceted research program. We developed a suite of automated tools leveraging Advanced Natural Language Processing (NLP) with transformer models for analyzing textual artifacts (e.g., issue reports) and domain-specific static code analysis for dissecting Python and ML codebases. This practical tool development is complemented by a systematic literature review to map the landscape of quality challenges in ML systems and an exploratory study on using Large Language Models (LLMs) for proactive code improvement. Contributions: The research yields six interconnected contributions. We demonstrate that specialized, fine-tuned transformer models (BEAConTD, QualiTagger) consistently outperform larger, general-purpose models for classifying TD and ISO 25010 quality attributes. We introduce domain-specific static analysis tools, PyExamine and MLScent, that effectively detect 49 traditional Python smells and 76 unique ML-specific anti-patterns, respectively. Further, we establish that fine-tuned LLMs can verifiably improve code maintainability metrics. These practical tools are underpinned by a comprehensive mapping of 41 maintainability and 13 scalability challenges in ML workflows, providing a foundational reference for the field. All tools and datasets are released as open-source artifacts. Implications: Collectively, this work represents a significant advancement in automated software quality management. It provides practitioners with a suite of validated tools (TD-Suite, PyExamine, MLScent, QualiTagger) for objective, scalable, and data-driven TD management. The findings establish new research directions at the intersection of software engineering and artificial intelligence, enabling quality control that can keep pace with the complexity of modern software.

View all works in NVA