Dag Sjøberg

• Florea, Raluca Madalina; Stray, Viktoria & Sjøberg, Dag (2023). On the roles of software testers: An exploratory study. Journal of Systems and Software. ISSN 0164-1212. 204. doi: 10.1016/j.jss.2023.111742. Full text in Research Archive Show summary

  Context: Software development organizations need testers with high skill levels in a broad range of technical areas and application domains. Accordingly, we need a better understanding of how testers meet such skill demands in the practice of their role. Objective: This work aims to deepen the understanding of the typical tester role. Method: We performed a thematic analysis of 19 in-depth, semi-structured interviews with software testers working in various industries. To investigate employers’ views on such roles, we conducted a thematic analysis of 400 job ads. Results: From the interviews, we identified five subroles of software testers: domain-specific tester, test automation specialist, test infrastructure specialist, user experience tester, and test manager. Most of the practitioners preferred to develop skills and act in one subrole. In contrast, most of the job ads requested that testers act in multiple subroles. Conclusion: Our findings provide a deeper understanding of the tester role, which may guide testers in their acquisition of skills and employers in the recruiting of testers.

• Sjøberg, Dag & Bergersen, Gunnar Rye (2023). Improving the Reporting of Threats to Construct Validity. In Babar, Muhammad Ali (Eds.), EASE '23: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. Association for Computing Machinery (ACM). ISSN 9798400700446. doi: 10.1145/3593434.3593449. Full text in Research Archive Show summary

  Background: Construct validity concerns the use of indicators to measure a concept that is not directly measurable. Aim: This study intends to identify, categorize, assess and quantify discussions of threats to construct validity in empirical software engineering literature and use the findings to suggest ways to improve the reporting of construct validity issues. Method: We analyzed 83 articles that report human-centric experiments published in five top-tier software engineering journals from 2015 to 2019. The articles’ text concerning threats to construct validity was divided into segments (the unit of analysis) based on predefined categories. The segments were then evaluated regarding whether they clearly discussed a threat and a construct. Results: Three-fifths of the segments were associated with topics not related to construct validity. Two-thirds of the articles discussed construct validity without using the definition of construct validity given in the article. The threats were clearly described in more than four-fifths of the segments, but the construct in question was clearly described in only two-thirds of the segments. The construct was unclear when the discussion was not related to construct validity but to other types of validity. Conclusions: The results show potential for improving the understanding of construct validity in software engineering. Recommendations addressing the identified weaknesses are given to improve the awareness and reporting of CV.

• Holgeid, Knut Kjetil; Krogstie, John; Mikalef, Patrick; Saur, Eirik Eide & Sjøberg, Dag (2022). Benefits management and Information Technology work distribution. IET Software. ISSN 1751-8806. 16(4), p. 438–454. doi: 10.1049/sfw2.12062. Full text in Research Archive Show summary

  Organisations spend much money on Information Technology (IT) development and maintenance activities with the intention that these activities will create results that enable benefits for the organisations. This paper seeks to understand potential associations between IT development and maintenance activities and the adoption of benefits management practices to realise value for the organization. The aim is also to uncover potential differences between public and private organisations. We surveyed 86 Norwegian public and private organisations, including data collected in similar surveys every five years since 1993. For the period between 1998 and 2018, we observe a stable pattern of IT work distribution. We found that organisations that managed benefits put more effort into advancing functionality for the end-users than other organisations, and they realised more benefits. This advantage was particularly true for organisations that managed benefits beyond the early stages of the development lifecycle. Private organisations both managed and realised benefits to a larger extent than public organisations. Our findings can enable organisations to be evidence-based when choosing management practices to achieve a higher return on investments in IT development and maintenance activities.

• Sjøberg, Dag & Bergersen, Gunnar Rye (2022). Construct Validity in Software Engineering. IEEE Transactions on Software Engineering. ISSN 0098-5589. 49(3), p. 1374–1396. doi: 10.1109/TSE.2022.3176725. Full text in Research Archive
• Toledo, Saulo Soares de; Martini, Antonio; Nguyen, Phu Hong & Sjøberg, Dag (2022). Accumulation and Prioritization of Architectural Debt in Three Companies Migrating to Microservices. IEEE Access. 10, p. 37422–37445. doi: 10.1109/ACCESS.2022.3158648. Full text in Research Archive Show summary

  Many companies migrate to microservices because they help deliver value to customers quickly and continuously. However, like any architectural style, microservices are prone to architectural technical debt (ATD), which can be costly if the debts are not timely identified, avoided, or removed. During the early stages of migration, microservice-specific ATDs (MS-ATDs) may accumulate. For example, practitioners may decide to continue using poorly defined APIs in microservices while attempting to maintain compatibility with old functionalities. The riskiest MS-ATDs must be prioritized. Nevertheless, there is limited research regarding the prioritization of MS-ATDs in companies migrating to microservices. This study aims to identify, during migration, which MS-ATDs occur, are the most severe, and are the most challenging to solve. In addition, we propose a way to prioritize these debts. We conducted a multiple exploratory case study of three large companies that were early in the migration process to microservices. We interviewed 47 practitioners with several roles to identify the debts in their contexts. We report the MS-ATDs detected during migration, the MS-ATDs that practitioners estimate to occur in the future, and the MS-ATDs that practitioners report as difficult to solve. We discuss the results in the context of the companies involved in this study. In addition, we used a risk assessment approach to propose a way for prioritizing MS-ATDs. Practitioners from other organizations and researchers may use this approach to provide rankings to help identify and prioritize which MS-ATDs should be avoided or solved in their contexts.

• Toledo, Saulo Soares De; Martini, Antonio & Sjøberg, Dag (2021). Identifying architectural technical debt, principal, and interest in microservices: A multiple-case study. Journal of Systems and Software. ISSN 0164-1212. 177. doi: 10.1016/j.jss.2021.110968. Full text in Research Archive Show summary

  Background: Using a microservices architecture is a popular strategy for software organizations to deliver value to their customers fast and continuously. However, scientific knowledge on how to manage architectural debt in microservices is scarce. Objectives: In the context of microservices applications, this paper aims to identify architectural technical debts (ATDs), their costs, and their most common solutions. Method: We conducted an exploratory multiple case study by conducting 25 interviews with practitioners working with microservices in seven large companies. Results: We found 16 ATD issues, their negative impact (interest), and common solutions to repay each debt together with the related costs (principal). Two examples of critical ATD issues found were the use of shared databases that, if not properly planned, leads to potential breaks on services every time the database schema changes and bad API designs, which leads to coupling among teams. We identified ATDs occurring in different domains and stages of development and created a map of the relationships among those debts. Conclusion: The findings may guide organizations in developing microservices systems that better manage and avoid architectural debts.

• Gjelsten, Børge Kile; Bergersen, Gunnar R.; Sjøberg, Dag & Cutts, Quintin (2021). No Gender Difference in CS1 Grade for Students with Programming from High School: An Exploratory Study, Koli Calling '21: Proceedings of the 21st Koli Calling International Conference on Computing Education Research. ACM Digital Library. ISSN 9781450384889. doi: 10.1145/3488042.3488071. Full text in Research Archive
• Toledo, Saulo Soares de; Martini, Antonio; Sjøberg, Dag; Przybyszewska, Agata & Frandsen, Johannes Skov (2021). Reducing Incidents in Microservices by Repaying Architectural Technical Debt, Proceedings of 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665427050. doi: 10.1109/SEAA53835.2021.00033. Full text in Research Archive Show summary

  Architectural technical debt (ATD) may create a substantial extra effort in software development, which is called interest. There is little evidence about whether repaying ATD in microservices reduces such interest. Objectives: We wanted to conduct a first study on investigating the effect of removing ATD on the occurrence of incidents in a microservices architecture. Method: We conducted a quantitative and qualitative case study of a project with approximately 1000 microservices in a large, international financing services company. We measured and compared the number of software incidents of different categories before and after repaying ATD. Results: The total number of incidents was reduced by 84%, and the numbers of critical- and high-priority incidents were both reduced by approximately 90% after the architectural refactoring. The number of incidents in the architecture with the ATD was mainly constant over time, but we observed a slight increase of low priority incidents related to inaccessibility and the environment in the architecture without the ATD. Conclusion: This study shows evidence that refactoring ATDs, such as lack of communication standards, poor management of dead-letter queues, and the use of inadequate technologies in microservices, reduces the number of critical- and high-priority incidents and, thus, part of its interest, although some low priority incidents may increase.

• Holgeid, Knut Kjetil; Jørgensen, Magne; Sjøberg, Dag & Krogstie, John (2021). Benefits management in software development: A systematic review of empirical studies. IET Software. ISSN 1751-8806. 15(1), p. 1–24. doi: 10.1049/sfw2.12007. Full text in Research Archive Show summary

  Considerable resources are wasted on software projects delivering less than the planned benefits. Herein, the objective is to synthesize empirical evidence of the adoption and impact of benefits management (BM) in software development, and to suggest directions for future research. A systematic review of the literature is performed and identified 4836 scientific papers of which the authors found 47 to include relevant research. While most organizations identify and structure benefits at the outset of a project, fewer organizations report implementing BM as a continuous process throughout the project lifecycle. Empirical evidence gives support for positive impact on project outcome from the following BM practices: identifying and structuring benefits, planning benefits realization, BM during project execution, benefits evaluation and the practice of having people responsible for benefits realization. The authors suggest four research directions to understand (1) why BM practices sometimes not are adopted, (2) BM in relation to other management practices, (3) BM in agile software development and (4) BM in the context of organizations' value creation logics.

• Toledo, Saulo Soares De; Martini, Antonio & Sjøberg, Dag I. K. (2020). Improving Agility by Managing Shared Libraries in Microservices. In Paasivaara, Maria & Kruchten, Philippe (Ed.), Lecture Notes in Business Information Processing, vol 396. Springer. ISSN 9783030588571. p. 195–202. doi: 10.1007/978-3-030-58858-8_20. Full text in Research Archive Show summary

  Using microservices is a way of supporting an agile architecture. However, if the microservices development is not properly managed, the teams’ development velocity may be affected, reducing agility and increasing architectural technical debt. This paper investigates how to manage the use of shared libraries in microservices to improve agility during development. We interviewed practitioners from four large international companies involved in microservices projects to identify problems when using shared libraries. Our results show that the participating companies had issues with shared libraries as follows: coupling among teams, delays on fixes due to overhead on libraries development teams, and need to maintain many versions of the libraries. Our results highlight that the use of shared libraries may hinder agility on microservices. Thus, their use should be restricted to situations where shared libraries cannot be replaced by a microservice and the costs of replicating the code on each service is very high.

• Stray, Viktoria; Moe, Nils Brede & Sjøberg, Dag (2020). Daily Stand-Up Meetings: Start Breaking the Rules! IEEE Software. ISSN 0740-7459. 37(3), p. 70–77. doi: 10.1109/MS.2018.2875988. Full text in Research Archive Show summary

  Daily stand-up meetings are commonly used for software teams to collaborate and exchange information, but conducting them in a way that benefits the whole team can be challenging. We describe factors that can affect meetings and propose recommendations for improving them.

• Toledo, Saulo Soares De; Martini, Antonio; Przybyszewska, Agata & Sjøberg, Dag (2019). Architectural Technical Debt in Microservices: A Case Study in a Large Company. In Avgeriou, Paris & Schmid, Klaus (Ed.), Proceedings of the Second International Conference on Technical Debt. IEEE Press. ISSN 9781728133713. doi: 10.1109/TechDebt.2019.00026. Full text in Research Archive Show summary

  Introduction: Software companies aim to achieve continuous delivery to constantly provide value to their customers. A popular strategy is to use microservices architecture. However, such an architecture is also subject to debt, which hinders the continuous delivery process and thus negatively affects the software released to the customers. Objectives: The aim of this study is to identify issues, solutions and risks related to Architecture Technical Debt in microservices. Method: We conducted an exploratory case study of a real life project with about 1000 services in a large, international company. Through qualitative analysis of documents and interviews, we investigated Architecture Technical Debt in the communication layer of a system with microservices architecture. Results: Our main contributions are a list of Architecture Technical Debt issues specific for the communication layer in a system with microservices architecture, as well as their associated negative impact (interest), a solution to repay the debt and the its cost (principal). Among the found Architecture Technical Debt issues were the existence of business logic in the communication layer and a high amount of point-to-point connections between services. The studied solution consists of the implementation of different canonical models specific to different domains, the removal of business logic from the communication layer, and migration from services to use the communication layer correctly. We also contributed with a list of possible risks that can affect the payment of the debt, as lack of funding and inadequate prioritization. Conclusion: We found issues, solutions and possible risks that are specific for microservices architectures not yet encountered in the current literature. Our results may be useful for practitioners that want to avoid or repay Technical Debt in their microservices architecture.

• Lindsjørn, Yngve; Bergersen, Gunnar R.; Dingsøyr, Torgeir & Sjøberg, Dag (2018). Teamwork Quality and Team Performance: Exploring Differences Between Small and Large Agile Projects. In Garbajosa, Juan; Wang, Xiaofeng & Aguiar, Ademar (Ed.), Agile Processes in Software Engineering and Extreme Programming, 19th International Conference, XP 2018, Proceedings. Springer. ISSN 9783319916019. p. 267–274. doi: 10.1007/978-3-319-91602-6_19. Full text in Research Archive Show summary

  Agile principles were originally developed for small projects but are now widely used in larger projects with hundreds of developers. Teamwork quality is essential in any development work, but how does teamwork quality differ in small and large agile projects? We report from an explorative survey with 64 agile teams and 320 team members and team leaders, from 31 teams in small projects and 33 teams in large projects. For small projects, teamwork quality was considered by both team members and team leaders to primarily affect product quality. For large projects, the effect of teamwork quality on product quality was positive when it was rated by team members but was negative when rated by team leaders. At a finer granularity, the six dimensions of teamwork quality that we investigated affected team performance differently in small and large projects. These findings question to what extent findings from previous studies on teamwork in agile development in small projects apply to large projects.

• Sjøberg, Dag (2018). An empirical study of WIP in kanban teams. In Oivo, Markku; Fernández, Daniel Méndez & Mockus, Audris (Ed.), Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM, Oulu, Finland — October 11 - 12, 2018. Association for Computing Machinery (ACM). ISSN 9781450358231. p. 13.1–13.8. doi: 10.1145/3239235.3239238. Full text in Research Archive Show summary

  Background: Limiting the amount of Work-In-Progress (WIP) is considered a fundamental principle in Kanban software development. However, no published studies from real cases exist that indicate what an optimal WIP limit should be. Aims: The primary aim is to study the effect of WIP on the performance of a Kanban team. The secondary aim is to illustrate methodological challenges when attempting to identify an optimal or appropriate WIP limit. Method: A quantitative case study was conducted in a software company that provided information about more than 8,000 work items developed over four years by five teams. Relationships between WIP, lead time and productivity were analyzed. Results: WIP correlates with lead time; that is, lower WIP indicates shorter lead times, which is consistent with claims in the literature. However, WIP also correlates with productivity, which is inconsistent with the claim in the literature that a low WIP (still above a certain threshold) will improve productivity. The collected data set did not include sufficient information to measure aspects of quality. There are several threats to the way productivity was measured. Conclusions: Indicating an optimal WIP limit is difficult in the studied company because a changing WIP gives contrasting results on different team performance variables. Because the effect of WIP has not been quantitatively examined before, this study clearly needs to be replicated in other contexts. In addition, studies that include other team performance variables, such as various aspects of quality, are requested. The methodological challenges illustrated in this paper need to be addressed.

• Feldt, robert; Zimmermann, Thomas; Bergersen, Gunnar R.; Falessi, Davide; Jedlitschka, Andreas & Juristo, Natalia [Show all 12 contributors for this article] (2018). Four commentaries on the use of students and professionals in empirical software engineering experiments. Empirical Software Engineering. ISSN 1382-3256. 23(6), p. 3801–3820. doi: 10.1007/s10664-018-9655-0. Full text in Research Archive Show summary

  The relative pros and cons of using students or practitioners in experiments in empirical software engineering have been discussed for a long time and continue to be an important topic. Following the recent publication of “Empirical software engineering experts on the use of students and professionals in experiments” by Falessi, Juristo, Wohlin, Turhan, Münch, Jedlitschka, and Oivo (EMSE, February 2018) we received a commentary by Sjøberg and Bergersen. Given that the topic is of great methodological interest to the community and requires nuanced treatment, we invited two editorial board members, Martin Shepperd and Per Runeson, respectively, to provide additional views

• Sjøberg, Dag (2016). The relationship between software process, context and outcome. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 10027, p. 3–11. doi: 10.1007/978-3-319-49094-6_1. Full text in Research Archive Show summary

  Most practitioners and researchers agree that when developing software, process affects product, and the usefulness of a process depends on the context. However, which processes are most useful for a specific company or project is generally unknown. When studying the relation between context, process and product, one challenge is that experiments often lack realism, which makes the transfer of results to industry difficult. In contrast, most of the important factors vary beyond the researcher’s control in case studies, which makes it difficult to identify cause and effect relationships. This paper reports a study where realism was combined with control over certain context and process factors. Four companies developed the same system, and the price varied by a factor of six. Certain patterns of relationships were expected (expensive company, low cost, schedule overrun); others were unexpected (cheap company, maintainable system because of small code). The community needs to identify the most important relationships among process, context and outcome.

• Lindsjørn, Yngve; Sjøberg, Dag; Dingsøyr, Torgeir; Bergersen, Gunnar R. & Dybå, Tore (2016). Teamwork Quality and Project Success in Software Development: A Survey of Agile Development Teams. Journal of Systems and Software. ISSN 0164-1212. 122, p. 274–286. doi: 10.1016/j.jss.2016.09.028. Full text in Research Archive Show summary

  Small, self-directed teams are central in agile development. This article investigates the effect of team- work quality on team performance, learning and work satisfaction in agile software teams, and whether this effect differs from that of traditional software teams. A survey was administered to 477 respondents from 71 agile software teams in 26 companies and analyzed using structural equation modeling. A posi- tive effect of teamwork quality on team performance was found when team members and team leaders rated team performance. In contrast, a negligible effect was found when product owners rated team per- formance. The effect of teamwork quality on team members ́learning and work satisfaction was strongly positive, but was only rated by the team members. Despite claims of the importance of teamwork in ag- ile teams, this study did not find teamwork quality to be higher than in a similar survey on traditional teams. The effect of teamwork quality on team performance was only marginally greater for the agile teams than for the traditional teams.

• Dybå, Tore; Bergersen, Gunnar R. & Sjøberg, Dag (2016). Evidence-Based Software Engineering. In Menzies, Tim; Williams, Laurie & Zimmermann, Thomas D. (Ed.), Perspectives on Data Science for Software Engineering. Elsevier. ISSN 9780128042069. p. 149–153. doi: 10.1016/B978-0-12-804206-9.00029-5. Full text in Research Archive Show summary

  A decade ago, Kitchenham, Dybå and Jørgensen coined the term and provided the foundations for evidence-based software engineering (EBSE). A trilogy of papers was written for researchers, practitioners, and educators. They suggested that practitioners consider EBSE as a mechanism to support and improve their technology adoption decisions, and that researchers should use systematic literature reviews as a methodology for performing unbiased aggregation of empirical results. This spurred significant international activity, and a renewed focus on research methods and theory, and on the future of empirical methods in SE research.

• Sjøberg, Dag; Bergersen, Gunnar R. & Dybå, Tore (2016). Why Theory Matters. In Menzies, Tim; Williams, Laurie & Zimmermann, Thomas D. (Ed.), Perspectives on Data Science for Software Engineering. Elsevier. ISSN 9780128042069. p. 29–33. doi: 10.1016/B978-0-12-804206-9.00006-4. Full text in Research Archive Show summary

  It is relatively easy to generate and acquire much data from software engineering activities. The challenge is to obtain meaning from the data that represents something true, rather than spurious. To increase knowledge and insight, more theories should be built and used.

• Stray, Viktoria; Sjøberg, Dag & Dybå, Tore (2016). The Daily Stand-up Meeting: A Grounded Theory Study. Journal of Systems and Software. ISSN 0164-1212. 114, p. 101–124. doi: 10.1016/j.jss.2016.01.004. Full text in Research Archive Show summary

  The daily stand-up meeting is one of the most used agile practices but has rarely been the subject of empirical research. The present study aims to identify how daily stand-up meetings are conducted and what the attitudes towards them are. A grounded theory study with 12 software teams in three companies in Malaysia, Norway, Poland and the United Kingdom was conducted. We interviewed 60 people, observed 79 daily stand-up meetings and collected supplementary data. The factors that contributed the most to a positive attitude towards the daily stand-up meeting were information sharing with the team and the opportunity to discuss and solve problems. The factors that contributed the most to a negative attitude were status reporting to the manager and that the frequency of the meeting was perceived to be too high and the duration too long. Based on our results, we developed a grounded theory of daily stand-up meetings and proposed empirically based recommendations and guidelines on how to organize them. Organizations should be aware of the factors that may affect the attitude towards daily stand-up meetings and should consider our recommendations and guidelines to make this agile practice as valuable as possible.

• Jørgensen, Magne; Dybå, Tore; Liestøl, Knut & Sjøberg, Dag (2016). Incorrect results in software engineering experiments: How to improve research practices. Journal of Systems and Software. ISSN 0164-1212. 116, p. 133–145. doi: 10.1016/j.jss.2015.03.065. Full text in Research Archive Show summary

  Context The trustworthiness of research results is a growing concern in many empirical disciplines. Aim The goals of this paper are to assess how much the trustworthiness of results reported in software engineering experiments is affected by researcher and publication bias, given typical statistical power and significance levels, and to suggest improved research practices. Method First, we conducted a small-scale survey to document the presence of researcher and publication biases in software engineering experiments. Then, we built a model that estimates the proportion of correct results for different levels of researcher and publication bias. A review of 150 randomly selected software engineering experiments published in the period 2002–2013 was conducted to provide input to the model. Results The survey indicates that researcher and publication bias is quite common. This finding is supported by the observation that the actual proportion of statistically significant results reported in the reviewed papers was about twice as high as the one expected assuming no researcher and publication bias. Our models suggest a high proportion of incorrect results even with quite conservative assumptions. Conclusion Research practices must improve to increase the trustworthiness of software engineering experiments. A key to this improvement is to avoid conducting studies with unsatisfactory low statistical power.

• Bergersen, Gunnar R.; Sjøberg, Dag & Dybå, Tore (2014). Construction and Validation of an Instrument for Measuring Programming Skill. IEEE Transactions on Software Engineering. ISSN 0098-5589. 40(12), p. 1163–1184. doi: 10.1109/TSE.2014.2348997. Full text in Research Archive Show summary

  Skilled workers are crucial to the success of software development. The current practice in research and industry for assessing programming skills is mostly to use proxy variables of skill, such as education, experience, and multiple-choice knowledge tests. There is as yet no valid and efficient way to measure programming skill. The aim of this research is to develop a valid instrument that measures programming skill by inferring skill directly from the performance on programming tasks. Over two days, 65 professional developers from eight countries solved 19 Java programming tasks. Based on the developers' performance, the Rasch measurement model was used to construct the instrument. The instrument was found to have satisfactory (internal) psychometric properties and correlated with external variables in compliance with theoretical expectations. Such an instrument has many implications for practice, for example, in job recruitment and project allocation.

• Stray, Viktoria Gulliksen; Lindsjørn, Yngve & Sjøberg, Dag (2013). Obstacles to Efficient Daily Meetings in Agile Development Projects: A Case Study. In Falessi, Davide (Eds.), ACM / IEEE International Symposium on Empirical Software Engineering and Measurement. Baltimore, 10-11 Oct. 2013. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9780769550565. p. 95–102. doi: 10.1109/ESEM.2013.30. Full text in Research Archive Show summary

  Context: Most of the software organizations that use agile methods organize daily team meetings. Aim: Our aim was to understand how daily meetings are conducted and identify obstacles that reduce their efficiency. Method: We observed 56 daily meetings and conducted 21 interviews in three different teams in two countries. We used the repertory grid technique in the interviews and to analyze the results. Results: We identified thirteen obstacles. The four most prominent ones were: (1) The daily meetings lasted too long (on average, 22 minutes instead of the scheduled 15 minutes). (2) In the meetings that were not self-organized, team members reported to the Scrum Master instead of sharing information among all team members. (3) The interruption caused by daily meetings required substantially more time than the actual meeting time due to overhead before and after the meetings. (4) Several team members had negative attitudes towards the daily meetings. Conclusion: Organizers of daily meetings should evaluate whether the obstacles we have identified are present in their organization and consider our suggestions to remove or reduce these obstacles.

• Kitchenham, Barbara; Sjøberg, Dag; Dybå, Tore; Brereton, O. Pearl; Budgen, David & Höst, Martin [Show all 7 contributors for this article] (2013). Trends in the Quality of Human-Centric Software Engineering Experiments-A Quasi-Experiment. IEEE Transactions on Software Engineering. ISSN 0098-5589. 39(7), p. 1002–1017. doi: 10.1109/TSE.2012.76. Full text in Research Archive Show summary

  Context: Several text books and papers published between 2000 and 2002 have attempted to introduce experimental design and statistical methods to software engineers undertaking empirical studies. Objective: This paper investigates whether there has been an increase in the quality of human-centric experimental and quasi-experimental journal papers over the time period 1993 to 2010. Method: Seventy experimental and quasi-experimental papers published in four general software engineering journals in the years 1992-2002 and 2006-2010 were each assessed for quality by three empirical software engineering researchers using two quality assessment methods (a questionnaire-based method and a subjective overall assessment). Regression analysis was used to assess the relationship between paper quality and the year of publication, publication date group (before 2003 and after 2005), source journal, average coauthor experience, citation of statistical text books and papers, and paper length. The results were validated both by removing papers for which the quality score appeared unreliable and using an alternative quality measure. Results: Paper quality was significantly associated with year, citing general statistical texts, and paper length (p <; 0.05). Paper length did not reach significance when quality was measured using an overall subjective assessment. Conclusions: The quality of experimental and quasi-experimental software engineering papers appears to have improved gradually since 1993.

• Sjøberg, Dag; Yamashita, Aiko Amparo Fallas; Anda, Bente Cecilie D; Mockus, Audris & Dybå, Tore (2013). Quantifying the Effect of Code Smells on Maintenance Effort. IEEE Transactions on Software Engineering. ISSN 0098-5589. 39(8), p. 1144–1156. doi: 10.1109/TSE.2012.89. Full text in Research Archive Show summary

  Context: Code smells are assumed to indicate bad design that leads to less maintainable code. However, this assumption has not been investigated in controlled studies with professional software developers. Aim: This paper investigates the relationship between code smells and maintenance effort. Method: Six developers were hired to perform three maintenance tasks each on four functionally equivalent Java systems originally implemented by different companies. Each developer spent three to four weeks. In total, they modified 298 Java files in the four systems. An Eclipse IDE plug-in measured the exact amount of time a developer spent maintaining each file. Regression analysis was used to explain the effort using file properties, including the number of smells. Results: None of the 12 investigated smells was significantly associated with increased effort after we adjusted for file size and the number of changes; Refused Bequest was significantly associated with decreased effort. File size and the number of changes explained almost all of the modeled variation in effort. Conclusion: The effects of the 12 smells on maintenance effort were limited. To reduce maintenance effort, a focus on reducing code size and the work practices that limit the number of changes may be more beneficial than refactoring code smells.

• Kitchenham, Barbara Ann; Sjøberg, Dag; Dybå, Tore; Pfahl, Dietmar Alfred; Brereton, Pearl & Budgen, David [Show all 8 contributors for this article] (2012). Three empirical studies on the agreement of reviewers about the quality of software engineering experiments. Information and Software Technology. ISSN 0950-5849. 54(8), p. 804–819. doi: 10.1016/j.infsof.2011.11.008. Full text in Research Archive
• Bergersen, Gunnar R. & Sjøberg, Dag (2012). Evaluating Methods and Technologies in Software Engineering with Respect to Developers’ Skill Level. In Mendes, Emilia & Genero, Marcela (Ed.), Proceedings, 16th International Conference on Evaluation and Assessment in Software Engineering. IET Research Journals. ISSN 9781849195416. p. 101–110. Full text in Research Archive
• Sjøberg, Dag; Johnsen, Anders & Solberg, Jørgen (2012). Quantifying the Effect of Using Kanban versus Scrum: A Case Study. IEEE Software. ISSN 0740-7459. 29(5), p. 47–53. doi: 10.1109/MS.2012.110. Full text in Research Archive
• Sjøberg, Dag; Anda, Bente Cecilie D & Mockus, Audris (2012). Questioning software maintenance metrics: a comparative case study. In Runeson, Per; Høst, Martin; Mendes, Emilia; Andrews, Anneliese Amschler & Harrison, Rachel (Ed.), ESEM '12. Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781450310567. p. 107–110. doi: 10.1145/2372251.2372269. Full text in Research Archive
• Dybå, Tore; Sjøberg, Dag & Cruzes, Daniela Soares (2012). What works for whom, where, when, and why?: on the role of context in empirical software engineering. In Runeson, Per; Høst, Martin; Mendes, Emilia; Andrews, Anneliese Amschler & Harrison, Rachel (Ed.), ESEM '12. Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781450310567. p. 19–28. doi: 10.1145/2372251.2372256. Full text in Research Archive Show summary

  Context is a central concept in empirical software engineering. It is one of the distinctive features of the discipline and it is an in-dispensable part of software practice. It is likely responsible for one of the most challenging methodological and theoretical problems: study-to-study variation in research findings. Still, empirical software engineering research is mostly concerned with attempts to identify universal relationships that are independent of how work settings and other contexts interact with the processes important to software practice. The aim of this paper is to provide an overview of how context affects empirical research and how empirical software engineering research can be better 'contextualized' in order to provide a better understanding of what works for whom, where, when, and why. We exemplify the importance of context with examples from recent systematic reviews and offer recommendations on the way forward.

• Bergersen, Gunnar; Hannay, Jo Erskine; Sjøberg, Dag; Dybå, Tore & Karahasanovic, Amela (2011). Inferring Skill from Tests of Programming Performance: Combining Time and Quality, Proceedings of International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781457722035. p. 305–314. doi: 10.1109/esem.2011.39. Full text in Research Archive Show summary

  The skills of software developers are important to the success of software projects. Also, when studying the general effect of a tool or method, it is important to control for individual differences in skill. However, the way skill is assessed is often ad hoc, or based on unvalidated methods. According to established test theory, validated tests of skill should infer skill levels from well-defined performance measures on multiple, small, representative tasks. In this respect, we show how time and quality, which are often analyzed separately, can be combined as task performance and subsequently be aggregated as an approximation of skill. Our results show significant positive correlations between our proposed measures of skill and other variables, such as seniority, lines of code written, and self-evaluated expertise. The method for combining time and quality is a promising first step to measuring programming skill in both industry and research settings.

• Sjøberg, Dag; Odberg, Erik & Warlo, Borger (2010). The Challenge of Assessing and Controlling Complexity in a Large Portfolio of Software Systems. In Hall, Tracy; Høst, Martin; Niazi, Mahmood; Mishra, Alok & Falessi, Davide (Ed.), Product Focused Software Development and Process Improvement (PROFES 2010, Limerick, Ireland, June 21-23. 2010). Universitalia. ISSN 9788865070673. p. 75–78. Full text in Research Archive
• Hannay, Jo Erskine; Arisholm, Erik; Engvik, Harald & Sjøberg, Dag (2010). Effects of Personality on Pair Programming. IEEE Transactions on Software Engineering. ISSN 0098-5589. 36(1), p. 61–80. doi: 10.1109/TSE.2009.41. Full text in Research Archive
• Olbrich, Steffen; Cruzes, Daniela Soares & Sjøberg, Dag (2010). Are all code smells harmful? A study of God Classes and Brain Classes in the evolution of three open source systems. Proceedings - Conferense on Software Maintenance. ISSN 1063-6773. 1, p. 1–10. doi: 10.1109/ICSM.2010.5609564. Full text in Research Archive
• Sjøberg, Dag (2010). Confronting the Myth of Rapid Obsolescence in Computing Research. Communications of the ACM. ISSN 0001-0782. 53(9), p. 62–67. doi: 10.1145/1810891.1810911. Full text in Research Archive
• Kitchenham, B.; Sjøberg, Dag; Brereton, O. Pearl; Budgen, David; Dybå, Tore & Høst, Martin [Show all 8 contributors for this article] (2010). Can we evaluate the quality of software engineering experiments? In Morisio, Maurizio (Eds.), Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781450300391. Full text in Research Archive
• Følstad, Asbjørn; Anda, Bente Cecilie D & Sjøberg, Dag (2010). The usability inspection performance of work-domain experts: An empirical study. Interacting with computers. ISSN 0953-5438. 22(2), p. 75–87. doi: 10.1016/j.intcom.2009.09.001. Full text in Research Archive
• Sjøberg, Dag & Grimstad, Stein (2010). Software Engineering: Why - How - What - What's Next. In Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (Ed.), Simula Research Laboratory - by thinking constantly about it. Springer. ISSN 9783642011559. p. 363–368. Full text in Research Archive
• Sjøberg, Dag (2010). The Industry is our Lab – Organization and Conducting of Empirical Studies in Software Engineering at Simula. In Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (Ed.), Simula Research Laboratory - by thinking constantly about it. Springer. ISSN 9783642011559. p. 443–458. Full text in Research Archive
• Sjøberg, Dag; Grimstad, Stein; Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (2009). Software Engineering: Why - How - What - What's Next. In Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (Ed.), Simula Research Laboratory - by thinking constantly about it. Springer. ISSN 9783642011559. p. 363–368. Full text in Research Archive
• Sjøberg, Dag; Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (2009). The Industry Is Our Lab - Organization and Conducting of Empirical Studies in Software Engineering at Simula. In Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (Ed.), Simula Research Laboratory - by thinking constantly about it. Springer. ISSN 9783642011559. p. 443–458. Full text in Research Archive Show summary

  The society is completely dependent on reliable and efficient software. Development of such software requires the use of appropriate processes, method and tools. To evaluate their appropriateness in various industrial settings, empirical studies are needed. At present, most studies in software engineering are run with students in artificial settings. The strategy of Simula Research Laboratory to invest in experiment infrastructures and hire professionals as subjects has contributed to a significant improvement of state of the art regarding the realism and scale of experiments and control in real-life case studies. During its first eight years, Simula has conducted studies that have included about 6000 professional software developers and managers from 273 companies in 25 countries. However, increasing the realism of tasks and systems in experiments and control in case studies is still a challenge for Simula and the rest of the research community.

• Hannay, Jo; Dybå, Tore; Arisholm, Erik & Sjøberg, Dag (2009). The Effectiveness of Pair-Programming: a Meta-Analysis. Information and Software Technology. ISSN 0950-5849. 55, p. 1110–1122. Full text in Research Archive Show summary

  Several experiments on the effects of pair versus solo programming have been reported in the literature. We present a metaanalysis of these studies. The analysis shows a small significant positive overall effect of pair programming on quality, a medium significant positive overall effect on duration, and a medium significant negative overall effect on effort. However, between-study variance is significant, and there are signs of publication bias among published studies on pair programming. A more detailed examination of the evidence suggests that pair programming is faster than solo programming when programming task complexity is low and yields code solutions of higher quality when task complexity is high. The higher quality for complex tasks comes at a price of considerably greater effort, while the reduced completion time for the simpler tasks comes at a price of noticeably lower quality. We conclude that greater attention should be given to moderating factors on the effects of pair programming.

• Hannay, Jo; Arisholm, Erik; Engvik, Harald & Sjøberg, Dag (2009). Effects of Personality on Pair Programming. IEEE Transactions on Software Engineering. ISSN 0098-5589. 36, p. 61–80. Full text in Research Archive Show summary

  Personality tests in various guises are commonly used in recruitment and career counseling industries. Such tests have also been considered as instruments for predicting programming and team performance in software engineering. However, research suggests that other human-related factors, such as motivation, general mental ability, expertise and task complexity also affect performance in general. This paper reports on a study of the impact of Big-Five personality traits on the performance of pair programmers together with the impact of expertise and task complexity. The study involved 196 software professionals from three countries forming 98 pairs. The analysis consisted of a confirmatory part and an exploratory part, and the results show that (1) our data does not confirm a meta-analysis-based model of the impact of certain personality traits on performance, and (2) personality traits in general have modest predictive value on pair programming performance compared with expertise, task complexity, and even country. Rather than focusing on direct effects of personality on pair programming performance, we conclude that (a) effort should be spent on elaborating on personality's (and other factors') indirect effects on performancemediated by formalized scales of collaboration, and (b) more effort should be spent on investigating other performance-related predictors such as programming skill, learning, motivation, expertise, and task complexity.

• Hannay, Jo Erskine; Dybå, Tore; Arisholm, Erik & Sjøberg, Dag (2009). The effectiveness of pair programming: A meta-analysis. Information and Software Technology. ISSN 0950-5849. 51(7), p. 1110–1122. doi: 10.1016/j.infsof.2009.02.001. Full text in Research Archive
• Anda, Bente Cecilie D; Sjøberg, Dag & Mockus, Audris (2009). Variability and Reproducibility in Software Engineering: A Study of Four Companies that Developed the Same System. IEEE Transactions on Software Engineering. ISSN 0098-5589. 35(3), p. 407–429. doi: 10.1109/TSE.2008.89. Full text in Research Archive
• Karahasanovic, Amela; Hinkel, Unni Nyhamar; Sjøberg, Dag & Thomas, R. (2009). Comparing of feedback-collection and think-aloud methods in program comprehension studies. Behaviour and Information Technology. ISSN 0144-929X. 28(2), p. 139–164. doi: 10.1080/01449290701682761. Full text in Research Archive
• Kampenes, Vigdis By; Dybå, T.; Hannay, Jo Erskine & Sjøberg, Dag (2009). A systematic review of quasi-experiments in software engineering. Information and Software Technology. ISSN 0950-5849. 51(1), p. 71–82. doi: 10.1016/j.infsof.2008.04.006. Full text in Research Archive
• Yamashita, Aiko Amparo Fallas; Benestad, Hans Christian; Anda, Bente Cecilie D; Arnstad, P.E.; Sjøberg, Dag & Moonen, L. (2009). Using Concept Mapping for Maintainability Assessments. In Carver, Jeffrey (Eds.), 3rd IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2009). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781424448425. p. 378–389. Full text in Research Archive
• Sjøberg, Dag; Dybå, T.; Anda, Bente Cecilie D & Hannay, Jo Erskine (2008). Building Theories in Software Engineering. In Shull, Forrest; Singer, Janice & Sjøberg, Dag (Ed.), Advanced Topics in Empirical Software Engineering. Springer. ISSN 9781848000438. Full text in Research Archive
• Hannay, Jo Erskine; Sjøberg, Dag & Dybå, Tore (2007). A Systematic Review of Theory Use in Software Engineering Experiments. IEEE Transactions on Software Engineering. ISSN 0098-5589. 33(2), p. 87–107. Full text in Research Archive
• Kampenes, Vigdis By; Dyba, T; Hannay, Jo Erskine & Sjøberg, Dag (2007). A systematic review of effect size in software engineering experiments. Information and Software Technology. ISSN 0950-5849. 49, p. 1073–1086. doi: 10.1016/j.infsof.2007.02.015. Full text in Research Archive
• Sjøberg, Dag (2007). Documenting Theories, Empirical Software Engineering Issues: Critical Assessment and Future Directions. Springer. ISSN 9783540713005. p. 111–114. Full text in Research Archive
• Sjøberg, Dag; Dybå, Tore & Jørgensen, Magne (2007). The Future of Empirical Methods in Software Engineering Research. In Briand, Lionel & Wolf, Alexander (Ed.), Future of Software Engineering. IEEEE-CS. ISSN 9780769528298. p. 358–378. Full text in Research Archive
• Dybå, Tore; Arisholm, Erik; Sjøberg, Dag; Hannay, J.E. & Shull, Forrest (2007). Are Two Heads Better than One? On the Effectiveness of Pair Programming. IEEE Software. ISSN 0740-7459. 24(6), p. 12–15. Full text in Research Archive
• Basili, Victor R.; Zelkowitz, Marvin V.; Sjøberg, Dag & Johnson, Philip (2007). Protocols in the use of Empirical Software Engineering Artifacts. Empirical Software Engineering. ISSN 1382-3256. 12(1), p. 107–119. Full text in Research Archive
• Arisholm, Erik; Gallis, Hans; Dybå, Tore & Sjøberg, Dag (2007). Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise. IEEE Transactions on Software Engineering. ISSN 0098-5589. 333(2), p. 65–86. Full text in Research Archive
• Sjøberg, Dag (2007). Knowledge Acquisition in Software Engineering Requires Sharing of Data and Artifacts, Empirical Software Engineering Issues: Critical Assessment and Future Directions. Springer. ISSN 9783540713005. p. 77–82. Full text in Research Archive
• Jørgensen, Magne & Sjøberg, Dag (2006). Expert Estimation of Software Development Work. In Madhavji, Nazim; Fernandez-Ramli, Juan & Perry, Dewayne (Ed.), Software Evolution and Feedback: Theory and Practice. IEEE Press. ISSN 9780470871805. p. 523–527. Full text in Research Archive
• Dybå, Tore; Kampenes, Vigdis By & Sjøberg, Dag (2006). A systematic review of statistical power in software engineering experiments. Information and Software Technology. ISSN 0950-5849. 48(8), p. 745–755. Full text in Research Archive
• Karahasanovic, Amela; Fjuk, Annita; Sjøberg, Dag & Thomas, R. (2006). Revealing object-oriented comprehension by feedback collection. In Fjuk, Annita; Karahasanovic, Amela & Kaasbøll, Jens Johan (Ed.), Comprehensive Object-Oriented Learning: The Learner's Perspective. Informing Science Press. ISSN 9788392233749. p. 111–130. Full text in Research Archive
• Conradi, Reidar; Dybå, Tore; Sjøberg, Dag I. K. & Ulsund, Tor (2006). Lessons Learned and Recommendations from two Large Norwegian SPI Programmes. In Conradi, Reidar; Dybå, Tore; Sjøberg, Dag I. K. & Ulsund, Tor (Ed.), Software Process Improvement: Results and Experience from the Field. Springer. ISSN 9783540321781. p. 3–18. Full text in Research Archive Show summary

  Software development is an experimental discipline, i.e. somewhat unpredictable. This suggests that software processes improvement should be based on the continuous iteration of characterization, goal setting, selection of improved technology, monitoring and analysis of its effects. This paper describes experiences from the empirical studies in two large SPI programmes in Norway. Five main lessons were learned: 1) It is a challenge for the industrial partners to invest enough resources in SPI activities. 2) The research partners must learn to know the companies, and 3) they must work as a multicompetent and coherent unit towards them. 4) Any SPI initiative must show visible, short-term payoff. 5) Establishing a solid baseline from which to improve is unrealistic. Based on these lessons, a set of operational recommendations for other researchers in the area are proposed. Also published in Flavio Oquendo (Ed.), 9th Int'l. Workshop on Software Process Technology, Helsinki, Finland, ISBN 3-540-40764-2

• Krogstie, John; Jahr, Arthur & Sjøberg, Dag I. K. (2006). A longitudinal study of development and maintenance in Norway: Report from the 2003 investigation. Information and Software Technology. ISSN 0950-5849. 48, p. 993–1005. Full text in Research Archive
• Anda, Bente Cecilie D & Sjøberg, Dag (2005). Investigating the Role of Use Cases in the Construction of Class Diagrams. Empirical Software Engineering. ISSN 1382-3256. 10(3), p. 285–309. Full text in Research Archive
• Karahasanovic, Amela; Anda, Bente Cecilie D; Arisholm, Erik; Hove, SE; Jørgensen, Magne & Sjøberg, Dag [Show all 7 contributors for this article] (2005). Collecting Feedback during Software Engineering Experiments. Empirical Software Engineering. ISSN 1382-3256. 10(2), p. 113–147. Full text in Research Archive
• Sjøberg, Dag; Hannay, J.E.; Kampenes, V.B.; Hansen, O; Karahasanovic, Amela & Rekdal, A.C. [Show all 7 contributors for this article] (2005). A Survey of Controlled Experiments in Software Engineering. IEEE Transactions on Software Engineering. ISSN 0098-5589. 31(9), p. 733–753. Full text in Research Archive
• Benestad, Hans Christian; Arisholm, Erik & Sjøberg, Dag (2005). How to Recruit Professionals as Subjects in Software Engineering Experiments. In Hustad, E.; Munkvold, B.E.; Rolland, K & Flak, L.S. (Ed.), IRIS (Information Systems Research in Scandinavia). Department of Information Systems. ISSN 9788271175542. Full text in Research Archive

View all works in NVA

• Shull, Forrest; Singer, Janice & Sjøberg, Dag (2008). Advanced Topics in Empirical Software Engineering. Springer. ISBN 9781848000438. 388 p. Full text in Research Archive
• Conradi, Reidar; Dybå, Tore; Sjøberg, Dag I. K. & Ulsund, Tor (2006). Software Process Improvement: Results and Experience from the Field. Springer. ISBN 9783540321781. 413 p. Full text in Research Archive Show summary

  This book reports on results and experience from three Norwegian, industrial R&D project on Software Process Improvement (SPI) - SPIQ, PROFIT and SPIKE - with emphasis on the latter. The book is organized in five sections, with totally 20 previously published papers. Section 1 is about SPI principles/general, and has 5 papers. Section 2: Knowledge Management for SPI and has 5 papers. Section 3 is about Process Modelling and Electronic Process Guides and has 4 papers. Section 4 is about Estimation Methods and has 3 papers. Section 5 is about Empirical Studies in OO and Component-based Systems and has 3 papers. The audience is software professionals as well as researchers and graduate students.

View all works in NVA

• Sjøberg, Dag (2023). Studie antyder at IT-fag ikke bør undervises på engelsk. Det blir ikke lett, mener professor. [Journal]. Digi.no. Full text in Research Archive
• Sjøberg, Dag (2023). AI Engineer. [Journal]. kode24. Full text in Research Archive
• Bergersen, Gunnar R. & Sjøberg, Dag Ingar Kondrup (2023). Education of software developers in a fast-growing job market: Giving more students with "non-standard" backgrounds an opportunity. Full text in Research Archive Show summary

  Software development is a fast-growing occupation worldwide and has grown by 50% in Norway during the last five years. Unlike many other professional occupations, system developers are seldom required to have a formal education to work professionally. For example, anyone with the right skills can build open-source software or offer freelance services worldwide on a project-by-project basis. Currently, educational institutions cannot educate enough developers to fill new jobs. However, there is also a discrepancy between what type of students that academic educations will admit and, potentially, could admit. In this context, a student with a "standard" background would enter a programme after upper secondary education with all the study requirements in place and a goal to complete a BSc or MSc. In contrast, a student with a "non-standard" background would typically be older, have prior work or education experience, and have different interests and motivations to study a few selected topics. However, this may not be possible as many would-be students lack advanced math, a formal requirement for attending many courses. We report results on an ongoing longitudinal study of software development students (N=854) in our department over four years. In addition to analyzing prior grades, math specialization, and programming-related electives from upper secondary school, the students provided self-reports on their prior experience, interests, motivations, and aspirations for studying software development. We also contrast these variables with grades and study credit production and show that a substantial group of students with non-standard backgrounds appear to do equally well or better than the students with standard backgrounds. We conclude that even though commonly established requirements for studying software development should be retained for students with standard backgrounds, one should revise how students with non-standard backgrounds are admitted. As calls for greater diversity in society linger, we highlight the importance of greater flexibility in university requirements for studying software development so that the software industry's increasing needs are met to a greater extent than it is today.

• Sjøberg, Dag (2023). Tall opplyser. [Journal]. Klassekampen. Full text in Research Archive
• Sjøberg, Dag (2022). Keynote: Construct Validity in Software Engineering. Full text in Research Archive
• Sjøberg, Dag (2021). Utenlandske forskere: Jobbene nordmenn ikke vil ha. [Journal]. Aftenposten. Full text in Research Archive
• Sjøberg, Dag (2018). Forskning er skriving. Forskerforum. ISSN 0800-1715. p. 43–43. Full text in Research Archive
• Tobiassen, Markus; Bergersen, Gunnar R. & Sjøberg, Dag (2016). Ser butikk i it-flauser. [Journal]. Dagens Næringsliv. Full text in Research Archive
• Sjøberg, Dag (2016). Programmering i skolen er hverken frelse eller rør. [Journal]. Aftenposten. Full text in Research Archive
• Sjøberg, Dag (2015). Programmering bør være allmennkunnskap. [Journal]. Dagens Næringsliv. Full text in Research Archive
• Sjøberg, Dag (2015). Vil ikke forske på it. [Journal]. Dagens Næringsliv. Full text in Research Archive
• Sjøberg, Dag (2015). Bruk av IT-konsulenter i offentlig sektor. [Journal]. Bergens Tidende. Full text in Research Archive
• Sjøberg, Dag (2014). Spekulativ sykkelforskning. Full text in Research Archive
• Sjøberg, Dag (2013). Viktigste faktorer for å redusere teknisk gjeld. Full text in Research Archive
• Sjøberg, Dag (2013). Kjemper innbitt mot stammespråket. [Journal]. Computerworld. Full text in Research Archive
• Sjøberg, Dag (2013). Smidig må finne sin form. [Journal]. Computerworld. Full text in Research Archive
• Sjøberg, Dag (2012). Organisering av offentlige IT-prosjekter. [Journal]. Aftenposten. Full text in Research Archive
• Sjøberg, Dag (2012). Kompleksiteten vil bli enorm dersom man skal samle alt på ett sted. [Journal]. Aftenposten. Full text in Research Archive
• Sjøberg, Dag (2012). Har brukt 425 millioner på IT-konsulenter. [Journal]. Aftenposten. Full text in Research Archive
• Sjøberg, Dag (2012). Smidig forskning. [Journal]. Computerworld. Full text in Research Archive
• Sjøberg, Dag (2012). Skandalene er kommet for å bli. [Journal]. Aftenposten. Full text in Research Archive
• Sjøberg, Dag (2012). Smidig utvikling krever opplæring. [Journal]. Computerworld. Full text in Research Archive
• Sjøberg, Dag (2011). Suksess med Kanban. [Journal]. Computerworld. Full text in Research Archive
• Sjøberg, Dag (2011). En kvantitativ studie av Kanban versus Scrum. Full text in Research Archive
• Sjøberg, Dag (2011). Empirical software research: an interview with Dag Sjøberg, University of Oslo, Norway. ACM Ubiquity. doi: 10.1145/1998372.1998374. Full text in Research Archive
• Mockus, Audris; Anda, Bente Cecilie D & Sjøberg, Dag (2010). Experiences from Replicating a Case Study to Investigate Reproducibility of Software Development. Full text in Research Archive
• Sjøberg, Dag (2010). IT-forskningen er treg. p. 32–33. Full text in Research Archive
• Sjøberg, Dag; Dybå, Tore; Conradi, Reidar; Hoff, Per Morten & Gundersen, Christer (2010). Derfor trenger Norge mer IKT-forskning. p. 24–24. Full text in Research Archive Show summary

  Dette er første motkronikk av 3. des. 2010 til startkronikk i Computerworld 19. nov. 2010: "http://www.idi.ntnu.no/grupper/su/publ/reidar/cw.19.11.startkronikk.pdf". Se også Conradi's andre motkronikk i Computerworld, 17. des. 2010: http://www.idi.ntnu.no/grupper/su/publ/reidar/cw.17.12.motkronikk2-rc.pdf.

• Sjøberg, Dag (2010). Universitetet i Oslo satser på Smidig. [Journal]. ComputerWorld. Full text in Research Archive
• Benestad, Hans Christian; Arisholm, Erik & Sjøberg, Dag (2005). How to Recruit Professionals as Subjects in Software Engineering Experiments. Full text in Research Archive
• LaToza, Thomas D; Ko, Amy J.; Shepherd, David C; Sjøberg, Dag & Xie, Benjamin (2023). Theories of Programming (Dagstuhl Seminar 22231). Schloss Dagstuhl - Leibniz-Zentrum fur Informatik. Full text in Research Archive Show summary

  Much of computer science research focuses on techniques to make programming easier, better, less error prone, more powerful, and even more just. But rarely do we try to explain any of these challenges. Why is programming hard? Why is it slow? Why is it error prone? Why is it powerful? How does it do harm? These why and how questions are what motivated the Dagstuhl Seminar 22231 on Theories of Programming. This seminar brought together 28 CS researchers from domains most concerned with programming human and social activities: software engineering, programming languages, human-computer interaction, and computing education. Together, we sketched new theories of programming and considered the role of theories more broadly in programming.

• Bergersen, Gunnar R.; Sjøberg, Dag & Dybå, Tore (2015). Measuring programming skill. Akademika forlag. Full text in Research Archive

View all works in NVA