Dag Sjøberg

 Tidsskriftartikler

53. Raluca Florea, Viktoria Stray, Dag I.K. Sjøberg. On the roles of software testers: An exploratory study. Journal of Systems and Software, 204, 111742, 2023. https://doi.org/10.1016/j.jss.2023.111742.

52. D.I.K. Sjøberg and G.R. Bergersen. Construct Validity in Software Engineering. IEEE Transactions on Software Engineering, 49(3):1374-1396, March 2023. doi: 10.1109/TSE.2022.3176725.

51. K.K. Holgeid, J. Krogstie, P. Mikalef, E.E. Saur, D.I.K. Sjøberg. Benefits management and Information Technology work distribution. IET Software, 2022

50. S.S. de Toledo, S. S., A. Martini, A., and D.I.K. Sjøberg. Identifying architectural technical debt, principal, and interest in microservices: A multiple-case study. Journal of Systems and Software, 177, 2021.

49. K.K Holgeid, M. Jørgensen, D.I.K. Sjøberg and J. Krogstie: Benefits management in software development: A systematic review of empirical studies, IET Software. 15(1):1-24, 2021

48. V. Stray, N.B. Moe and D.I.K. Sjøberg. Daily Stand-Up Meetings: Start Breaking the Rules. IEEE Software. 37(3): 70-77, 2020.

47. R. Feldt, T. Zimmermann, G.R. Bergersen, D. Falessi, A. Jedlitschka, N. Juristo, J.Münch, M. Oivo, P.Runeson, M.J. Shepperd, D.I.K. Sjøberg and B.Turhan. Four commentaries on the use of students and professionals in empirical software engineering experiments. Empirical Software Engineering, 23(6): 3801-3820, 2018.

46. Y. Lindsjørn, D.I.K. Sjøberg, T. Dingsøyr, G.R. Bergersen and T. Dybå. Teamwork Quality and Project Success in Software Development: A Survey of Agile Development Teams. Journal of Systems and Software, 116:274–286, December 2016.

45. M. Jørgensen, T. Dybå, K. Liestøl and D.I.K. Sjøberg. Incorrect Results in Software Engineering Experiments: How to Improve Research Practices. Journal of Systems and Software, 116:133–145, June 2016.

44. V. Stray, D.I.K. Sjøberg and T. Dybå. The Daily Stand-Up Meeting: A Grounded Theory Study. Journal of Systems and Software, 114:101-124, April 2016.

43. G.R. Bergersen, D.I.K. Sjøberg and T. Dybå. Construction and validation of an instrument for measuring programming skill. IEEE Transactions on Software Engineering, 40(12):1163-1184, December 2014. doi:10.1109/TSE.2014.2348997.

42. D.I.K. Sjøberg, A. Yamashita, B. Anda, A. Mockus and T. Dybå. Quantifying the Effect of Code Smells on Maintenance Effort. IEEE Transactions on Software Engineering, 39(8):1144-1156, August 2013. pdf

(Relatert artikkel: Questioning software maintenance metrics: a comparative case study. I: ESEM '12. Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE conference proceedings 2012 ISBN 978-1-4503-1056-7. s. 107-110, pdf)

41. B. Kitchenham, D.I.K. Sjøberg, T. Dybå, O.P. Brereton, D. Budgen, M. Höst and P. Runeson,  Trends in the quality of human-intensive software engineering experiments – A quasi-experiment. IEEE Transactions on Software Engineering, 39(7):1002-1017, July 2013.

40. D.I.K. Sjøberg, A. Johnsen and J. Solberg. Quantifying the Effect of Using Kanban versus Scrum: A Case Study. IEEE Software, 29(5):47-53, September/October 2012. pdf

39. B. Kitchenham, D.I.K. Sjøberg, T. Dybå, D. Pfahl , O.P. Brereton, D. Budgen, M. Höst and P. Runeson. Three empirical studies on the agreement of reviewers about the quality of software engineering experiments. Information and Software Technology 54:804–819, 2012.

38. D.I.K. Sjøberg. Confronting the Myth of Rapid Obsolescence in Computing Research, Contributed Article, Communications of the ACM 53(9):62-67, September 2010.

37. J. Hannay, E. Arisholm, H. Engvik and D.I.K. Sjøberg. Effects of Personality on Pair Programming, IEEE Transactions on Software Engineering 36(1):61-80, 2010.

36. A. Folstad, B.C.D. Anda and D.I.K. Sjøberg. The Usability Inspection Performance of Work-domain Experts: An Empirical Study. Interacting with Computers 22(2):75-87, 2010

35. B.C.D. Anda, D.I.K. Sjøberg and A. Mockus. Variability and Reproducibility in Software Engineering: A Study of four Companies that Developed the same System, IEEE Transactions on Software Engineering 35(3):407-429, 2009.

34. J.E. Hannay, T. Dybå, E. Arisholm and D.I.K. Sjøberg. The Effectiveness of Pair-Programming: A Meta-Analysis, Information and Software Technology 55(7):1110-1122, 2009.

33. V.B. Kampenes, T. Dybå, J.E. Hannay and D.I.K. Sjøberg. A systematic review of quasi-experiments in software engineering, Information and Software Technology 51(1):71-82, 2009.

32. A. Karahasanovic, U.N. Hinkel, D.I.K. Sjøberg and R. Thomas. Comparing of Feedback Collection and Think-Aloud Methods in Program Comprehension Studies, Journal of Behaviour and Information Technology 28(2):139-164, 2009.

31. E. Arisholm, H.E. Gallis, T. Dybå and D.I.K. Sjøberg. Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise, IEEE Transactions on Software Engineering 33(2):65-86, 2007.

30. V. Basili, M. Zelkowitz, D.I.K. Sjøberg, P. Johnson and T. Cowling. Protocols in the use of Empirical Software Engineering Artifacts, Empirical Software Engineering 12(1):107-119, 2007.

29. T. Dybå, E. Arisholm, D.I.K. Sjøberg, J.E. Hannay and F. Shull. Are Two Heads Better than One? On the Effectiveness of Pair Programming, IEEE Software 24(6):12-15, 2007.

28. J.E. Hannay, D.I.K. Sjøberg and T. Dybå. A Systematic Review of Theory Use in Software Engineering Experiments, IEEE Transactions on Software Engineering 33(2):87-107, 2007.

27. V.B. Kampenes, T. Dybå, J.E. Hannay and D.I.K. Sjøberg. A Systematic Review of Effect Size in Software Engineering Experiments, Information and Software Technology 49(11-12):1073-1086, 2007.

26. T. Dybå, V.B. Kampenes and D.I.K. Sjøberg. A systematic review of statistical power in software engineering experiments, Journal of Information and Software Technology 48(8):745-755, 2006.

25. J. Krogstie, A. Jahr and D.I.K. Sjøberg. A Longitudinal Study of General and Functional Maintenance in Norway, Information and Software Technology 48(11):993-1005, 2006.

24. B.C.D. Anda and D.I.K. Sjøberg. Investigating the Role of Use Cases in the Construction of Class Diagrams, Empirical Software Engineering 10(3):285-309, 2005.

23. A. Karahasanovic, B.C.D. Anda, E. Arisholm, S. E. Hove, M. Jørgensen, D.I.K. Sjøberg and R. Welland. Collecting Feedback during Software Engineering Experiments, Empirical Software Engineering 10(2):113-147, 2005.

22. D.I.K. Sjøberg, J.E. Hannay, O. Hansen, V.B. Kampenes, A. Karahasanovic, N.K. Liborg and A.C. Rekdal. A Survey of Controlled Experiments in Software Engineering, IEEE Transactions on Software Engineering 31(9):733-753, 2005.

21. E. Arisholm and D.I.K. Sjøberg. Evaluating the Effect of a Delegated versus Centralized Control Style on the Maintainability of Object-Oriented Software, IEEE Transactions on Software Engineering 30(8):521-534, 2004.

20. M. Jørgensen and D.I.K. Sjøberg. The impact of customer expectation on software development effort estimates, International Journal of Project Management 22(4):317-325, 2004.

19. M. Vokác, W. Tichy, D.I.K. Sjøberg, E. Arisholm and M. Aldrin. A Controlled Experiment Comparing the Maintainability of Programs Designed with and without Design Patterns - a Replication in a real Programming Environment, Empirical Software Engineering 9(3):149-195, 2004.

18. M. Jørgensen and D.I.K. Sjøberg. An effort prediction interval approach based on the empirical distribution of previous estimation accuracy, Journal of Information and Software Technology 45(3):123-136, 2003.

17. M. Jørgensen, D.I.K. Sjøberg and U. Indahl. Software Effort Estimation by Analogy and Regression Toward the Mean, Journal of Systems and Software 68(3):253-262, 2003.

16. E. Arisholm, D.I.K. Sjøberg, G. J. Carelius and Y. Lindsjørn. A Web-based Support Environment for Software Engineering Experiments, Nordic Journal of Computing 9(4):231-247, 2002.

15. M. Jørgensen and D.I.K. Sjøberg. Impact of Experience on Maintenance Skills, Software Maintenance: Research and Practice 14(2):123-146, 2002.

14. E. Arisholm, D.I.K. Sjøberg and M. Jørgensen. Assessing the Changeability of two Object-Oriented Design Alternatives - a Controlled Experiment, Empirical Software Engineering 6(3):231-277, 2001.

13. M. Jørgensen and D.I.K. Sjøberg. Impact of Effort Estimates on Software Project Work, Information and Software Technology 43(15):939-948, 2001.

12. M. Jørgensen and D.I.K. Sjøberg. Software Process Improvement and Human Judgement Heuristics, Scandinavian Journal of Information Systems 13(1):99-122, 2001.

11. E. Arisholm and Sjøberg, D.I.K. Towards a Framework for Assessing Changeability Decay. Journal of Systems and Software, 53(1):3-14, 2000

10. K.K. Holgeid, J. Krogstie, D.I.K. Sjøberg: A study of development and maintenance in Norway: assessing the efficiency of information systems support using functional maintenance. Information and Software Technology 42(10):687-700, 2000.

9. A.L. Opdahl, D.I.K. Sjøberg: Programming and Software Development Environment Research in the Nordic Countries. Nordic Journal of Computing 7(2):65-66, 2000.

8. K. Bennett, E. Burd, C. Kemerer, M.M. Lehman, M. Lee, R, Madachy, C. Mair, D.I.K. Sjoberg, S. Slaughter: Empirical Studies of Evolving Systems, Empirical Software Engineering (4):368-378, December 1999.

7. M. Jørgensen, Sjøberg, D.I.K. and Conradi R. Reuse of software development experience - a case study', Telektronikk, vol. 95, no. 1, 1999, pp. 48-54.

6. D.I.K. Sjøberg, R. Welland, R. and M.P. Atkinson. Software Constraints for Large Application Systems, The Computer Journal, Vol. 40, No. 10, pp. 598-616, 1997.

5. Sjøberg, D.I.K., Welland, R., Atkinson, M.P., Philbrow, P. and Waite, C. Exploiting Persistence in Build Management. Software - Practice and Experience. Vol. 27, No. 4, pp. 447-480, April 1997.

4. Sjøberg, D.I.K., Welland, R., Atkinson, M.P., Philbrow, P., Waite, C. and Macneill, S. The Persistent Workshop - a Programming Environment for Napier88. Nordic Journal of Computing, Vol. 4, pp. 123-149, 1997.

3. Sjøberg, D., Quantifying Schema Evolution. Information and Software Technology, Vol. 35, No. 1, pp. 35-44, January 1993.

2. Bendvold, E., Kravdal, Ø., Moe, N., Sjøberg, D. and Skjaeraasen, J., Marital Break-up among Couples Raising Families by Artificial Insemination by Donor. Fertility and Sterility, Vol. 51, No. 6, pp. 980-983, 1989.

1. Lindegaard, K.F., Lundar, T., Wiberg, J., Sjøberg, D., Aaslid, R. and Nornes, H., Variations in Middle Cerebral Artery Blood Flow Investigated With Noninvasive Transcranial Blood Velocity Measurements. Stroke, Vol. 18, No. 6, pp. 1025-1030, 1987.

• Florea, Raluca Madalina; Stray, Viktoria & Sjøberg, Dag (2023). On the roles of software testers: An exploratory study. Journal of Systems and Software. ISSN 0164-1212. 204. doi: 10.1016/j.jss.2023.111742. Fulltekst i vitenarkiv Vis sammendrag

  Context: Software development organizations need testers with high skill levels in a broad range of technical areas and application domains. Accordingly, we need a better understanding of how testers meet such skill demands in the practice of their role. Objective: This work aims to deepen the understanding of the typical tester role. Method: We performed a thematic analysis of 19 in-depth, semi-structured interviews with software testers working in various industries. To investigate employers’ views on such roles, we conducted a thematic analysis of 400 job ads. Results: From the interviews, we identified five subroles of software testers: domain-specific tester, test automation specialist, test infrastructure specialist, user experience tester, and test manager. Most of the practitioners preferred to develop skills and act in one subrole. In contrast, most of the job ads requested that testers act in multiple subroles. Conclusion: Our findings provide a deeper understanding of the tester role, which may guide testers in their acquisition of skills and employers in the recruiting of testers.

• Sjøberg, Dag & Bergersen, Gunnar Rye (2023). Improving the Reporting of Threats to Construct Validity. I Babar, Muhammad Ali (Red.), EASE '23: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. Association for Computing Machinery (ACM). ISSN 9798400700446. doi: 10.1145/3593434.3593449. Fulltekst i vitenarkiv Vis sammendrag

  Background: Construct validity concerns the use of indicators to measure a concept that is not directly measurable. Aim: This study intends to identify, categorize, assess and quantify discussions of threats to construct validity in empirical software engineering literature and use the findings to suggest ways to improve the reporting of construct validity issues. Method: We analyzed 83 articles that report human-centric experiments published in five top-tier software engineering journals from 2015 to 2019. The articles’ text concerning threats to construct validity was divided into segments (the unit of analysis) based on predefined categories. The segments were then evaluated regarding whether they clearly discussed a threat and a construct. Results: Three-fifths of the segments were associated with topics not related to construct validity. Two-thirds of the articles discussed construct validity without using the definition of construct validity given in the article. The threats were clearly described in more than four-fifths of the segments, but the construct in question was clearly described in only two-thirds of the segments. The construct was unclear when the discussion was not related to construct validity but to other types of validity. Conclusions: The results show potential for improving the understanding of construct validity in software engineering. Recommendations addressing the identified weaknesses are given to improve the awareness and reporting of CV.

• Holgeid, Knut Kjetil; Krogstie, John; Mikalef, Patrick; Saur, Eirik Eide & Sjøberg, Dag (2022). Benefits management and Information Technology work distribution. IET Software. ISSN 1751-8806. 16(4), s. 438–454. doi: 10.1049/sfw2.12062. Fulltekst i vitenarkiv Vis sammendrag

  Organisations spend much money on Information Technology (IT) development and maintenance activities with the intention that these activities will create results that enable benefits for the organisations. This paper seeks to understand potential associations between IT development and maintenance activities and the adoption of benefits management practices to realise value for the organization. The aim is also to uncover potential differences between public and private organisations. We surveyed 86 Norwegian public and private organisations, including data collected in similar surveys every five years since 1993. For the period between 1998 and 2018, we observe a stable pattern of IT work distribution. We found that organisations that managed benefits put more effort into advancing functionality for the end-users than other organisations, and they realised more benefits. This advantage was particularly true for organisations that managed benefits beyond the early stages of the development lifecycle. Private organisations both managed and realised benefits to a larger extent than public organisations. Our findings can enable organisations to be evidence-based when choosing management practices to achieve a higher return on investments in IT development and maintenance activities.

• Sjøberg, Dag & Bergersen, Gunnar Rye (2022). Construct Validity in Software Engineering. IEEE Transactions on Software Engineering. ISSN 0098-5589. 49(3), s. 1374–1396. doi: 10.1109/TSE.2022.3176725. Fulltekst i vitenarkiv
• Toledo, Saulo Soares de; Martini, Antonio; Nguyen, Phu Hong & Sjøberg, Dag (2022). Accumulation and Prioritization of Architectural Debt in Three Companies Migrating to Microservices. IEEE Access. 10, s. 37422–37445. doi: 10.1109/ACCESS.2022.3158648. Fulltekst i vitenarkiv Vis sammendrag

  Many companies migrate to microservices because they help deliver value to customers quickly and continuously. However, like any architectural style, microservices are prone to architectural technical debt (ATD), which can be costly if the debts are not timely identified, avoided, or removed. During the early stages of migration, microservice-specific ATDs (MS-ATDs) may accumulate. For example, practitioners may decide to continue using poorly defined APIs in microservices while attempting to maintain compatibility with old functionalities. The riskiest MS-ATDs must be prioritized. Nevertheless, there is limited research regarding the prioritization of MS-ATDs in companies migrating to microservices. This study aims to identify, during migration, which MS-ATDs occur, are the most severe, and are the most challenging to solve. In addition, we propose a way to prioritize these debts. We conducted a multiple exploratory case study of three large companies that were early in the migration process to microservices. We interviewed 47 practitioners with several roles to identify the debts in their contexts. We report the MS-ATDs detected during migration, the MS-ATDs that practitioners estimate to occur in the future, and the MS-ATDs that practitioners report as difficult to solve. We discuss the results in the context of the companies involved in this study. In addition, we used a risk assessment approach to propose a way for prioritizing MS-ATDs. Practitioners from other organizations and researchers may use this approach to provide rankings to help identify and prioritize which MS-ATDs should be avoided or solved in their contexts.

• Toledo, Saulo Soares De; Martini, Antonio & Sjøberg, Dag (2021). Identifying architectural technical debt, principal, and interest in microservices: A multiple-case study. Journal of Systems and Software. ISSN 0164-1212. 177. doi: 10.1016/j.jss.2021.110968. Fulltekst i vitenarkiv Vis sammendrag

  Background: Using a microservices architecture is a popular strategy for software organizations to deliver value to their customers fast and continuously. However, scientific knowledge on how to manage architectural debt in microservices is scarce. Objectives: In the context of microservices applications, this paper aims to identify architectural technical debts (ATDs), their costs, and their most common solutions. Method: We conducted an exploratory multiple case study by conducting 25 interviews with practitioners working with microservices in seven large companies. Results: We found 16 ATD issues, their negative impact (interest), and common solutions to repay each debt together with the related costs (principal). Two examples of critical ATD issues found were the use of shared databases that, if not properly planned, leads to potential breaks on services every time the database schema changes and bad API designs, which leads to coupling among teams. We identified ATDs occurring in different domains and stages of development and created a map of the relationships among those debts. Conclusion: The findings may guide organizations in developing microservices systems that better manage and avoid architectural debts.

• Gjelsten, Børge Kile; Bergersen, Gunnar R.; Sjøberg, Dag & Cutts, Quintin (2021). No Gender Difference in CS1 Grade for Students with Programming from High School: An Exploratory Study, Koli Calling '21: Proceedings of the 21st Koli Calling International Conference on Computing Education Research. ACM Digital Library. ISSN 9781450384889. doi: 10.1145/3488042.3488071. Fulltekst i vitenarkiv
• Toledo, Saulo Soares de; Martini, Antonio; Sjøberg, Dag; Przybyszewska, Agata & Frandsen, Johannes Skov (2021). Reducing Incidents in Microservices by Repaying Architectural Technical Debt, Proceedings of 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665427050. doi: 10.1109/SEAA53835.2021.00033. Fulltekst i vitenarkiv Vis sammendrag

  Architectural technical debt (ATD) may create a substantial extra effort in software development, which is called interest. There is little evidence about whether repaying ATD in microservices reduces such interest. Objectives: We wanted to conduct a first study on investigating the effect of removing ATD on the occurrence of incidents in a microservices architecture. Method: We conducted a quantitative and qualitative case study of a project with approximately 1000 microservices in a large, international financing services company. We measured and compared the number of software incidents of different categories before and after repaying ATD. Results: The total number of incidents was reduced by 84%, and the numbers of critical- and high-priority incidents were both reduced by approximately 90% after the architectural refactoring. The number of incidents in the architecture with the ATD was mainly constant over time, but we observed a slight increase of low priority incidents related to inaccessibility and the environment in the architecture without the ATD. Conclusion: This study shows evidence that refactoring ATDs, such as lack of communication standards, poor management of dead-letter queues, and the use of inadequate technologies in microservices, reduces the number of critical- and high-priority incidents and, thus, part of its interest, although some low priority incidents may increase.

• Holgeid, Knut Kjetil; Jørgensen, Magne; Sjøberg, Dag & Krogstie, John (2021). Benefits management in software development: A systematic review of empirical studies. IET Software. ISSN 1751-8806. 15(1), s. 1–24. doi: 10.1049/sfw2.12007. Fulltekst i vitenarkiv Vis sammendrag

  Considerable resources are wasted on software projects delivering less than the planned benefits. Herein, the objective is to synthesize empirical evidence of the adoption and impact of benefits management (BM) in software development, and to suggest directions for future research. A systematic review of the literature is performed and identified 4836 scientific papers of which the authors found 47 to include relevant research. While most organizations identify and structure benefits at the outset of a project, fewer organizations report implementing BM as a continuous process throughout the project lifecycle. Empirical evidence gives support for positive impact on project outcome from the following BM practices: identifying and structuring benefits, planning benefits realization, BM during project execution, benefits evaluation and the practice of having people responsible for benefits realization. The authors suggest four research directions to understand (1) why BM practices sometimes not are adopted, (2) BM in relation to other management practices, (3) BM in agile software development and (4) BM in the context of organizations' value creation logics.

• Toledo, Saulo Soares De; Martini, Antonio & Sjøberg, Dag I. K. (2020). Improving Agility by Managing Shared Libraries in Microservices. I Paasivaara, Maria & Kruchten, Philippe (Red.), Lecture Notes in Business Information Processing, vol 396. Springer. ISSN 9783030588571. s. 195–202. doi: 10.1007/978-3-030-58858-8_20. Fulltekst i vitenarkiv Vis sammendrag

  Using microservices is a way of supporting an agile architecture. However, if the microservices development is not properly managed, the teams’ development velocity may be affected, reducing agility and increasing architectural technical debt. This paper investigates how to manage the use of shared libraries in microservices to improve agility during development. We interviewed practitioners from four large international companies involved in microservices projects to identify problems when using shared libraries. Our results show that the participating companies had issues with shared libraries as follows: coupling among teams, delays on fixes due to overhead on libraries development teams, and need to maintain many versions of the libraries. Our results highlight that the use of shared libraries may hinder agility on microservices. Thus, their use should be restricted to situations where shared libraries cannot be replaced by a microservice and the costs of replicating the code on each service is very high.

• Stray, Viktoria; Moe, Nils Brede & Sjøberg, Dag (2020). Daily Stand-Up Meetings: Start Breaking the Rules! IEEE Software. ISSN 0740-7459. 37(3), s. 70–77. doi: 10.1109/MS.2018.2875988. Fulltekst i vitenarkiv Vis sammendrag

  Daily stand-up meetings are commonly used for software teams to collaborate and exchange information, but conducting them in a way that benefits the whole team can be challenging. We describe factors that can affect meetings and propose recommendations for improving them.

• Toledo, Saulo Soares De; Martini, Antonio; Przybyszewska, Agata & Sjøberg, Dag (2019). Architectural Technical Debt in Microservices: A Case Study in a Large Company. I Avgeriou, Paris & Schmid, Klaus (Red.), Proceedings of the Second International Conference on Technical Debt. IEEE Press. ISSN 9781728133713. doi: 10.1109/TechDebt.2019.00026. Fulltekst i vitenarkiv Vis sammendrag

  Introduction: Software companies aim to achieve continuous delivery to constantly provide value to their customers. A popular strategy is to use microservices architecture. However, such an architecture is also subject to debt, which hinders the continuous delivery process and thus negatively affects the software released to the customers. Objectives: The aim of this study is to identify issues, solutions and risks related to Architecture Technical Debt in microservices. Method: We conducted an exploratory case study of a real life project with about 1000 services in a large, international company. Through qualitative analysis of documents and interviews, we investigated Architecture Technical Debt in the communication layer of a system with microservices architecture. Results: Our main contributions are a list of Architecture Technical Debt issues specific for the communication layer in a system with microservices architecture, as well as their associated negative impact (interest), a solution to repay the debt and the its cost (principal). Among the found Architecture Technical Debt issues were the existence of business logic in the communication layer and a high amount of point-to-point connections between services. The studied solution consists of the implementation of different canonical models specific to different domains, the removal of business logic from the communication layer, and migration from services to use the communication layer correctly. We also contributed with a list of possible risks that can affect the payment of the debt, as lack of funding and inadequate prioritization. Conclusion: We found issues, solutions and possible risks that are specific for microservices architectures not yet encountered in the current literature. Our results may be useful for practitioners that want to avoid or repay Technical Debt in their microservices architecture.

• Lindsjørn, Yngve; Bergersen, Gunnar R.; Dingsøyr, Torgeir & Sjøberg, Dag (2018). Teamwork Quality and Team Performance: Exploring Differences Between Small and Large Agile Projects. I Garbajosa, Juan; Wang, Xiaofeng & Aguiar, Ademar (Red.), Agile Processes in Software Engineering and Extreme Programming, 19th International Conference, XP 2018, Proceedings. Springer. ISSN 9783319916019. s. 267–274. doi: 10.1007/978-3-319-91602-6_19. Fulltekst i vitenarkiv Vis sammendrag

  Agile principles were originally developed for small projects but are now widely used in larger projects with hundreds of developers. Teamwork quality is essential in any development work, but how does teamwork quality differ in small and large agile projects? We report from an explorative survey with 64 agile teams and 320 team members and team leaders, from 31 teams in small projects and 33 teams in large projects. For small projects, teamwork quality was considered by both team members and team leaders to primarily affect product quality. For large projects, the effect of teamwork quality on product quality was positive when it was rated by team members but was negative when rated by team leaders. At a finer granularity, the six dimensions of teamwork quality that we investigated affected team performance differently in small and large projects. These findings question to what extent findings from previous studies on teamwork in agile development in small projects apply to large projects.

• Sjøberg, Dag (2018). An empirical study of WIP in kanban teams. I Oivo, Markku; Fernández, Daniel Méndez & Mockus, Audris (Red.), Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM, Oulu, Finland — October 11 - 12, 2018. Association for Computing Machinery (ACM). ISSN 9781450358231. s. 13.1–13.8. doi: 10.1145/3239235.3239238. Fulltekst i vitenarkiv Vis sammendrag

  Background: Limiting the amount of Work-In-Progress (WIP) is considered a fundamental principle in Kanban software development. However, no published studies from real cases exist that indicate what an optimal WIP limit should be. Aims: The primary aim is to study the effect of WIP on the performance of a Kanban team. The secondary aim is to illustrate methodological challenges when attempting to identify an optimal or appropriate WIP limit. Method: A quantitative case study was conducted in a software company that provided information about more than 8,000 work items developed over four years by five teams. Relationships between WIP, lead time and productivity were analyzed. Results: WIP correlates with lead time; that is, lower WIP indicates shorter lead times, which is consistent with claims in the literature. However, WIP also correlates with productivity, which is inconsistent with the claim in the literature that a low WIP (still above a certain threshold) will improve productivity. The collected data set did not include sufficient information to measure aspects of quality. There are several threats to the way productivity was measured. Conclusions: Indicating an optimal WIP limit is difficult in the studied company because a changing WIP gives contrasting results on different team performance variables. Because the effect of WIP has not been quantitatively examined before, this study clearly needs to be replicated in other contexts. In addition, studies that include other team performance variables, such as various aspects of quality, are requested. The methodological challenges illustrated in this paper need to be addressed.

• Feldt, robert; Zimmermann, Thomas; Bergersen, Gunnar R.; Falessi, Davide; Jedlitschka, Andreas & Juristo, Natalia [Vis alle 12 forfattere av denne artikkelen] (2018). Four commentaries on the use of students and professionals in empirical software engineering experiments. Empirical Software Engineering. ISSN 1382-3256. 23(6), s. 3801–3820. doi: 10.1007/s10664-018-9655-0. Fulltekst i vitenarkiv Vis sammendrag

  The relative pros and cons of using students or practitioners in experiments in empirical software engineering have been discussed for a long time and continue to be an important topic. Following the recent publication of “Empirical software engineering experts on the use of students and professionals in experiments” by Falessi, Juristo, Wohlin, Turhan, Münch, Jedlitschka, and Oivo (EMSE, February 2018) we received a commentary by Sjøberg and Bergersen. Given that the topic is of great methodological interest to the community and requires nuanced treatment, we invited two editorial board members, Martin Shepperd and Per Runeson, respectively, to provide additional views

• Sjøberg, Dag (2016). The relationship between software process, context and outcome. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 10027, s. 3–11. doi: 10.1007/978-3-319-49094-6_1. Fulltekst i vitenarkiv Vis sammendrag

  Most practitioners and researchers agree that when developing software, process affects product, and the usefulness of a process depends on the context. However, which processes are most useful for a specific company or project is generally unknown. When studying the relation between context, process and product, one challenge is that experiments often lack realism, which makes the transfer of results to industry difficult. In contrast, most of the important factors vary beyond the researcher’s control in case studies, which makes it difficult to identify cause and effect relationships. This paper reports a study where realism was combined with control over certain context and process factors. Four companies developed the same system, and the price varied by a factor of six. Certain patterns of relationships were expected (expensive company, low cost, schedule overrun); others were unexpected (cheap company, maintainable system because of small code). The community needs to identify the most important relationships among process, context and outcome.

• Lindsjørn, Yngve; Sjøberg, Dag; Dingsøyr, Torgeir; Bergersen, Gunnar R. & Dybå, Tore (2016). Teamwork Quality and Project Success in Software Development: A Survey of Agile Development Teams. Journal of Systems and Software. ISSN 0164-1212. 122, s. 274–286. doi: 10.1016/j.jss.2016.09.028. Fulltekst i vitenarkiv Vis sammendrag

  Small, self-directed teams are central in agile development. This article investigates the effect of team- work quality on team performance, learning and work satisfaction in agile software teams, and whether this effect differs from that of traditional software teams. A survey was administered to 477 respondents from 71 agile software teams in 26 companies and analyzed using structural equation modeling. A posi- tive effect of teamwork quality on team performance was found when team members and team leaders rated team performance. In contrast, a negligible effect was found when product owners rated team per- formance. The effect of teamwork quality on team members ́learning and work satisfaction was strongly positive, but was only rated by the team members. Despite claims of the importance of teamwork in ag- ile teams, this study did not find teamwork quality to be higher than in a similar survey on traditional teams. The effect of teamwork quality on team performance was only marginally greater for the agile teams than for the traditional teams.

• Dybå, Tore; Bergersen, Gunnar R. & Sjøberg, Dag (2016). Evidence-Based Software Engineering. I Menzies, Tim; Williams, Laurie & Zimmermann, Thomas D. (Red.), Perspectives on Data Science for Software Engineering. Elsevier. ISSN 9780128042069. s. 149–153. doi: 10.1016/B978-0-12-804206-9.00029-5. Fulltekst i vitenarkiv Vis sammendrag

  A decade ago, Kitchenham, Dybå and Jørgensen coined the term and provided the foundations for evidence-based software engineering (EBSE). A trilogy of papers was written for researchers, practitioners, and educators. They suggested that practitioners consider EBSE as a mechanism to support and improve their technology adoption decisions, and that researchers should use systematic literature reviews as a methodology for performing unbiased aggregation of empirical results. This spurred significant international activity, and a renewed focus on research methods and theory, and on the future of empirical methods in SE research.

• Sjøberg, Dag; Bergersen, Gunnar R. & Dybå, Tore (2016). Why Theory Matters. I Menzies, Tim; Williams, Laurie & Zimmermann, Thomas D. (Red.), Perspectives on Data Science for Software Engineering. Elsevier. ISSN 9780128042069. s. 29–33. doi: 10.1016/B978-0-12-804206-9.00006-4. Fulltekst i vitenarkiv Vis sammendrag

  It is relatively easy to generate and acquire much data from software engineering activities. The challenge is to obtain meaning from the data that represents something true, rather than spurious. To increase knowledge and insight, more theories should be built and used.

• Stray, Viktoria; Sjøberg, Dag & Dybå, Tore (2016). The Daily Stand-up Meeting: A Grounded Theory Study. Journal of Systems and Software. ISSN 0164-1212. 114, s. 101–124. doi: 10.1016/j.jss.2016.01.004. Fulltekst i vitenarkiv Vis sammendrag

  The daily stand-up meeting is one of the most used agile practices but has rarely been the subject of empirical research. The present study aims to identify how daily stand-up meetings are conducted and what the attitudes towards them are. A grounded theory study with 12 software teams in three companies in Malaysia, Norway, Poland and the United Kingdom was conducted. We interviewed 60 people, observed 79 daily stand-up meetings and collected supplementary data. The factors that contributed the most to a positive attitude towards the daily stand-up meeting were information sharing with the team and the opportunity to discuss and solve problems. The factors that contributed the most to a negative attitude were status reporting to the manager and that the frequency of the meeting was perceived to be too high and the duration too long. Based on our results, we developed a grounded theory of daily stand-up meetings and proposed empirically based recommendations and guidelines on how to organize them. Organizations should be aware of the factors that may affect the attitude towards daily stand-up meetings and should consider our recommendations and guidelines to make this agile practice as valuable as possible.

• Jørgensen, Magne; Dybå, Tore; Liestøl, Knut & Sjøberg, Dag (2016). Incorrect results in software engineering experiments: How to improve research practices. Journal of Systems and Software. ISSN 0164-1212. 116, s. 133–145. doi: 10.1016/j.jss.2015.03.065. Fulltekst i vitenarkiv Vis sammendrag

  Context The trustworthiness of research results is a growing concern in many empirical disciplines. Aim The goals of this paper are to assess how much the trustworthiness of results reported in software engineering experiments is affected by researcher and publication bias, given typical statistical power and significance levels, and to suggest improved research practices. Method First, we conducted a small-scale survey to document the presence of researcher and publication biases in software engineering experiments. Then, we built a model that estimates the proportion of correct results for different levels of researcher and publication bias. A review of 150 randomly selected software engineering experiments published in the period 2002–2013 was conducted to provide input to the model. Results The survey indicates that researcher and publication bias is quite common. This finding is supported by the observation that the actual proportion of statistically significant results reported in the reviewed papers was about twice as high as the one expected assuming no researcher and publication bias. Our models suggest a high proportion of incorrect results even with quite conservative assumptions. Conclusion Research practices must improve to increase the trustworthiness of software engineering experiments. A key to this improvement is to avoid conducting studies with unsatisfactory low statistical power.

• Bergersen, Gunnar R.; Sjøberg, Dag & Dybå, Tore (2014). Construction and Validation of an Instrument for Measuring Programming Skill. IEEE Transactions on Software Engineering. ISSN 0098-5589. 40(12), s. 1163–1184. doi: 10.1109/TSE.2014.2348997. Fulltekst i vitenarkiv Vis sammendrag

  Skilled workers are crucial to the success of software development. The current practice in research and industry for assessing programming skills is mostly to use proxy variables of skill, such as education, experience, and multiple-choice knowledge tests. There is as yet no valid and efficient way to measure programming skill. The aim of this research is to develop a valid instrument that measures programming skill by inferring skill directly from the performance on programming tasks. Over two days, 65 professional developers from eight countries solved 19 Java programming tasks. Based on the developers' performance, the Rasch measurement model was used to construct the instrument. The instrument was found to have satisfactory (internal) psychometric properties and correlated with external variables in compliance with theoretical expectations. Such an instrument has many implications for practice, for example, in job recruitment and project allocation.

• Stray, Viktoria Gulliksen; Lindsjørn, Yngve & Sjøberg, Dag (2013). Obstacles to Efficient Daily Meetings in Agile Development Projects: A Case Study. I Falessi, Davide (Red.), ACM / IEEE International Symposium on Empirical Software Engineering and Measurement. Baltimore, 10-11 Oct. 2013. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9780769550565. s. 95–102. doi: 10.1109/ESEM.2013.30. Fulltekst i vitenarkiv Vis sammendrag

  Context: Most of the software organizations that use agile methods organize daily team meetings. Aim: Our aim was to understand how daily meetings are conducted and identify obstacles that reduce their efficiency. Method: We observed 56 daily meetings and conducted 21 interviews in three different teams in two countries. We used the repertory grid technique in the interviews and to analyze the results. Results: We identified thirteen obstacles. The four most prominent ones were: (1) The daily meetings lasted too long (on average, 22 minutes instead of the scheduled 15 minutes). (2) In the meetings that were not self-organized, team members reported to the Scrum Master instead of sharing information among all team members. (3) The interruption caused by daily meetings required substantially more time than the actual meeting time due to overhead before and after the meetings. (4) Several team members had negative attitudes towards the daily meetings. Conclusion: Organizers of daily meetings should evaluate whether the obstacles we have identified are present in their organization and consider our suggestions to remove or reduce these obstacles.

• Kitchenham, Barbara; Sjøberg, Dag; Dybå, Tore; Brereton, O. Pearl; Budgen, David & Höst, Martin [Vis alle 7 forfattere av denne artikkelen] (2013). Trends in the Quality of Human-Centric Software Engineering Experiments-A Quasi-Experiment. IEEE Transactions on Software Engineering. ISSN 0098-5589. 39(7), s. 1002–1017. doi: 10.1109/TSE.2012.76. Fulltekst i vitenarkiv Vis sammendrag

  Context: Several text books and papers published between 2000 and 2002 have attempted to introduce experimental design and statistical methods to software engineers undertaking empirical studies. Objective: This paper investigates whether there has been an increase in the quality of human-centric experimental and quasi-experimental journal papers over the time period 1993 to 2010. Method: Seventy experimental and quasi-experimental papers published in four general software engineering journals in the years 1992-2002 and 2006-2010 were each assessed for quality by three empirical software engineering researchers using two quality assessment methods (a questionnaire-based method and a subjective overall assessment). Regression analysis was used to assess the relationship between paper quality and the year of publication, publication date group (before 2003 and after 2005), source journal, average coauthor experience, citation of statistical text books and papers, and paper length. The results were validated both by removing papers for which the quality score appeared unreliable and using an alternative quality measure. Results: Paper quality was significantly associated with year, citing general statistical texts, and paper length (p <; 0.05). Paper length did not reach significance when quality was measured using an overall subjective assessment. Conclusions: The quality of experimental and quasi-experimental software engineering papers appears to have improved gradually since 1993.

• Sjøberg, Dag; Yamashita, Aiko Amparo Fallas; Anda, Bente Cecilie D; Mockus, Audris & Dybå, Tore (2013). Quantifying the Effect of Code Smells on Maintenance Effort. IEEE Transactions on Software Engineering. ISSN 0098-5589. 39(8), s. 1144–1156. doi: 10.1109/TSE.2012.89. Fulltekst i vitenarkiv Vis sammendrag

  Context: Code smells are assumed to indicate bad design that leads to less maintainable code. However, this assumption has not been investigated in controlled studies with professional software developers. Aim: This paper investigates the relationship between code smells and maintenance effort. Method: Six developers were hired to perform three maintenance tasks each on four functionally equivalent Java systems originally implemented by different companies. Each developer spent three to four weeks. In total, they modified 298 Java files in the four systems. An Eclipse IDE plug-in measured the exact amount of time a developer spent maintaining each file. Regression analysis was used to explain the effort using file properties, including the number of smells. Results: None of the 12 investigated smells was significantly associated with increased effort after we adjusted for file size and the number of changes; Refused Bequest was significantly associated with decreased effort. File size and the number of changes explained almost all of the modeled variation in effort. Conclusion: The effects of the 12 smells on maintenance effort were limited. To reduce maintenance effort, a focus on reducing code size and the work practices that limit the number of changes may be more beneficial than refactoring code smells.

• Kitchenham, Barbara Ann; Sjøberg, Dag; Dybå, Tore; Pfahl, Dietmar Alfred; Brereton, Pearl & Budgen, David [Vis alle 8 forfattere av denne artikkelen] (2012). Three empirical studies on the agreement of reviewers about the quality of software engineering experiments. Information and Software Technology. ISSN 0950-5849. 54(8), s. 804–819. doi: 10.1016/j.infsof.2011.11.008. Fulltekst i vitenarkiv
• Bergersen, Gunnar R. & Sjøberg, Dag (2012). Evaluating Methods and Technologies in Software Engineering with Respect to Developers’ Skill Level. I Mendes, Emilia & Genero, Marcela (Red.), Proceedings, 16th International Conference on Evaluation and Assessment in Software Engineering. IET Research Journals. ISSN 9781849195416. s. 101–110. Fulltekst i vitenarkiv
• Sjøberg, Dag; Johnsen, Anders & Solberg, Jørgen (2012). Quantifying the Effect of Using Kanban versus Scrum: A Case Study. IEEE Software. ISSN 0740-7459. 29(5), s. 47–53. doi: 10.1109/MS.2012.110. Fulltekst i vitenarkiv
• Sjøberg, Dag; Anda, Bente Cecilie D & Mockus, Audris (2012). Questioning software maintenance metrics: a comparative case study. I Runeson, Per; Høst, Martin; Mendes, Emilia; Andrews, Anneliese Amschler & Harrison, Rachel (Red.), ESEM '12. Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781450310567. s. 107–110. doi: 10.1145/2372251.2372269. Fulltekst i vitenarkiv
• Dybå, Tore; Sjøberg, Dag & Cruzes, Daniela Soares (2012). What works for whom, where, when, and why?: on the role of context in empirical software engineering. I Runeson, Per; Høst, Martin; Mendes, Emilia; Andrews, Anneliese Amschler & Harrison, Rachel (Red.), ESEM '12. Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781450310567. s. 19–28. doi: 10.1145/2372251.2372256. Fulltekst i vitenarkiv Vis sammendrag

  Context is a central concept in empirical software engineering. It is one of the distinctive features of the discipline and it is an in-dispensable part of software practice. It is likely responsible for one of the most challenging methodological and theoretical problems: study-to-study variation in research findings. Still, empirical software engineering research is mostly concerned with attempts to identify universal relationships that are independent of how work settings and other contexts interact with the processes important to software practice. The aim of this paper is to provide an overview of how context affects empirical research and how empirical software engineering research can be better 'contextualized' in order to provide a better understanding of what works for whom, where, when, and why. We exemplify the importance of context with examples from recent systematic reviews and offer recommendations on the way forward.

• Bergersen, Gunnar; Hannay, Jo Erskine; Sjøberg, Dag; Dybå, Tore & Karahasanovic, Amela (2011). Inferring Skill from Tests of Programming Performance: Combining Time and Quality, Proceedings of International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781457722035. s. 305–314. doi: 10.1109/esem.2011.39. Fulltekst i vitenarkiv Vis sammendrag

  The skills of software developers are important to the success of software projects. Also, when studying the general effect of a tool or method, it is important to control for individual differences in skill. However, the way skill is assessed is often ad hoc, or based on unvalidated methods. According to established test theory, validated tests of skill should infer skill levels from well-defined performance measures on multiple, small, representative tasks. In this respect, we show how time and quality, which are often analyzed separately, can be combined as task performance and subsequently be aggregated as an approximation of skill. Our results show significant positive correlations between our proposed measures of skill and other variables, such as seniority, lines of code written, and self-evaluated expertise. The method for combining time and quality is a promising first step to measuring programming skill in both industry and research settings.

• Sjøberg, Dag; Odberg, Erik & Warlo, Borger (2010). The Challenge of Assessing and Controlling Complexity in a Large Portfolio of Software Systems. I Hall, Tracy; Høst, Martin; Niazi, Mahmood; Mishra, Alok & Falessi, Davide (Red.), Product Focused Software Development and Process Improvement (PROFES 2010, Limerick, Ireland, June 21-23. 2010). Universitalia. ISSN 9788865070673. s. 75–78. Fulltekst i vitenarkiv
• Hannay, Jo Erskine; Arisholm, Erik; Engvik, Harald & Sjøberg, Dag (2010). Effects of Personality on Pair Programming. IEEE Transactions on Software Engineering. ISSN 0098-5589. 36(1), s. 61–80. doi: 10.1109/TSE.2009.41. Fulltekst i vitenarkiv
• Olbrich, Steffen; Cruzes, Daniela Soares & Sjøberg, Dag (2010). Are all code smells harmful? A study of God Classes and Brain Classes in the evolution of three open source systems. Proceedings - Conferense on Software Maintenance. ISSN 1063-6773. 1, s. 1–10. doi: 10.1109/ICSM.2010.5609564. Fulltekst i vitenarkiv
• Sjøberg, Dag (2010). Confronting the Myth of Rapid Obsolescence in Computing Research. Communications of the ACM. ISSN 0001-0782. 53(9), s. 62–67. doi: 10.1145/1810891.1810911. Fulltekst i vitenarkiv
• Kitchenham, B.; Sjøberg, Dag; Brereton, O. Pearl; Budgen, David; Dybå, Tore & Høst, Martin [Vis alle 8 forfattere av denne artikkelen] (2010). Can we evaluate the quality of software engineering experiments? I Morisio, Maurizio (Red.), Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781450300391. Fulltekst i vitenarkiv
• Følstad, Asbjørn; Anda, Bente Cecilie D & Sjøberg, Dag (2010). The usability inspection performance of work-domain experts: An empirical study. Interacting with computers. ISSN 0953-5438. 22(2), s. 75–87. doi: 10.1016/j.intcom.2009.09.001. Fulltekst i vitenarkiv
• Sjøberg, Dag & Grimstad, Stein (2010). Software Engineering: Why - How - What - What's Next. I Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (Red.), Simula Research Laboratory - by thinking constantly about it. Springer. ISSN 9783642011559. s. 363–368. Fulltekst i vitenarkiv
• Sjøberg, Dag (2010). The Industry is our Lab – Organization and Conducting of Empirical Studies in Software Engineering at Simula. I Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (Red.), Simula Research Laboratory - by thinking constantly about it. Springer. ISSN 9783642011559. s. 443–458. Fulltekst i vitenarkiv
• Sjøberg, Dag; Grimstad, Stein; Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (2009). Software Engineering: Why - How - What - What's Next. I Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (Red.), Simula Research Laboratory - by thinking constantly about it. Springer. ISSN 9783642011559. s. 363–368. Fulltekst i vitenarkiv
• Sjøberg, Dag; Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (2009). The Industry Is Our Lab - Organization and Conducting of Empirical Studies in Software Engineering at Simula. I Tveito, Aslak; Bruaset, Are Magnus & Lysne, Olav (Red.), Simula Research Laboratory - by thinking constantly about it. Springer. ISSN 9783642011559. s. 443–458. Fulltekst i vitenarkiv Vis sammendrag

  The society is completely dependent on reliable and efficient software. Development of such software requires the use of appropriate processes, method and tools. To evaluate their appropriateness in various industrial settings, empirical studies are needed. At present, most studies in software engineering are run with students in artificial settings. The strategy of Simula Research Laboratory to invest in experiment infrastructures and hire professionals as subjects has contributed to a significant improvement of state of the art regarding the realism and scale of experiments and control in real-life case studies. During its first eight years, Simula has conducted studies that have included about 6000 professional software developers and managers from 273 companies in 25 countries. However, increasing the realism of tasks and systems in experiments and control in case studies is still a challenge for Simula and the rest of the research community.

• Hannay, Jo; Dybå, Tore; Arisholm, Erik & Sjøberg, Dag (2009). The Effectiveness of Pair-Programming: a Meta-Analysis. Information and Software Technology. ISSN 0950-5849. 55, s. 1110–1122. Fulltekst i vitenarkiv Vis sammendrag

  Several experiments on the effects of pair versus solo programming have been reported in the literature. We present a metaanalysis of these studies. The analysis shows a small significant positive overall effect of pair programming on quality, a medium significant positive overall effect on duration, and a medium significant negative overall effect on effort. However, between-study variance is significant, and there are signs of publication bias among published studies on pair programming. A more detailed examination of the evidence suggests that pair programming is faster than solo programming when programming task complexity is low and yields code solutions of higher quality when task complexity is high. The higher quality for complex tasks comes at a price of considerably greater effort, while the reduced completion time for the simpler tasks comes at a price of noticeably lower quality. We conclude that greater attention should be given to moderating factors on the effects of pair programming.

• Hannay, Jo; Arisholm, Erik; Engvik, Harald & Sjøberg, Dag (2009). Effects of Personality on Pair Programming. IEEE Transactions on Software Engineering. ISSN 0098-5589. 36, s. 61–80. Fulltekst i vitenarkiv Vis sammendrag

  Personality tests in various guises are commonly used in recruitment and career counseling industries. Such tests have also been considered as instruments for predicting programming and team performance in software engineering. However, research suggests that other human-related factors, such as motivation, general mental ability, expertise and task complexity also affect performance in general. This paper reports on a study of the impact of Big-Five personality traits on the performance of pair programmers together with the impact of expertise and task complexity. The study involved 196 software professionals from three countries forming 98 pairs. The analysis consisted of a confirmatory part and an exploratory part, and the results show that (1) our data does not confirm a meta-analysis-based model of the impact of certain personality traits on performance, and (2) personality traits in general have modest predictive value on pair programming performance compared with expertise, task complexity, and even country. Rather than focusing on direct effects of personality on pair programming performance, we conclude that (a) effort should be spent on elaborating on personality's (and other factors') indirect effects on performancemediated by formalized scales of collaboration, and (b) more effort should be spent on investigating other performance-related predictors such as programming skill, learning, motivation, expertise, and task complexity.

• Hannay, Jo Erskine; Dybå, Tore; Arisholm, Erik & Sjøberg, Dag (2009). The effectiveness of pair programming: A meta-analysis. Information and Software Technology. ISSN 0950-5849. 51(7), s. 1110–1122. doi: 10.1016/j.infsof.2009.02.001. Fulltekst i vitenarkiv
• Anda, Bente Cecilie D; Sjøberg, Dag & Mockus, Audris (2009). Variability and Reproducibility in Software Engineering: A Study of Four Companies that Developed the Same System. IEEE Transactions on Software Engineering. ISSN 0098-5589. 35(3), s. 407–429. doi: 10.1109/TSE.2008.89. Fulltekst i vitenarkiv
• Karahasanovic, Amela; Hinkel, Unni Nyhamar; Sjøberg, Dag & Thomas, R. (2009). Comparing of feedback-collection and think-aloud methods in program comprehension studies. Behaviour and Information Technology. ISSN 0144-929X. 28(2), s. 139–164. doi: 10.1080/01449290701682761. Fulltekst i vitenarkiv
• Kampenes, Vigdis By; Dybå, T.; Hannay, Jo Erskine & Sjøberg, Dag (2009). A systematic review of quasi-experiments in software engineering. Information and Software Technology. ISSN 0950-5849. 51(1), s. 71–82. doi: 10.1016/j.infsof.2008.04.006. Fulltekst i vitenarkiv
• Yamashita, Aiko Amparo Fallas; Benestad, Hans Christian; Anda, Bente Cecilie D; Arnstad, P.E.; Sjøberg, Dag & Moonen, L. (2009). Using Concept Mapping for Maintainability Assessments. I Carver, Jeffrey (Red.), 3rd IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2009). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781424448425. s. 378–389. Fulltekst i vitenarkiv
• Sjøberg, Dag; Dybå, T.; Anda, Bente Cecilie D & Hannay, Jo Erskine (2008). Building Theories in Software Engineering. I Shull, Forrest; Singer, Janice & Sjøberg, Dag (Red.), Advanced Topics in Empirical Software Engineering. Springer. ISSN 9781848000438. Fulltekst i vitenarkiv
• Hannay, Jo Erskine; Sjøberg, Dag & Dybå, Tore (2007). A Systematic Review of Theory Use in Software Engineering Experiments. IEEE Transactions on Software Engineering. ISSN 0098-5589. 33(2), s. 87–107. Fulltekst i vitenarkiv
• Kampenes, Vigdis By; Dyba, T; Hannay, Jo Erskine & Sjøberg, Dag (2007). A systematic review of effect size in software engineering experiments. Information and Software Technology. ISSN 0950-5849. 49, s. 1073–1086. doi: 10.1016/j.infsof.2007.02.015. Fulltekst i vitenarkiv
• Sjøberg, Dag (2007). Documenting Theories, Empirical Software Engineering Issues: Critical Assessment and Future Directions. Springer. ISSN 9783540713005. s. 111–114. Fulltekst i vitenarkiv
• Sjøberg, Dag; Dybå, Tore & Jørgensen, Magne (2007). The Future of Empirical Methods in Software Engineering Research. I Briand, Lionel & Wolf, Alexander (Red.), Future of Software Engineering. IEEEE-CS. ISSN 9780769528298. s. 358–378. Fulltekst i vitenarkiv
• Dybå, Tore; Arisholm, Erik; Sjøberg, Dag; Hannay, J.E. & Shull, Forrest (2007). Are Two Heads Better than One? On the Effectiveness of Pair Programming. IEEE Software. ISSN 0740-7459. 24(6), s. 12–15. Fulltekst i vitenarkiv
• Basili, Victor R.; Zelkowitz, Marvin V.; Sjøberg, Dag & Johnson, Philip (2007). Protocols in the use of Empirical Software Engineering Artifacts. Empirical Software Engineering. ISSN 1382-3256. 12(1), s. 107–119. Fulltekst i vitenarkiv
• Arisholm, Erik; Gallis, Hans; Dybå, Tore & Sjøberg, Dag (2007). Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise. IEEE Transactions on Software Engineering. ISSN 0098-5589. 333(2), s. 65–86. Fulltekst i vitenarkiv
• Sjøberg, Dag (2007). Knowledge Acquisition in Software Engineering Requires Sharing of Data and Artifacts, Empirical Software Engineering Issues: Critical Assessment and Future Directions. Springer. ISSN 9783540713005. s. 77–82. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2006). Expert Estimation of Software Development Work. I Madhavji, Nazim; Fernandez-Ramli, Juan & Perry, Dewayne (Red.), Software Evolution and Feedback: Theory and Practice. IEEE Press. ISSN 9780470871805. s. 523–527. Fulltekst i vitenarkiv
• Dybå, Tore; Kampenes, Vigdis By & Sjøberg, Dag (2006). A systematic review of statistical power in software engineering experiments. Information and Software Technology. ISSN 0950-5849. 48(8), s. 745–755. Fulltekst i vitenarkiv
• Karahasanovic, Amela; Fjuk, Annita; Sjøberg, Dag & Thomas, R. (2006). Revealing object-oriented comprehension by feedback collection. I Fjuk, Annita; Karahasanovic, Amela & Kaasbøll, Jens Johan (Red.), Comprehensive Object-Oriented Learning: The Learner's Perspective. Informing Science Press. ISSN 9788392233749. s. 111–130. Fulltekst i vitenarkiv
• Conradi, Reidar; Dybå, Tore; Sjøberg, Dag I. K. & Ulsund, Tor (2006). Lessons Learned and Recommendations from two Large Norwegian SPI Programmes. I Conradi, Reidar; Dybå, Tore; Sjøberg, Dag I. K. & Ulsund, Tor (Red.), Software Process Improvement: Results and Experience from the Field. Springer. ISSN 9783540321781. s. 3–18. Fulltekst i vitenarkiv Vis sammendrag

  Software development is an experimental discipline, i.e. somewhat unpredictable. This suggests that software processes improvement should be based on the continuous iteration of characterization, goal setting, selection of improved technology, monitoring and analysis of its effects. This paper describes experiences from the empirical studies in two large SPI programmes in Norway. Five main lessons were learned: 1) It is a challenge for the industrial partners to invest enough resources in SPI activities. 2) The research partners must learn to know the companies, and 3) they must work as a multicompetent and coherent unit towards them. 4) Any SPI initiative must show visible, short-term payoff. 5) Establishing a solid baseline from which to improve is unrealistic. Based on these lessons, a set of operational recommendations for other researchers in the area are proposed. Also published in Flavio Oquendo (Ed.), 9th Int'l. Workshop on Software Process Technology, Helsinki, Finland, ISBN 3-540-40764-2

• Krogstie, John; Jahr, Arthur & Sjøberg, Dag I. K. (2006). A longitudinal study of development and maintenance in Norway: Report from the 2003 investigation. Information and Software Technology. ISSN 0950-5849. 48, s. 993–1005. Fulltekst i vitenarkiv
• Anda, Bente Cecilie D & Sjøberg, Dag (2005). Investigating the Role of Use Cases in the Construction of Class Diagrams. Empirical Software Engineering. ISSN 1382-3256. 10(3), s. 285–309. Fulltekst i vitenarkiv
• Karahasanovic, Amela; Anda, Bente Cecilie D; Arisholm, Erik; Hove, SE; Jørgensen, Magne & Sjøberg, Dag [Vis alle 7 forfattere av denne artikkelen] (2005). Collecting Feedback during Software Engineering Experiments. Empirical Software Engineering. ISSN 1382-3256. 10(2), s. 113–147. Fulltekst i vitenarkiv
• Sjøberg, Dag; Hannay, J.E.; Kampenes, V.B.; Hansen, O; Karahasanovic, Amela & Rekdal, A.C. [Vis alle 7 forfattere av denne artikkelen] (2005). A Survey of Controlled Experiments in Software Engineering. IEEE Transactions on Software Engineering. ISSN 0098-5589. 31(9), s. 733–753. Fulltekst i vitenarkiv
• Benestad, Hans Christian; Arisholm, Erik & Sjøberg, Dag (2005). How to Recruit Professionals as Subjects in Software Engineering Experiments. I Hustad, E.; Munkvold, B.E.; Rolland, K & Flak, L.S. (Red.), IRIS (Information Systems Research in Scandinavia). Department of Information Systems. ISSN 9788271175542. Fulltekst i vitenarkiv
• Karahasanovic, Amela; Fjuk, A; Sjøberg, Dag & Thomas, R. (2004). A Controlled Experiment to Evaluate the Reactivity and Usefulness of the Think-Aloud Tool. I Khosrowpour, M (Red.), Information Resources Management Association International Conference (IRMA'04), Computer Interaction Track. Idea Group Publishing. s. 1033–1034. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2004). The impact of customer expectation on software development effort estimates. International Journal of Project Management. ISSN 0263-7863. 22(4), s. 317–325. Fulltekst i vitenarkiv
• Arisholm, Erik & Sjøberg, Dag (2004). Evaluating the Effect of a Delegated versus Centralized Control Style on the Maintainability of Object-Oriented Software. IEEE Transactions on Software Engineering. ISSN 0098-5589. 30(8), s. 521–534. Fulltekst i vitenarkiv
• Vokác, Marek; Tichy, Walter; Sjøberg, Dag I. K.; Arisholm, Erik & Aldrin, Magne (2004). A Controlled Experiment Comparing the Maintainability of Programs Designed with and without Design Patterns -- a Replication in a real Programming Environment. Empirical Software Engineering. ISSN 1382-3256. 9(3), s. 149–195. doi: 10.1023/EMSE.0000027778.69251.1f. Fulltekst i vitenarkiv Vis sammendrag

  Software “design patterns” seek to package proven solutions to design problems in a form that makes it possible to find, adapt and reuse them. To support the industrial use of design patterns, this research investigates when, and how, using patterns is beneficial, and whether some patterns are more difficult to use than others. This paper describes a replication of an earlier controlled experiment on design patterns in maintenance, with major extensions. Experimental realism was increased by using a real programming environment instead of pen and paper, and paid professionals from multiple major consultancy companies as subjects. Measurements of elapsed time and correctness were analyzed using regression models and an estimation method that took into account the correlations present in the raw data. Together with on-line logging of the subjects’ work, this made possible a better qualitative understanding of the results. The results indicate quite strongly that some patterns are much easier to understand and use than others. In particular, the Visitor pattern caused much confusion. Conversely, the patterns Observer and, to a certain extent, Decorator were grasped and used intuitively, even by subjects with little or no knowledge of patterns. The implication is that design patterns are not universally good or bad, but must be used in a way that matches the problem and the people. When approaching a program with documented design patterns, even basic training can improve both the speed and quality of maintenance activities.

• Jørgensen, Magne & Sjøberg, Dag (2004). Generalization and Theory-Building in Software Engineering Research. I Linkman, Stephen (Red.), Empirical Assessment in Software Engineering (EASE2004). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9780769521633. s. 29–36. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2003). An Effort Prediction Interval Approach Based on the Empirical Distribution of Previous Estimation Accuracy. Information and Software Technology. ISSN 0950-5849. 45, s. 123–136. Fulltekst i vitenarkiv Vis sammendrag

  When estimating software development effort, it may be useful to describe the uncertainty of the estimate through an effort prediction interval (PI). An effort PI consists of a minimum and a maximum effort value and a confidence level. We introduce and evaluate a software development effort PI approach that is based on the assumption that the estimation accuracy of earlier software projects predicts the effort PIs of new projects. First, we demonstrate the applicability and different variants of the approach on a data set of 145 software development tasks. Then, we experimentally compare the performance of one variant of the approach with human (software professionals') judgment and regression analysis-based effort PIs on a data set of 15 development tasks. Finally, based on the experiment and analytical considerations, we discuss when to base effort PIs on human judgment, regression analysis, or our approach.

• Sjøberg, Dag; Anda, Bente Cecilie D; Arisholm, Erik; Dybå, Tore; Jørgensen, Magne & Karahasanovic, Amela [Vis alle 7 forfattere av denne artikkelen] (2003). Challenges and Recommendations When Increasing the Realism of Controlled Software Engineering Experiments. I Conradi, Reidar & Wang, Alf Inge (Red.), Empirical Methods and Studies in Software Engineering: Experiences from ESERNET. Springer. ISSN 9783540406723. s. 24–38. Fulltekst i vitenarkiv
• Jørgensen, Magne; Sjøberg, Dag & Indahl, Ulf Geir (2003). Software Effort Estimation by Analogy and Regression Toward the Mean. Journal of Systems and Software. ISSN 0164-1212. 68(3), s. 252–262. Fulltekst i vitenarkiv
• Syversen, Erik; Anda, Bente Cecilie D & Sjøberg, Dag (2003). An Evaluation of Applying Use Cases to Construct Design versus Validate Design. I Dennis, Eileen (Red.), Hawaii International Conference on System Sciences (HICSS-36). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9780769518749. s. 327–340. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2003). Journal of Information and Software Technology. Information and Software Technology. ISSN 0950-5849. 45(3), s. 123–136. Fulltekst i vitenarkiv
• Anda, Bente Cecilie D & Sjøberg, Dag (2003). Applying use cases to design versus validate class diagrams - a controlled experiment using a professional modeling tool. I Cantone, C. (Red.), ISESE 2003 - International Symposium on Empirical Software Engineering. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9780769520025. s. 50–60. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2002). Impact of Experience on Maintenance Skills. Journal of Software Maintenance and Evolution: Research and Practice. 14, s. 123–146. Fulltekst i vitenarkiv Vis sammendrag

  This study reports results from an empirical study of 54 software maintainers in the software maintenance department of a Norwegian company. The study addresses the relationship between amount of experience and maintenance skills. The findings were, amongst others: (1) While there may have been a reduction in the frequency of major unexpected problems from tasks solved by very inexperienced to medium experienced maintainers, additional years of general software maintenance experience did not lead to further reduction. More application specific experience, however, further reduced the frequency of major unexpected problems. (2) The most experienced maintainers did not predict maintenance problems better than maintainers with little or medium experience. (3) A simple one-variable model outperformed the maintainers' predictions of maintenance problems, i.e., the average prediction performance of the maintainers seems poor. An important reason for the weak correlation between length of experience and ability to predict maintenance problems may be the lack of meaningful feedback on the predictions.

• Arisholm, Erik; Sjøberg, Dag; Carelius, Gunnar J. & Lindsjørn, Yngve (2002). A Web-based Support Environment for Software Engineering Experiments. Nordic Journal of Computing. ISSN 1236-6064. s. 231–247. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2002). Impact of Experience on Maintance Skills. Software Maintenance: Research and Practice. s. 123–146. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2001). Impact of Effort Estimates on Software Project Work. Information and Software Technology. ISSN 0950-5849. 43, s. 939–948. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2001). Software Process Improvement and Human Judgement Heuristics. Scandinavian Journal of Information Systems (SJIS). ISSN 0905-0167. 13, s. 99–122. Fulltekst i vitenarkiv
• Arisholm, Erik; Sjøberg, Dag & Jørgensen, Magne (2001). Assessing the Changeability of two Object-Oriented Design Alternatives - a Controlled Experiment. Empirical Software Engineering. ISSN 1382-3256. 6(3), s. 231–277. Fulltekst i vitenarkiv
• Sjøberg, Dag; Welland, R.; Atkinson, M. P.; Philbrow, P. & Waite, C. (2000). The Persistent Workshop -a Programming Environment for Napier88. I Atkinson, M. P. & Welland, R. (Red.), Fully Integrated Data Environment. Springer. s. 523–527. Fulltekst i vitenarkiv
• Holgeid, K.K.; Krogstie, J. & Sjøberg, Dag (2000). A Study of Development and Maintenance in Norway: Assessing the Efficiency of Information Systems Support Using Functional Maintenance. Information and Software Technology. ISSN 0950-5849. 42(10), s. 687–700. Fulltekst i vitenarkiv
• Opdahl, Andreas L. & Sjøberg, Dag (2000). Programming and Software Development Environment Research in the Nordic Countries, Special Issue on Programming Environments. Nordic Journal of Computing. ISSN 1236-6064. 7(2), s. 65–66. Fulltekst i vitenarkiv
• Arisholm, Erik & Sjøberg, Dag (2000). Towards a Framework for Empirical Assessment of Changeability Decay. Journal of Systems and Software. ISSN 0164-1212. 53, s. 3–14. Fulltekst i vitenarkiv
• Bennet, K.; Burd, E.; Kemerer, C.; Lehman, M. M.; Lee, Maan Gee & Madachy, R. [Vis alle 9 forfattere av denne artikkelen] (1999). Empirical Studies of Evolving Systems. Empirical Software Engineering. ISSN 1382-3256. s. 370–380. Fulltekst i vitenarkiv
• Jørgensen, Magne; Sjøberg, Dag & Conradi, Reidar (1999). Reuse of software development experience - a case study. Telektronikk. ISSN 0085-7130. 95(1), s. 48–54. Fulltekst i vitenarkiv
• Sjøberg, Dag; Welland, R.; Atkinson, M. P.; Philbrow, P.; Waite, C. & Macneill, S. (1999). The Persistent Workshop - a Programming Environment for Napier88. I Atkinson, M. P. & Welland, R. (Red.), Fully Integrated Data Environment. Springer. s. 523–527. Fulltekst i vitenarkiv
• Sjøberg, Dag; Welland, Atkinson & Philbrow, Wait (1997). The Persistent Workshop - a Programming Environment for Napier88. Nordic Journal of Computing. ISSN 1236-6064. 4, s. 123–149. Fulltekst i vitenarkiv
• Sjøberg, Dag (1997). Software Constraints for Large Application Systems. Computer journal. ISSN 0010-4620. 40(10), s. 598–616. Fulltekst i vitenarkiv
• Sjøberg, Dag (1997). Exploiting Persistence in Build Management. Software, Practice & Experience. ISSN 0038-0644. 27(4), s. 447–480. Fulltekst i vitenarkiv
• Sjøberg, Dag & Christophersen, Nils Damm (1996). Den menneskefiendtlige informasjonsteknologien ­ og andre myter, Schola. Fulltekst i vitenarkiv
• Sjøberg, Dag (1994). Managing Change in Information Systems : technological challenges. I Juustila, Anttie; Järvinen, Janne & Kerola, Pentti (Red.), Proceedings of the 17th Information Systems Research Seminar in Scandinavia (IRIS 17), Syöte Conference Centre, Finland, August 6.-9. 1994. University of Oulu. Fulltekst i vitenarkiv Vis sammendrag

  Information systems and other computer-based systems must continuously undergo change in order to reflect change in their environments. The present technology used to implement such systems, including models, methods, tools and languages, does not have an inherent understanding of the nature of evolution. The rigidity of existing systems is a hindrance for user requested enhancements.Propagating changes correctly is a particular problem. It is common to find that necessary changes consequent on some other change have not been made, so that the system is inconsistent and will eventually fail to operate correctly. The paper discusses tools for system maintenance and focuses on the issue of automation. A tool that automatically generates and maintains all the information it needs is presented. To provide more information about the form and extent of the evolution in real-world systems, the same tool was instructed to collect change measurements. Information about the evolution of a large health management system was recorded over a period of 18 months. Methods for and problems of automatic change measurements collection are discussed.

• Sjøberg, Dag; Cutts, Quintin; Welland, Ray & Atkinson, Malcolm P. (1994). Analysing Persistent Language Applications. I Atkinson, Malcolm; Maier, David & Benzaken, Véronique (Red.), Persistent Object Systems. Proceedings of the Sixth International Workshop on Persistent Object Systems, Tarascon, Provence, France, 5–9 September 1994. Springer. ISSN 9781447121220. doi: 10.1007/978-1-4471-2122-0_20. Fulltekst i vitenarkiv Vis sammendrag

  Most research into persistent programming has been directed towards the design and implementation of languages and object stores. There are few reports on the characteristics of systems exploiting such technology. This paper reports on a study of the source code of 20 applications consisting of more than 108,000 lines of persistent language code. The authors of the applications range from students to experienced programmers. The programs have been categorised and examined with respect to a persistent application model and the extent of inconsistencies relative to this model is presented. The results confirm the need for and give input to the design of programming methodologies and tools for persistent software engineering. Measurements also include the use of names, types, (polymorphic) procedures and persistent bindings. It is hoped that analysis of the measurements will be used as input to the next generation of languages and programming environments. As part of this new generation, a measurements system is outlined operating entirely within the persistent environment, thus simplifying access to and measurement of both static and dynamic information. The definitive version is available at springerlink.com

• Sjøberg, Dag; Atkinson, Malcolm; Lopes, J. & Trinder, P.W. (1994). Building an Integrated Persistent Application. I Beeri, C. (Red.), Database Programming Languages, 6th International Workshop, DBPL-6. Springer. ISSN 9783540198536. s. 359–375. Fulltekst i vitenarkiv Vis sammendrag

  The major motivation for database programming language (DBPL) research is to facilitate the construction and maintenance of large dataintensive applications. To fully bene t from DBPLs, supporting methodologies and tools are needed. This paper reports requirements for such methodologies and tools that were experienced when constructing a multiauthored, multi-levelled Thesaurus Application (TA) in a higher order DBPL. Although built in a speci c language (Napier88), the major principles discovered apply to other DBPLs. The TA comprises several looselyintegrated components constructed by di erent programmers. The components were themselves implemented using general purpose sub-components, including libraries. We experienced that a realistic application could be constructed quickly in a DBPL. Rapid construction was facilitated by the use of libraries, code reuse and an incremental construction methodology supported by the persistent store. Language features such as a polymorphic type system and structural type equivalence were important. Nevertheless, some problems were encountered with code reuse, with integrating independently constructed components, with the lack of concurrency and with build management such as installation and recompilation. Our experiences lead us to suggest several improvements, including models, methodologies and supporting tools for persistent application construction and maintenance.

• Sjøberg, Dag (1993). Quantifying schema evolution;. Information and Software Technology. ISSN 0950-5849. doi: 10.1016/0950-5849(93)90027-Z. Fulltekst i vitenarkiv Vis sammendrag

  Achieving correct changes is the dominant activity in the application software industry. Modification of database schemata is one kind of change which may have severe consequences for database applications. The paper presents a method for measuring modifications to database schemata and their consequences, by using a thesaurus tool. Measurements of the evolution of a large-scale database application currently running in several hospitals in the UK are presented and interpreted. The kind of measurements provided by this in-depth study is useful input to the design of change management tools.

• Atkinson, Malcolm; Sjøberg, Dag & Morrison, Ron (1993). Managing Change in Persistent Object Systems. I Nishio, S. & Yonezawa, A. (Red.), First JSSST International Symposium on Object Technologies for Advanced Software. Springer. ISSN 9783540573425. s. 315–338. Fulltekst i vitenarkiv Vis sammendrag

  Persistent object systems are highly-valued technology because they o er an e ec- tive foundation for building very long-lived persistent application systems (PAS). The technology becomes more e ective as it o ers a more consistently integrated computational context. For it to be feasible to design and construct a PAS it must be possible to in- crementally add program and data to the existing collection. For a PAS to endure it must o er exibility: a capacity to evolve and change. This paper examines the capacity of persistent object systems to accommodate incremental construction and change. Established store based technologies can support incremental construction but methodologies are needed to deploy them e ectively. Evolving data description is one motivation for inheritance but inheritance alone is not enough to support change management. The case for supporting incremental change is very persuasive. The challenge is to provide technologies that will facilitate it and methodologies that will organise it. This paper identi es change absorbers as a means of describing how changes should propagate. It is argued that if we systematically develop an adequate reper- toire of change absorbers then they will facilitate much better quality change man- agement.

• Sjøberg, Dag (1993). Quantifying Schema Evolution. Information and Software Technology. ISSN 0950-5849. s. 35–44. Fulltekst i vitenarkiv
• Bendvold, E.; Kravdal, Ø.; Moe, Narve; Sjøberg, Dag & Skjæraasen, Julie S. (1989). Marital Break-up among Couples Raising Families by Artificial Insemination by Donor. Fertility and Sterility. ISSN 0015-0282. 51(6), s. 980–983. Fulltekst i vitenarkiv
• Lindsjørn, Yngve & Sjøberg, Dag (1988). Database consepts discussed in an object-oriented perspective. Ukjent. Fulltekst i vitenarkiv
• Lindegaard, K.F.; Lundar, Tryggve; Wiberg, Jan; Sjøberg, Dag; Aaslid, R. & Nornes, Helge (1987). Variations in Middle Cerebral Artery Blood Flow Investigated With Noninvasive Transcranial Blood Velocity Measurements. Stroke. ISSN 0039-2499. 18(6), s. 1025–1030. Fulltekst i vitenarkiv

Se alle arbeider i NVA

• Shull, Forrest; Singer, Janice & Sjøberg, Dag (2008). Advanced Topics in Empirical Software Engineering. Springer. ISBN 9781848000438. 388 s. Fulltekst i vitenarkiv
• Conradi, Reidar; Dybå, Tore; Sjøberg, Dag I. K. & Ulsund, Tor (2006). Software Process Improvement: Results and Experience from the Field. Springer. ISBN 9783540321781. 413 s. Fulltekst i vitenarkiv Vis sammendrag

  This book reports on results and experience from three Norwegian, industrial R&D project on Software Process Improvement (SPI) - SPIQ, PROFIT and SPIKE - with emphasis on the latter. The book is organized in five sections, with totally 20 previously published papers. Section 1 is about SPI principles/general, and has 5 papers. Section 2: Knowledge Management for SPI and has 5 papers. Section 3 is about Process Modelling and Electronic Process Guides and has 4 papers. Section 4 is about Estimation Methods and has 3 papers. Section 5 is about Empirical Studies in OO and Component-based Systems and has 3 papers. The audience is software professionals as well as researchers and graduate students.

• Kirby, G.N.C.; Dearle, A. & Sjøberg, Dag (2001). Persistent Object Systems. Springer. Fulltekst i vitenarkiv

Se alle arbeider i NVA

• Sjøberg, Dag (2023). Studie antyder at IT-fag ikke bør undervises på engelsk. Det blir ikke lett, mener professor. [Journal]. Digi.no. Fulltekst i vitenarkiv
• Sjøberg, Dag (2023). AI Engineer. [Journal]. kode24. Fulltekst i vitenarkiv
• Bergersen, Gunnar R. & Sjøberg, Dag Ingar Kondrup (2023). Education of software developers in a fast-growing job market: Giving more students with "non-standard" backgrounds an opportunity. Fulltekst i vitenarkiv Vis sammendrag

  Software development is a fast-growing occupation worldwide and has grown by 50% in Norway during the last five years. Unlike many other professional occupations, system developers are seldom required to have a formal education to work professionally. For example, anyone with the right skills can build open-source software or offer freelance services worldwide on a project-by-project basis. Currently, educational institutions cannot educate enough developers to fill new jobs. However, there is also a discrepancy between what type of students that academic educations will admit and, potentially, could admit. In this context, a student with a "standard" background would enter a programme after upper secondary education with all the study requirements in place and a goal to complete a BSc or MSc. In contrast, a student with a "non-standard" background would typically be older, have prior work or education experience, and have different interests and motivations to study a few selected topics. However, this may not be possible as many would-be students lack advanced math, a formal requirement for attending many courses. We report results on an ongoing longitudinal study of software development students (N=854) in our department over four years. In addition to analyzing prior grades, math specialization, and programming-related electives from upper secondary school, the students provided self-reports on their prior experience, interests, motivations, and aspirations for studying software development. We also contrast these variables with grades and study credit production and show that a substantial group of students with non-standard backgrounds appear to do equally well or better than the students with standard backgrounds. We conclude that even though commonly established requirements for studying software development should be retained for students with standard backgrounds, one should revise how students with non-standard backgrounds are admitted. As calls for greater diversity in society linger, we highlight the importance of greater flexibility in university requirements for studying software development so that the software industry's increasing needs are met to a greater extent than it is today.

• Sjøberg, Dag (2023). Tall opplyser. [Journal]. Klassekampen. Fulltekst i vitenarkiv
• Sjøberg, Dag (2022). Keynote: Construct Validity in Software Engineering. Fulltekst i vitenarkiv
• Sjøberg, Dag (2021). Utenlandske forskere: Jobbene nordmenn ikke vil ha. [Journal]. Aftenposten. Fulltekst i vitenarkiv
• Sjøberg, Dag (2018). Forskning er skriving. Forskerforum. ISSN 0800-1715. s. 43–43. Fulltekst i vitenarkiv
• Tobiassen, Markus; Bergersen, Gunnar R. & Sjøberg, Dag (2016). Ser butikk i it-flauser. [Journal]. Dagens Næringsliv. Fulltekst i vitenarkiv
• Sjøberg, Dag (2016). Programmering i skolen er hverken frelse eller rør. [Journal]. Aftenposten. Fulltekst i vitenarkiv
• Sjøberg, Dag (2015). Programmering bør være allmennkunnskap. [Journal]. Dagens Næringsliv. Fulltekst i vitenarkiv
• Sjøberg, Dag (2015). Vil ikke forske på it. [Journal]. Dagens Næringsliv. Fulltekst i vitenarkiv
• Sjøberg, Dag (2015). Bruk av IT-konsulenter i offentlig sektor. [Journal]. Bergens Tidende. Fulltekst i vitenarkiv
• Sjøberg, Dag (2014). Spekulativ sykkelforskning. Fulltekst i vitenarkiv
• Sjøberg, Dag (2013). Viktigste faktorer for å redusere teknisk gjeld. Fulltekst i vitenarkiv
• Sjøberg, Dag (2013). Kjemper innbitt mot stammespråket. [Journal]. Computerworld. Fulltekst i vitenarkiv
• Sjøberg, Dag (2013). Smidig må finne sin form. [Journal]. Computerworld. Fulltekst i vitenarkiv
• Sjøberg, Dag (2012). Organisering av offentlige IT-prosjekter. [Journal]. Aftenposten. Fulltekst i vitenarkiv
• Sjøberg, Dag (2012). Kompleksiteten vil bli enorm dersom man skal samle alt på ett sted. [Journal]. Aftenposten. Fulltekst i vitenarkiv
• Sjøberg, Dag (2012). Har brukt 425 millioner på IT-konsulenter. [Journal]. Aftenposten. Fulltekst i vitenarkiv
• Sjøberg, Dag (2012). Smidig forskning. [Journal]. Computerworld. Fulltekst i vitenarkiv
• Sjøberg, Dag (2012). Skandalene er kommet for å bli. [Journal]. Aftenposten. Fulltekst i vitenarkiv
• Sjøberg, Dag (2012). Smidig utvikling krever opplæring. [Journal]. Computerworld. Fulltekst i vitenarkiv
• Sjøberg, Dag (2011). Suksess med Kanban. [Journal]. Computerworld. Fulltekst i vitenarkiv
• Sjøberg, Dag (2011). En kvantitativ studie av Kanban versus Scrum. Fulltekst i vitenarkiv
• Sjøberg, Dag (2011). Empirical software research: an interview with Dag Sjøberg, University of Oslo, Norway. ACM Ubiquity. doi: 10.1145/1998372.1998374. Fulltekst i vitenarkiv
• Mockus, Audris; Anda, Bente Cecilie D & Sjøberg, Dag (2010). Experiences from Replicating a Case Study to Investigate Reproducibility of Software Development. Fulltekst i vitenarkiv
• Sjøberg, Dag (2010). IT-forskningen er treg. s. 32–33. Fulltekst i vitenarkiv
• Sjøberg, Dag; Dybå, Tore; Conradi, Reidar; Hoff, Per Morten & Gundersen, Christer (2010). Derfor trenger Norge mer IKT-forskning. s. 24–24. Fulltekst i vitenarkiv Vis sammendrag

  Dette er første motkronikk av 3. des. 2010 til startkronikk i Computerworld 19. nov. 2010: "http://www.idi.ntnu.no/grupper/su/publ/reidar/cw.19.11.startkronikk.pdf". Se også Conradi's andre motkronikk i Computerworld, 17. des. 2010: http://www.idi.ntnu.no/grupper/su/publ/reidar/cw.17.12.motkronikk2-rc.pdf.

• Sjøberg, Dag (2010). Universitetet i Oslo satser på Smidig. [Journal]. ComputerWorld. Fulltekst i vitenarkiv
• Benestad, Hans Christian; Arisholm, Erik & Sjøberg, Dag (2005). How to Recruit Professionals as Subjects in Software Engineering Experiments. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2004). Generalization and Theory-Building in Software Engineering Research. Fulltekst i vitenarkiv
• Karahasanovic, Amela; Fjuk, Annita; Sjøberg, Dag & Thomas, Richard (2004). Controlled Experiment to Evaluate the Reactivity and Usefulness of the Think-Aloud Tool, Information Resources Management Association International Conference IRMA'04, Human Computer Interaction Track, New Orleans, Louisiana, USA, Idea Group Publishing, 1033-1034 / 2004. Fulltekst i vitenarkiv
• Karahasanovic, Amela; Fjuk, A; Sjøberg, Dag & Thomas, R. (2004). A Controlled Experiment to Evaluate the Reactivity and Usefulness of the Think-Aloud Tool. Fulltekst i vitenarkiv
• Karahasanovic, Amela; Anda, Bente; Arisholm, Erik; Hove, Siw; Jørgensen, Magne & Sjøberg, Dag (2003). A Think-Aloud Support Tool for Collecting Feedback in Large-Scale Software Engineering Experiments. Fulltekst i vitenarkiv Vis sammendrag

  The think-aloud method has proven useful for studying how a technology is used. However, it is labour-intensive and makes demands on resources for studies with many subjects. This paper describes a tool, called the think-aloud tool, that we have developed to support the collection of feedback from subjects in large-scale software engineering experiments. This tool instructs the subjects, at regular intervals, to write down their thoughts on a web-based screen and collects their feedback. An empirical evaluation of this tool is reported. The results from four controlled experiments suggest that the think-aloud tool is a valuable means for increasing the quality of large-scale software engineering experiments. The results also indicate that the participants were mostly positive to the tool and had no significant problems in verbalising their thoughts. The tool we proposed adds to an available set of tools for data collection in software engineering experiments. Whereas we have experience with the think-aloud tool in software engineering experiments, there is also potential for using such a tool in other areas in which the think-aloud method is used in studies involving a computer system.

• Karahasanovic, Amela & Sjøberg, Dag (2002). Visualising Impacts of Change in Evolving Object-Oriented Systems: An Explorative Study. Fulltekst i vitenarkiv
• Sjøberg, Dag; Anda, Bente Cecilie D; Arisholm, Erik; Dybå, Tore; Jørgensen, Magne & Karahasanovic, Amela [Vis alle 8 forfattere av denne artikkelen] (2002). Conducting Realistic Experiments in Software Engineering. Fulltekst i vitenarkiv
• Anda, Bente Cecilie D & Sjøberg, Dag (2002). Towards an Inspection Technique for Use Case Models. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2002). A simple effort prediction interval approach. Fulltekst i vitenarkiv
• Arisholm, Erik; Sjøberg, Dag; Carelius, Gunnar J. & Lindsjørn, Yngve (2002). SESE - an Experiment Support Environment for Evaluating Software Engineering Technologies. Fulltekst i vitenarkiv
• Jørgensen, Magne; Indahl, Ulf & Sjøberg, Dag (2001). Software Effort Estimation by Analogy and Regression Toward the Mean. Fulltekst i vitenarkiv Vis sammendrag

  Estimation by analogy is, simplified, the process of finding one or more projects that are similar to the one to be estimated and then derive the estimate from the values of these projects. If the selected projects have an unusual high or low productivity, then we should adjust the estimates toward productivity values of more average projects. The size of the adjustments depends on the expected accuracy of the estimation model. This paper evaluates one adjustment approach, based on the findings made by Sir Francis Galton in the late 1800s regarding the statistical phenomenon “regression toward the mean” (RTM). We evaluate this approach on several data sets and find indications that it improves the estimation accuracy. Surprisingly, current analogy based effort estimation models do not, as far as we know, include adjustments related to extreme analogues and inaccurate estimation models. An analysis of several industrial software development and maintenance projects indicates that the effort estimates provided by software professionals, i.e., expert estimates, to some extent are RTM-adjusted. A student experiment confirms this finding, but also indicates a rather large variance in how well the need for RTM-adjustments is understood among software developers.

• Anda, Bente; Sjøberg, Dag; Jørgensen, Magne & Knudsen, Lindskov (2001). Quality and Understandability of Use Case Models. Fulltekst i vitenarkiv Vis sammendrag

  Use case models are used in object-oriented analysis for capturing and describing the functional requirements of a system. Use case models are also used in communication between stakeholders in development projects. It is therefore important that the use case models are constructed in such a way that they support the development process and promote a good understanding of the requirements among the stakeholders. Despite this, there are few guidelines on how to construct use case models. This paper describes an explorative study where three different sets of guidelines were used for constructing and documenting use case models. An experiment with 139 undergraduate students divided into 31 groups was conducted. Each group used one out of the three sets of guidelines when constructing a use case model from an informal requirements specification. After completing the use case model, each student answered a questionnaire. The results of the experiment indicate that guidelines based on templates support the construction of use case models that are easier to understand for the readers, than guidelines without specific details on how to document each use case. The guidelines based on templates were also considered as the most useful when constructing use cases. In addition to better understandability, our experiment indicates that the guidelines based on templates result in better use case models regarding also other quality attributes. Our results further indicate that it may be beneficial to combine the template guidelines with another set of guidelines that focus on the documentation of the flow of events of each use case. Keywords. Object-oriented analysis, Requirements specification, Use Cases, UML, Understandability, Experiment

• Jørgensen, Magne; Indahl, Ulf & Sjøberg, Dag (2001). Software Effort Estimation by Analogy and Regression Toward the Mean. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2001). Anchoring Effects: An Important Cause of Too Optimistic Effort Estimates in Software Development Projects. Fulltekst i vitenarkiv
• Sjøberg, Dag; Arisholm, Erik & Jørgensen, Magne (2001). Conducting Experiments on Software Evolution. Fulltekst i vitenarkiv
• Anda, Bente Cecilie D; Sjøberg, Dag & Jørgensen, Magne (2001). Quality and Understandability in Use Case Models. Fulltekst i vitenarkiv
• Anda, Bente Cecilie D; Dreiem, D.; Sjøberg, Dag & Jørgensen, Magne (2001). Estimating Software Development Effort Based on Use Cases - Experiences from Industry. Fulltekst i vitenarkiv
• Karahasanovic, Amela; Sjøberg, Dag & Jørgensen, Magne (2001). Data Collection in Software Engineering Experiments. Fulltekst i vitenarkiv
• Karahasanovic, Amela & Sjøberg, Dag (2001). Visualizing Impacts of Database Schema Changes -- A Controlled Experiment. Fulltekst i vitenarkiv Vis sammendrag

  Research in schema evolution has been driven by the need for more effective software development and maintenance. Finding impacts of schema changes on the applications and presenting them in an appropriate way are particularly challenging. We have developed a tool that finds impacts of schema changes on applications in object-oriented systems. This tool displays components (packages, classes, interfaces, methods and fields) of a database application system as a graph. Components potentially affected by a change are indicated by changing the shape of the boxes representing those components. Two versions of the tool are available. One version identifies affected parts of applications at the granularity of packages, classes, and interfaces, whereas the other version identifies affected parts at the finer granularity of fields and methods. This paper presents the design and results of a controlled student experiment testing these two granularity levels with respect to productivity and user satisfaction. There are indications that identifying impacts at the finer granularity can reduce the time needed to conduct schema changes and reduce the number of errors. Our results also show that the subjects of the experiment appreciated the idea of visualizing the impacts of schema changes.

• Dybå, Tore; Wedde, K.; Stålhane, T.; Moe, N.; Conradi, R. & Dingsøyr, T. [Vis alle 8 forfattere av denne artikkelen] (2000). SPIQ Metodehåndbok. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2000). Empirical studies on effort estimation in software development projects. Fulltekst i vitenarkiv
• Jørgensen, Magne; Sjøberg, Dag & Kirkebøen, Geir (2000). The prediction ability of experienced software maintainers. Fulltekst i vitenarkiv
• Sjøberg, Dag (2000). Læring av erfaring - Estimering. Fulltekst i vitenarkiv
• Jørgensen, Magne & Sjøberg, Dag (2000). The importance of NOT Learning from Experience. Fulltekst i vitenarkiv
• Jørgensen, Magne; Kirkebøen, Geir; Sjøberg, Dag; Anda, Bente Cecilie D & Bratthall, Lars (2000). Human judgement in effort estimation of software projects. Fulltekst i vitenarkiv
• Holgeir, K. K.; Krogstie, J. & Sjøberg, Dag (1999). A Longitudinal Study of Software Maintenance in Norway. Fulltekst i vitenarkiv
• Arneberg, G.; Sagli, K. & Sjøberg, Dag (1999). Configuration Management in Component-Based Development Projects. Fulltekst i vitenarkiv
• Sjøberg, Dag (1999). Software Constraints in Integrated Data Environments. Fulltekst i vitenarkiv
• Jørgensen, Magne; Sjøberg, Dag & Conradi, Reidar (1999). Reuse of software development experience at Telenor Telecom Software. Fulltekst i vitenarkiv
• Grimstad, S.; Sjøberg, Dag; Atkinson, M. P. & Welland, R. (1999). Evaluating Usability Aspects of PJama based on Source Code Measurements. Fulltekst i vitenarkiv
• Arisholm, Erik; Skandsen, Jon; Sagli, K. & Sjøberg, Dag (1999). Improving an Evolutionary Development Process - A Case Study, Fulltekst i vitenarkiv
• Arisholm, Erik & Sjøberg, Dag (1999). Empirical Assessment of Changeability Decay in Object-Oriented Software. Fulltekst i vitenarkiv
• Arisholm, Erik; Anda, Bente Cecilie D; Jørgensen, Magne & Sjøberg, Dag (1999). Guidelines on Conducting Software Process Improvement Studies in Industry. Fulltekst i vitenarkiv
• Karahasanovic, Amela & Sjøberg, Dag (1999). Supporting Database Schema Evolution by Impact Analysis. Fulltekst i vitenarkiv
• Jørgensen, Magne; Sjøberg, Dag & Conradi, R. (1998). Reuse of Software Development Experience at Telenor Telecom Software. Fulltekst i vitenarkiv
• Sjøberg, Dag & Arnstad, P.E. (1997). Driftsrevolusjonen. ComputerWorld Norge. ISSN 1501-6595. Fulltekst i vitenarkiv
• Sjøberg, Dag & Arnstad, P.E. (1997). Komponenter - basis for utvikling etter ferdighusprinsippter. ComputerWorld Norge. ISSN 1501-6595. Fulltekst i vitenarkiv
• Sjøberg, Dag (1997). Empirical Analysis based on Automatic Tool Logging. Fulltekst i vitenarkiv
• Sjøberg, Dag (1996). The Persistent Workshop - a Programming Environment for Napier88. Fulltekst i vitenarkiv
• Sjøberg, Dag; Jørgensen, Magne & Maus, Arne (1996). Evaluating Software Maintenance Technology. Fulltekst i vitenarkiv
• Lysne, Olav & Sjøberg, Dag (1995). Internett - Ikke Livsnødvendig. Fulltekst i vitenarkiv
• Sjøberg, Dag (1995). Towards a Methodology for Evaluating Software Maintenance Technology. Fulltekst i vitenarkiv
• Sjøberg, Dag (1995). Analysing Persistent Language Applications. Fulltekst i vitenarkiv
• Lysne, Olav & Sjøberg, Dag (1995). Internett -- ikke livsnødvendig. Fulltekst i vitenarkiv
• Sjøberg, Dag (1994). Building an Integrated Persistent Application. Fulltekst i vitenarkiv
• Sjøberg, Dag (1994). Software Constraint Models ­ A Means to Improve Maintainability and Consistency. Fulltekst i vitenarkiv Vis sammendrag

  As application systems live longer and grow in size and complexity, there is an ever increasing need for methodologies, models and tools that can aid software builders in developing maintainable, correct and consistent systems. Imposing constraints, representing architectures, conventions, guidelines, etc., on the software is one step in that direction. One may distinguish between constraints within programs, constraints between programs, and constraints between programs and secondary storage. A coherent set of constraints are collected in a software constraint model. Automatic verification tools are crucial to the usefulness of such models. The paper describes a constraint model and a corresponding verification tool that have been developed in a persistent programming environment.

• Sjøberg, Dag (1994). Managing Change in Information Systems: Technological Challenges. Fulltekst i vitenarkiv
• Sjøberg, Dag (1994). Thesaurus-Based Software Environments. Fulltekst i vitenarkiv
• Lindsjørn, Yngve & Sjøberg, Dag (1988). Database Concepts Described in an Object-Oriented Perspective. Fulltekst i vitenarkiv
• LaToza, Thomas D; Ko, Amy J.; Shepherd, David C; Sjøberg, Dag & Xie, Benjamin (2023). Theories of Programming (Dagstuhl Seminar 22231). Schloss Dagstuhl - Leibniz-Zentrum fur Informatik. Fulltekst i vitenarkiv Vis sammendrag

  Much of computer science research focuses on techniques to make programming easier, better, less error prone, more powerful, and even more just. But rarely do we try to explain any of these challenges. Why is programming hard? Why is it slow? Why is it error prone? Why is it powerful? How does it do harm? These why and how questions are what motivated the Dagstuhl Seminar 22231 on Theories of Programming. This seminar brought together 28 CS researchers from domains most concerned with programming human and social activities: software engineering, programming languages, human-computer interaction, and computing education. Together, we sketched new theories of programming and considered the role of theories more broadly in programming.

• Bergersen, Gunnar R.; Sjøberg, Dag & Dybå, Tore (2015). Measuring programming skill. Akademika forlag. Fulltekst i vitenarkiv
• Dearle, A.; Kirby, G.N.C. & Sjøberg, Dag (2000). Preprints of Ninth International Workshop on Persistent Object Systems. Fulltekst i vitenarkiv
• Mughal, K. A.; Opdahl, Andreas L. & Sjøberg, Dag (2000). Proceedings of NWPER'2000 - The Ninth Nordic Workshop on Programming and Software Development Environment Research. ISSN 9788273682314. Fulltekst i vitenarkiv
• Dybå, Tore; Wedde, Kari Juul; Moe, Nils Brede; Conradi, Reidar; Dingsøyr, Torgeir & Sjøberg, Dag [Vis alle 7 forfattere av denne artikkelen] (2000). SPIQ - Metodehåndbok. Ifi, Uio. ISSN 9788273682307. Fulltekst i vitenarkiv
• Sjøberg, Dag (1993). Thesaurus-Based Methodologies and Tools for Maintaining Persistent Application Systems. University of Glasgow. Fulltekst i vitenarkiv

Se alle arbeider i NVA