Department of IT Management
The department conducts research on software development methods, human judgment and decision making in software development contexts, benefits management, cost estimation, risk and uncertainty management, use of contracts and models for user involvements. The main research approach is empirical, including controlled experiments, observational studies, case studies and surveys.
The department’s main goal is to discover important relationships and connections related to the management of software development work, and use this to contribute to substantial improvement in software development processes and products. For this purpose, we work in close collaboration with industry partners in most of our research. We emphasize the transfer of new and useful knowledge to the industry by publishing and presenting not only in academic venues, but also at industry venues.
The IT Management Department also runs Hovedstadsområdets nettverk for IT-styring og ledelse (HIT-Nettverket), which is a network with the goal of increasing competance through sharing experiences in IT management. The network consist of research institutes, IT companies, consultant companies, public stakeholders and clients with an interest in IT management. The themes includes focuses both in the client and the supplier perspective, in addition to strategic, administrative and project related challenges. Read more about HIT here.
In addition, the department is in charge of the EDOS center (Effective Organization of Public Sector). This center is funded by the Ministry of Local Government and Modernization and was created in 2020. EDOS conducts surveys and analyzes information on digitalization in the public sector. The research aims to provide knowledge about what leads to successful digitalization in the public sector and to disseminate knowledge that leads to higher efficiency and more value creation. Read more about EDOS here.
People at Department of IT Management
Who we are?
Simula Metropolitan employees are researchers, postdoctoral fellows, PhD students, engineers and administrative people. We are from all over the world, ranging from newly educated to experienced researchers, all working on making research in digital engineering at the highest international level possible.
Publications at Department of IT Management
Journal Article
When should we (not) use the mean magnitude of relative error (MMRE) as an error measure in software development effort estimation?
Information and Software Technology 143 (2022).Status: Accepted
When should we (not) use the mean magnitude of relative error (MMRE) as an error measure in software development effort estimation?
Context: The mean magnitude of relative error (MMRE) is an error measure frequently used to evaluate and compare the estimation performance of prediction models and software professionals.
Objective: This paper examines conditions for proper use of MMRE in effort estimation contexts.
Method: We apply research on scoring functions to identify the type of estimates that minimizes the expected value of the MMRE.
Results: We show that the MMRE is a proper error measure for estimates of the most likely (mode) effort, but not for estimates of the median or mean effort, provided that the effort usage is approximately log-normally distributed, which we argue is a reasonable assumption in many software development contexts. The relevance of the findings is demonstrated on real-world software development data.
Conclusion: MMRE is not a proper measure of the accuracy of estimates of the median or mean effort, but may be used for the accuracy evaluation of estimates of most likely effort.
| Afilliation | Software Engineering |
| Project(s) | Department of IT Management, EDOS: Effective Digitalization of Public Sector |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | Information and Software Technology |
| Volume | 143 |
| Date Published | 03/2022 |
| Publisher | Elsevier |
When 2 + 2 should be 5: The summation fallacy in time prediction
Journal of Behavioral Decision Making (2022).Status: Accepted
When 2 + 2 should be 5: The summation fallacy in time prediction
Predictions of time (e.g., work hours) are often based on the aggregation of estimates of elements (e.g., activities, subtasks). The only types of estimates that can be safely aggregated by summation are those reflecting predicted average outcomes (expected values). The sums of other types of estimates, such as bounds of confidence intervals or estimates of the mode, do not have the same interpretation as their components (e.g., the sum of the 90% upper bounds is not the appropriate 90% upper bound of the sum). This can be a potential source of bias in predictions of time, as shown in Studies 1 and 2, where professionals with experience in estimation provided total estimates of time that were inconsistent with their estimates of individual tasks. Study 3 shows that this inconsistency can be attributed to improper aggregation of time estimates and demonstrates how this can produce both over- and underestimation—and also time prediction intervals that are far too wide. Study 4 suggests that the results may reflect a more general fallacy in the aggregation of probabilistic quantities. Our observations are consistent with that inconsistencies and biases are driven by a tendency towards applying a naïve summation (2+2=4) of probabilistic (stochastic) values, in situations where this is not appropriate. This summation fallacy may be in particular consequential in a context where informal estimation methods (expert-judgment based estimation) are used.
| Afilliation | Software Engineering |
| Project(s) | Department of IT Management |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | Journal of Behavioral Decision Making |
| Publisher | Wiley |
Proceedings, refereed
Benefit Considerations in Project Decisions
In International Conference on Product-Focused Software Process Improvement. PROFES 2022. 23th ed. Springer, Cham, 2022.Status: Published
Benefit Considerations in Project Decisions
Software project success is often characterized in terms of time, cost and scope – despite that delivering benefit is the main purpose of a project. In this paper, we explore 1) to what degree benefit considerations influence major project decisions, 2) to what degree a specific set of benefits management challenge are handled and influence major project decisions and 3) if there is any realization (over time) that benefit considerations should receive greater attention. We investigate influence in projects with four types of problem severity: completed projects with only minor problems, completed projects with major problems, projects that were disrupted but completed, and projects that were terminated before completion. We asked 45 software professionals to what degree time, cost, scope, benefit and benefit/cost, as well as benefits management challenges, influence major roject decisions. Our findings indicate that time, cost and scope have a significantly higher degree of influence on project decisions than benefit and benefit/cost. However, practitioners think that benefit and benefit/cost should have significantly more influence on decisions than cost. The benefits management challenges are found to have less influence in the more severe projects. We argue that giving benefits considerations a stronger voice in project decisions would be in line with the desire of practitioners and the prime objective of delivering benefit to stakeholders. We conclude that it is important to understand how to handle benefits management challenges at different stages of project life and that handling such challenges should be integrated with other prime drivers of project success.
| Afilliation | Software Engineering |
| Project(s) | Department of IT Management, EDOS: Effective Digitalization of Public Sector |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | International Conference on Product-Focused Software Process Improvement. PROFES 2022 |
| Edition | 23 |
| Pagination | 217-234 |
| Date Published | 11/2022 |
| Publisher | Springer, Cham |
| ISBN Number | 978-3-031-21388-5 |
| Keywords | Benefit, Benefits management challenges, Cost, Scope, Software project severity, Time |
| DOI | 10.1007/978-3-031-21388-5_15 |
Perceived Challenges in Benefits Management - A Study of Public Sector Information Systems Engineering Projects
In Conference on Business Informatics (CBI). 24th ed. IEEE Computer Society Digital Library, 2022.Status: Published
Perceived Challenges in Benefits Management - A Study of Public Sector Information Systems Engineering Projects
The field of benefits management gives guidelines on how to plan and realize benefits throughout the life-cycle of a system. However, realizing benefits from information systems projects has proven to be challenging in practice. In this paper, we investigate specific benefits management challenges as perceived by practitioners involved in information systems engineering projects. We conducted 22 interviews with respondents representing nine public sector projects, where challenges in managing benefits were elicited and identified. We elicited six specific benefits management challenges: A - Identifying and describing benefits, B - Alignment of work with planned benefits, C - Reception and acceptance of the planned benefits, D - Organizational issues, E - Alternative or competing solutions, F - Measuring and evaluating benefits. Overlaying these challenges with current normative models on benefits management, we find that: 1. Normative models on benefits management lack sufficient guidance on operative work on how to create information systems fit for realizing benefits and how to introduce these solutions to ensure benefits realization, and 2. Normative models on benefits management do not explicitly leverage the rapid project learning promoted by modern engineering methods. We conclude that more specific benefits management models should be elaborated, which are integrated into modern information systems engineering practices. This will enable best practices on the continuous adjustments of cost and scope according to evolving knowledge in projects to also be adapted to the management of benefits.
| Afilliation | Software Engineering |
| Project(s) | Department of IT Management, EDOS: Effective Digitalization of Public Sector |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | Conference on Business Informatics (CBI) |
| Edition | 24 |
| Pagination | 156-165 |
| Date Published | 06/2022 |
| Publisher | IEEE Computer Society Digital Library |
| Keywords | Benefits management challenges, Benefits management models, Information systems engineering, Public sector |
| DOI | 10.1109/CBI54897.2022.00024 |
Research Incentives in Academia Leading to Unethical Behavior
In Research Challenges in Information Science. Vol. 44624156469341838. Cham: Springer International Publishing, 2022.Status: Published
Research Incentives in Academia Leading to Unethical Behavior
A current practice in academia is to reward researchers for achieving outstanding performance. Although intended to boost productivity, such a practice also promotes competitiveness and could lead to unethical behavior. This position paper exposes common misconducts that arise when researchers try to game the system. It calls the research community to take preventive actions to reduce misconduct and treat such a pervasive environment with proper acknowledgment of researchers’ efforts and rewards on quality rather than quantity.
| Afilliation | Software Engineering |
| Project(s) | Department of IT Management |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | Research Challenges in Information Science |
| Volume | 44624156469341838 |
| Pagination | 744 - 751 |
| Date Published | 05/2022 |
| Publisher | Springer International Publishing |
| Place Published | Cham |
| ISBN Number | 978-3-031-05759-5 |
| ISSN Number | 1865-1348 |
| Keywords | Incentives, Misconduct, Research ethics, Research quality, Researcher performance |
| Notes | Part of the Lecture Notes in Business Information Processing book series (LNBIP,volume 446) |
| URL | https://link.springer.com/chapter/10.1007/978-3-031-05760-1_51 |
| DOI | 10.1007/978-3-031-05760-1_51 |
Measurement of software development effort estimation bias: Avoiding biased measures of estimation bias
In 11th International Conference on Software Engineering and Applications (SEA 2022), 2022.Status: Accepted
Measurement of software development effort estimation bias: Avoiding biased measures of estimation bias
In this paper, we propose improvements in how estimation bias, e.g., the tendency towards under-estimating the effort, is measured. The proposed approach emphasizes the need to know what the estimates are meant to represent, i.e., the type of estimate we evaluate and the need for a match between the type of estimate given and the bias measure used. We show that even perfect estimates of the mean effort will not lead to an expectation of zero estimation bias when applying the frequently used bias measure: (actual effort – estimated effort)/actual effort. This measure will instead reward under-estimates of the mean effort. We also provide examples of bias measures that match estimates of the mean and the median effort, and argue that there are, in general, no practical bias measures for estimates of the most likely effort. The paper concludes with implications for the evaluation of bias of software development effort estimates.
| Afilliation | Software Engineering |
| Project(s) | Department of IT Management |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | 11th International Conference on Software Engineering and Applications (SEA 2022) |
Book
Benefit/Cost-Driven Software Development - with Benefit Points and Size Points
Cambridge, UK: Springer, 2021.Status: Published
Benefit/Cost-Driven Software Development - with Benefit Points and Size Points
This open access book presents a set of basic techniques for estimating the benefit of IT development projects and portfolios. It also offers methods for monitoring how much of that estimated benefit is being achieved during projects.
Readers can then use these benefit estimates together with cost estimates to create a benefit/cost index to help them decide which functionalities to send into construction and in what order. This allows them to focus on constructing the functionality that offers the best value for money at an early stage.
Although benefits management involves a wide range of activities in addition to estimation and monitoring, the techniques in this book provides a clear guide to achieving what has always been the goal of project and portfolio stakeholders: developing systems that produce as much usefulness and value as possible for the money invested. The techniques can also help deal with vicarious motives and obstacles that prevent this happening. The book equips readers to recognize when a project budget should not be spent in full and resources be allocated elsewhere in a portfolio instead. It also provides development managers and upper management with common ground as a basis for making informed decisions.
| Afilliation | Software Engineering |
| Project(s) | Department of IT Management |
| Publication Type | Book |
| Year of Publication | 2021 |
| Publisher | Springer |
| Place Published | Cambridge, UK |
| ISBN Number | 978-3-030-74217-1 |
| Keywords | benefit points, benefit/costs index, benefits management, earned business value management, periodization, uncertainty assessment |
| URL | https://link.springer.com/book/10.1007/978-3-030-74218-8#toc |
| DOI | 10.1007/978-3-030-74218-8 |
Journal Article
Benefits management in software development: A systematic review of empirical studies
IET Software 15, no. 1 (2021): 1-24.Status: Published
Benefits management in software development: A systematic review of empirical studies
Considerable resources are wasted on software projects delivering less than the planned benefits. Herein, the objective is to synthesize empirical evidence of the adoption and impact of benefits management (BM) in software development, and to suggest directions for future research. A systematic review of the literature is performed and identified 4836 scientific papers of which the authors found 47 to include relevant research. While most organizations identify and structure benefits at the outset of a project, fewer organizations report implementing BM as a continuous process throughout the project lifecycle. Empirical evidence gives support for positive impact on project outcome from the following BM practices: identifying and structuring benefits, planning benefits realization, BM during project execution, benefits evaluation and the practice of having people responsible for benefits realization. The authors suggest four research directions to understand (1) why BM practices sometimes not are adopted, (2) BM in relation to other management practices, (3) BM in agile software development and (4) BM in the context of organizations' value creation logics.
| Afilliation | Software Engineering |
| Project(s) | Department of IT Management |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | IET Software |
| Volume | 15 |
| Issue | 1 |
| Pagination | 1-24 |
| Publisher | IET |
Backsourcing of Information Technology - A Systematic Literature Review
Information Systems Research (2021).Status: Submitted
Backsourcing of Information Technology - A Systematic Literature Review
Context: Backsourcing is the process of insourcing previously outsourced activities. When companies experience environmental or strategic changes, or challenges with outsourcing, backsourcing can be a viable alternative. While outsourcing and related processes have been extensively studied in information technology, few studies report experiences with backsourcing.
Objectives: We intend to summarize the results of the research literature on the backsourcing of IT, with a focus on software development. By identifying practical relevance experience, we aim to present findings that may help companies considering backsourcing. In addition, we aim to identify gaps in the current research literature and point out areas for future work.
Method: Our systematic literature review (SLR) started with a search for empirical studies on the backsourcing of IT. From each study we identified the contexts in which backsourcing occurs, the factors leading to the decision to backsource, the backsourcing process itself, and the outcomes of backsourcing. We employed inductive coding to extract textual data from the papers identified and qualitative cross-case analysis to synthesize the evidence from backsourcing experiences.
Results: We identified 17 papers that reported 26 cases of backsourcing, six of which were related to software development. The cases came from a variety of contexts. The most common reasons for backsourcing were improving quality, reducing costs, and regaining control of outsourced activities. The backsourcing process can be described as containing five sub-processes: change management, vendor relationship management, competence building, organizational build-up, and transfer of ownership. Furthermore, we identified 14 positive outcomes and nine negative outcomes of backsourcing. Finally, we aggregated the evidence and detailed three relationships of potential use to companies considering backsourcing.
Conclusion: The backsourcing of IT is a complex process; its implementation depends on the prior outsourcing relationship and other contextual factors. Our systematic literature review may contribute to a better understanding of this process by identifying its components and their relationships based on the peer-reviewed literature. Our results may also serve as a motivation and baseline for further research on backsourcing and may provide guidelines and process fragments from which practitioners can benefit when they engage in backsourcing.
| Afilliation | Software Engineering |
| Project(s) | EDOS: Effective Digitalization of Public Sector, Department of IT Management |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | Information Systems Research |
| Publisher | Institute for Operations Research and the Management Sciences |
| Keywords | backshoring, Backsourcing, information technology, software development, Software Engineering, systematic literature review |
Leveraging Network-Centric Strategic Goals in Capabilities
Journal of Military Studies 10 (2021): 90-104.Status: Published
Leveraging Network-Centric Strategic Goals in Capabilities
The vision of network-centric operations is to increase operational capabilities through networked collaboration. NATO and its member nations state this vision in strategic documents at a very high level of abstraction. While suitable for giving an overall feel, current documentation renders the steps toward implementing those visions largely unsupported. We outline a method, based on agile requirements engineering, for converting high-level strategic visions into capabilities, whose form lend themselves to incremental implementation. We illustrate the use of this method in two cases that concern both operational capabilities and technical capabilities. We also show how the method enables one to prioritize which capabilities to develop first. We conclude that some form of explicit methodology to span the gap between strategic visions and implementing those visions is necessary.
| Afilliation | Software Engineering |
| Project(s) | EDOS: Effective Digitalization of Public Sector, Department of IT Management |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | Journal of Military Studies |
| Volume | 10 |
| Number | 1 |
| Pagination | 90-104 |
| Publisher | De Gruyter |
| Keywords | benefits management, capabilities, requirements engineering, strategic goals |
| URL | https://www.sciendo.com/article/10.2478/jms-2021-0001 |
| DOI | 10.2478/jms-2021-0001 |