A database for publications published by researchers and students at SimulaMet.
Research area
Publication type
- All (132)
- Journal articles (36) Remove Journal articles <span class="counter">(36)</span> filter
- Books (2)
- Edited books (1)
- Proceedings, refereed (30)
- Book chapters (5)
- Talks, keynote (13)
- Proceedings, non-refereed (1)
- Posters (1)
- Technical reports (2)
- Manuals (1)
- Talks, invited (25)
- Talks, contributed (1)
- Public outreach (11)
- Master's theses (1)
- Miscellaneous (2)
Journal articles
When should we (not) use the mean magnitude of relative error (MMRE) as an error measure in software development effort estimation?
Information and Software Technology 143 (2022): 106784.Status: Published
When should we (not) use the mean magnitude of relative error (MMRE) as an error measure in software development effort estimation?
Context: The mean magnitude of relative error (MMRE) is an error measure frequently used to evaluate and compare the estimation performance of prediction models and software professionals.
Objective: This paper examines conditions for proper use of MMRE in effort estimation contexts.
Method: We apply research on scoring functions to identify the type of estimates that minimizes the expected value of the MMRE.
Results: We show that the MMRE is a proper error measure for estimates of the most likely (mode) effort, but not for estimates of the median or mean effort, provided that the effort usage is approximately log-normally distributed, which we argue is a reasonable assumption in many software development contexts. The relevance of the findings is demonstrated on real-world software development data.
Conclusion: MMRE is not a proper measure of the accuracy of estimates of the median or mean effort, but may be used for the accuracy evaluation of estimates of most likely effort.
Afilliation | Software Engineering |
Project(s) | Department of IT Management, EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | Information and Software Technology |
Volume | 143 |
Pagination | 106784 |
Date Published | 03/2022 |
Publisher | Elsevier |
Relative estimates of software development effort: Are they more accurate or less time-consuming to produce than absolute estimates, and to what extent are they person-independent?
Information and Software Technology 143 (2022): 106782.Status: Published
Relative estimates of software development effort: Are they more accurate or less time-consuming to produce than absolute estimates, and to what extent are they person-independent?
Context: Estimates of software development effort may be given as judgments of relationships between the use of efforts on different tasks – that is, as relative estimates. The use of relative estimates has increased with the introduction of story points in agile software development contexts.
Objective: This study examines to what extent relative estimates are likely to be more accurate or less time-consuming to produce than absolute software development effort estimates and to what extent relative estimates can be considered developer-independent.
Method: We conducted two experiments. In the first experiment, we collected estimates from 102 professional software developers estimating the same tasks and randomly allocated to providing relative estimates in story points or absolute estimates in work-hours. In the second experiment, we collected the actual efforts of 20 professional software developers completing the same 5 programming tasks and used these to analyse the variance in relative efforts.
Results: The results from the first experiment indicates that the relative estimates were less accurate than the absolute estimates, and that the time consumed completing the estimation work was higher for those using relative estimation, even when only considering developers with extensive prior experience in story point–based estimation for both tasks. The second experiment revealed that the relative effort was far from developer-independent, especially for the least productive developers. This suggests that relative estimates to a large extent are developer-dependent.
Conclusions: Although there may be good reasons for the continued use of relative estimates, we interpret our results as not supporting that the use of relative estimates is connected with higher estimation accuracy or less time consumed on producing the estimates. Neither do our results support a high degree of developer-independence in relative estimates.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | Information and Software Technology |
Volume | 143 |
Pagination | 106782 |
Date Published | 03/2022 |
Publisher | Wiley |
Realizing benefits in public IT projects: A multiple case study
IET Software (2022).Status: Published
Realizing benefits in public IT projects: A multiple case study
IT investments in the public sector are large, and it is essential that they lead to benefits for the organizations themselves and for the wider society. While there is evidence suggesting a positive connection between the existence of benefits management practices and benefits realization, less is known about how to implement such practices effectively. The paper aims to provide insights into when benefits are most likely to be realized, and how benefits management practices and roles should be implemented in order to have a positive effect on the projects’ success in terms of realizing benefits. The authors collected data relating to ten Norwegian public IT projects. For each project, they collected data on benefits management from project documents, by interviewing the project owners and benefits owners, and follow-up surveys. The benefits internal to the organization were those with the highest degree of realization, while the societal benefits were those with the lowest degree. Projects assessed to have more specific, measurable, accountable, and realistically planned benefits were more successful in realizing benefits. Benefits owners were most effective when they were able to attract attention towards the benefits to be realized, had a strong mandate, and had domain expertise.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | IET Software |
Date Published | 12/2022 |
Publisher | IET |
Enabling Autonomous Teams and Continuous Deployment at Scale
IT Professional 24, no. 6 (2022): 47-53.Status: Published
Enabling Autonomous Teams and Continuous Deployment at Scale
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | IT Professional |
Volume | 24 |
Issue | 6 |
Pagination | 47 - 53 |
Date Published | Jan-11-2022 |
Publisher | IEEE Computer Society |
Place Published | New York City |
ISSN | 1520-9202 |
URL | https://ieeexplore.ieee.org/document/10017407/http://xplorestaging.ieee.... |
DOI | 10.1109/MITP.2022.3209871 |
State of Gender Equality in and by Artificial Intelligence
IADIS International Journal on Computer Science & Information Systems 17 (2022): 31-48.Status: Published
State of Gender Equality in and by Artificial Intelligence
When talking about sustainability, we usually think that it is only about safeguarding the environment; nothing is further from reality. Of course, the environment is a crucial component of sustainability and our survival, but it is important to recall that the society and the economy play important roles in this regard, and without the interconnection and development of these three perspectives it will not be possible to achieve sustainable progress. The Sustainable Development Goals (SDGs) established by the United Nations (UN) defend this idea and address the main challenges that humanity faces. One of these challenges is gender equality, which is identified in the perspective of social sustainability through SDG 5. Gender equality is a very complex and difficult challenge to address due to the great cultural diversity of our society. Thus, achieving this goal will require laying a solid foundation and working together by combining very different fields of knowledge. In this sense, Artificial Intelligence (AI) is one of the fields that is currently having the greatest impact and relevance for the development of new technologies and for the advancement of numerous areas. This growing evolution of AI demonstrates that its repercussions at the social level must be analyzed and addressed in such a way that AI becomes a positive asset for sustainability and, in this particular case, for gender equality.
For all these reasons, this study aims to analyze the current state of the art and collect the existing knowledge in the fields of AI and gender equality, by conducting a Systematic Mapping Study (SMS). The obtained results and findings have allowed us to identify the most relevant advances in this regard, as well as the gaps and drawbacks that currently exist and on which we must urgently focus to address gender equality both in and by AI. In the same way, these findings demonstrate the limited joint development of both fields, but also indicate an increase in the relevance and the number of proposals that these fields are receiving in recent years.
Afilliation | Software Engineering |
Project(s) | Department of IT Management, EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | IADIS International Journal on Computer Science & Information Systems |
Volume | 17 |
Number | 2 |
Pagination | 31-48 |
Date Published | 12/2022 |
Publisher | IADIS |
ISSN | 1646-3692 |
Keywords | artificial intelligence, Gender Equality, Literature Analysis, Literature Review, Social Sustainability |
URL | https://www.iadisportal.org/ijcsis/papers/2022170203.pdf |
Journal articles
Risks and risk mitigation in global software development: an update
Journal of Software: Evolution and Process 33, no. 11 (2021): e2370.Status: Published
Risks and risk mitigation in global software development: an update
Context: Interest in global software development (GSD) has led to the publication of numerous studies. Over time, these studies should be updated to verify if their findings and conclusions remain valid.
Objective: To update a tertiary study, published in 2014, focused on investigating risks and risk mitigation advice in the context of GSD.
Method: We conducted a systematic literature study based on forward snowballing, out of which we identified and selected 25 unique studies.
Results: We extracted: a) 122 risks (75 of which already identified in the original work), and b) 123 mitigation advice (44 of which were identified in the original work). The evidence supporting these risks and mitigation were extracted from 619 and 389 primary sources, respectively. Given the amount of evidence found, we have reported only those with higher empirical support. The raw data, including all the values obtained, are available online as complementary material.
Conclusions: Interest in GSD, its potential risks, and possible mitigation strategies, remains high. This update has allowed increasing the level of empirical support provided by the findings of the original work.
Afilliation | Software Engineering |
Project(s) | Department of IT Management, EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Journal of Software: Evolution and Process |
Volume | 33 |
Issue | 11 |
Pagination | e2370 |
Date Published | 09/2021 |
Publisher | John Wiley & Sons Ltd |
URL | https://onlinelibrary.wiley.com/doi/10.1002/smr.2370 |
DOI | 10.1002/smr.2370 |
Experiential Learning Approach for Software Engineering Courses at Higher Education Level
arXiv preprint arXiv:2012.14178 (2021).Status: Submitted
Experiential Learning Approach for Software Engineering Courses at Higher Education Level
Background: Software project management activities help to introducing software process models in Software Engineering courses. However, these activities should be adequately aligned with the learning outcomes and support student's progression.
Objective: Present and evaluate an approach to help students acquire theoretical and practical knowledge and experience real-world software projects' challenges. The approach combines a serious game and a design-implement task in which students develop a controlled-scale software system.
Methods: To evaluate our approach, we analyzed the students' perceptions collected through an online survey, their project plans, and their final reports using thematic analysis.
Results: Results suggest that the approach promotes knowledge acquisition, enables students' progression, reinforces theoretical concepts, and is properly aligned with the course's learning outcomes.
Conclusion: The approach seems to help introducing software process models in Software Engineering courses. Our experience can also be inspiring for educators willing to apply our approach in similar courses.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | arXiv preprint arXiv:2012.14178 |
Publisher | Springer |
Risks and risk mitigation in global software development: An update
Journal of Software: Evolution and Process 33, no. 11 (2021).Status: Published
Risks and risk mitigation in global software development: An update
Context: Interest in global software development (GSD) has led to the publication of numerous studies. Over time, these studies should be updated to verify if their findings and conclusions remain valid.
Objective: To update a tertiary study, published in 2014, focused on investigating risks and risk mitigation advice in the context of GSD.
Method: We conducted a systematic literature study based on forward snowballing, out of which we identified and selected 25 unique studies.
Results: We extracted: (a) 118 risks (75 of which already identified in the original work), and (b) 167 mitigation advice (44 of which were identified in the original work). Out of 43 new risks identified, just 10 are specific to GSD settings; the others are risks of the software process described in the context of GSD. The evidence supporting these risks and mitigation were extracted from 619 and 389 primary sources, respectively. Given the amount of evidence found, we have reported only those with higher empirical support. The raw data is available online as complementary material.
Conclusions: Interest in GSD, its potential risks, and mitigation strategies, remains high. This update has allowed increasing the level of empirical support provided by the findings of the original work.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Journal of Software: Evolution and Process |
Volume | 33 |
Issue | 11 |
Publisher | John Wiley & Sons Ltd |
Leveraging Network-Centric Strategic Goals in Capabilities
Journal of Military Studies 10 (2021): 90-104.Status: Published
Leveraging Network-Centric Strategic Goals in Capabilities
The vision of network-centric operations is to increase operational capabilities through networked collaboration. NATO and its member nations state this vision in strategic documents at a very high level of abstraction. While suitable for giving an overall feel, current documentation renders the steps toward implementing those visions largely unsupported. We outline a method, based on agile requirements engineering, for converting high-level strategic visions into capabilities, whose form lend themselves to incremental implementation. We illustrate the use of this method in two cases that concern both operational capabilities and technical capabilities. We also show how the method enables one to prioritize which capabilities to develop first. We conclude that some form of explicit methodology to span the gap between strategic visions and implementing those visions is necessary.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector, Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Journal of Military Studies |
Volume | 10 |
Number | 1 |
Pagination | 90-104 |
Publisher | De Gruyter |
Keywords | benefits management, capabilities, requirements engineering, strategic goals |
URL | https://www.sciendo.com/article/10.2478/jms-2021-0001 |
DOI | 10.2478/jms-2021-0001 |
A field experiment on trialsourcing and the effect of contract types on outsourced software development
Information and Software Technology 134 (2021): 106559.Status: Published
A field experiment on trialsourcing and the effect of contract types on outsourced software development
Context: To ensure the success of software projects, it is essential to select skilled developers and to use suitable work contracts. Objective: This study tests two hypotheses: (i) the use of work-sample testing (trialsourcing) improves the selection of skilled software developers; and (ii) the use of contracts based on hourly payment leads to better software project outcomes than fixed-price contracts. Method: Fifty-seven software freelancers with relevant experience and good evaluation scores from previous clients were invited to complete a two-hour long trialsourcing task to qualify for a software development project. Thirty-six developers completed the trialsourcing task with acceptable performance, and, based on a stratified allocation process, were asked to give a proposal based on an hourly payment or a fixed-price contract. Eight hourly payment-based and eight fixed-priced proposals were accepted. The process and product characteristics of the completion of these 16 projects were collected and analysed. Results and Conclusion: Only partial support for our hypotheses was observed. While the use of trialsourcing may have prevented the selection of developers with insufficient skills, the performance on the trialsourcing task of the selected developers did not predict performance on the project. The use of hourly payments led to lower costs than fixed-price contracts, but not to improved processes or products. We plan to follow up these, to us unexpected, results with research on how to design more skill-predictive trialsourcing tasks, and when and why different project contexts give different contract consequences.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Information and Software Technology |
Volume | 134 |
Pagination | 106559 |
Date Published | June 2021 |
Publisher | Elsevier |