A database for publications published by researchers and students at SimulaMet.
Status
Research area
Publication type
- All (124)
- Journal articles (33) Remove Journal articles <span class="counter">(33)</span> filter
- Books (2)
- Edited books (1)
- Proceedings, refereed (28)
- Book chapters (5)
- Talks, keynote (11)
- Proceedings, non-refereed (1)
- Posters (1)
- Technical reports (2)
- Manuals (1)
- Talks, invited (24)
- Talks, contributed (1)
- Public outreach (11)
- Master's theses (1)
- Miscellaneous (2)
Journal articles
Determining a core view of research quality in empirical software engineering
Computer Standards & Interfaces 84 (2023).Status: Published
Determining a core view of research quality in empirical software engineering
Context:
Research quality is intended to appraise the design and reporting of studies. It comprises a set of standards such as methodological rigor, practical relevance, and conformance to ethical standards. Depending on the perspective, different views of importance are given to the standards for research quality.
Objective:
To investigate the suitability of a conceptual model of research quality to Software Engineering (SE), from the perspective of researchers engaged in Empirical Software Engineering (ESE) research, in order to understand the core value of research quality.
Method:
We conducted a mixed-methods approach with two distinct group perspectives: (i) a research group; and (ii) the empirical SE research community. Our data collection approach comprised a questionnaire survey and a complementary focus group. We carried out a hierarchical voting prioritization to collect relative values for importance of standards for research quality.
Results:
In the context of this research, ‘internally valid’, ‘relevant research idea’, and ‘applicable results’ are perceived as the core standards for research quality in empirical SE. The alignment at the research group level was higher compared to that at the community level.
Conclusion:
The conceptual model was seen to express fairly the standards for research quality in the SE context. It presented limitations regarding its structure and components’ description, which resulted in an updated model.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2023 |
Journal | Computer Standards & Interfaces |
Volume | 84 |
Publisher | Elsevier |
A longitudinal explanatory case study of coordination in a very large development programme: the impact of transitioning from a first- to a second-generation large-scale agile development method
Empirical Software Engineering 28, no. 1 (2023).Status: Published
A longitudinal explanatory case study of coordination in a very large development programme: the impact of transitioning from a first- to a second-generation large-scale agile development method
Large-scale agile development has gained widespread interest in the software industry, but it is a topic with few empirical studies of practice. Development projects at scale introduce a range of new challenges in managing a large number of people and teams, often with high uncertainty about product requirements and technical solutions. The coordination of teams has been identified as one of the main challenges. This study presents a rich longitudinal explanatory case study of a very large software development programme with 10 development teams. We focus on inter-team coordination in two phases: one that applies a first-generation agile development method and another that uses a second-generation one. We identified 27 coordination mechanisms in the first phase, and 14 coordination mechanisms in the second. Based on an analysis of coordination strategies and mechanisms, we develop five propositions on how the transition from a first- to a second-generation method impacts coordination. These propositions have implications for theory and practice.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2023 |
Journal | Empirical Software Engineering |
Volume | 28 |
Issue | 1 |
Date Published | Jan-01-2023 |
Publisher | Springer Nature |
ISSN | 1382-3256 |
Keywords | coordination mechanisms, inter-team coordination, large-scale agile development, multiteam systems, software development process, Software Engineering |
URL | https://rdcu.be/c3FQ4 |
DOI | 10.1007/s10664-022-10230-6 |
A logic-based event controller for means-end reasoning in simulation environments
SIMULATION 61 (2023).Status: Published
A logic-based event controller for means-end reasoning in simulation environments
Simulation games are designed to cultivate expertise and rehearse particular skill sets. To yield longitudinal effects, sequences of events must be crafted to yield intended learning outcomes, sometimes by focusing on particularly difficult situations and replaying variants. The present paper develops a logic-based approach for encoding the interrelation between action, events, and objects in a manner that allows the resulting scenario description to immediately be executed in a game development environment. This has the dual effect of decoupling the description of a scenario from the simulation platform itself, as well as supporting iterative and flexible development of learning content. To this end, we provide three interrelated components: First, we develop a scenario description language based on Answer Set Programming. The language is designed to allow an automated reasoner to deduce a schedule of the future events that are caused by an action taken in a given simulation environment. Second, we define a protocol for exchanging actions and computed futures between, respectively, the simulation environment and the external automated reasoner. Finally, as a proof of concept, we develop an Application Programming Interface (API) for the Unity Real-Time Development Platform that implements the protocol and offers a software framework for connecting the computed future events to concrete game objects. This allows the game to evolve coherently from the specification. We argue that the resulting system inherits capabilities for artificial commonsense reasoning from its declarative basis which are useful for reasoning about an evolving emergency incident or training scenario.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2023 |
Journal | SIMULATION |
Volume | 61 |
Date Published | 03/2023 |
Publisher | SAGE journals |
ISSN | 0037-5497 |
URL | http://journals.sagepub.com/doi/10.1177/00375497231157384http://journals... |
DOI | 10.1177/00375497231157384 |
Characteristics and generative mechanisms of software development productivity distributions
Information and Software Technology (2023).Status: Published
Characteristics and generative mechanisms of software development productivity distributions
Context: There is considerable variation in the productivity of software developers. Better knowledge about this variation may provide valuable inputs for the design of skill tests and recruitment processes. Objective: This paper aims to identify properties of software development productivity distributions and gain insight into mechanisms that potentially explain these productivity differences. Method: Four data sets that contain the results of software developers solving the same programming tasks were collected. The properties of the productivity distributions were analyzed, the fits of different types of distributions to the productivity data were compared, and potential generative mechanisms that would lead to the types of distributions with the best fit to the productivity data were evaluated. Results: The coefficient of variance of the productivity of the software developers was, on average, 0.55, with the top 50% of developers having average productivity that was 2.44 times higher than the bottom 50% of developers. All productivity samples were right-skewed, with an average skew of 1.79. About 30% of the observed productivity variance was explained by non-systematic, i.e., within-developer, variance. The distributions with the best fit to the empirical productivity data were the lognormal and power-law-with-an-exponential-cutoff distributions. The analysis of the mechanisms leading to productivity differences found no support for the "rich-getting-richer" explanation proposed for other disciplines. Instead, it suggests a constant productivity difference with increasing experience. Conclusion: The substantial difference in productivity among software developers solving programming tasks indicates that a thorough evaluation of skill in the recruitment process can be rewarding. In particular, the long tail towards higher productivity values demonstrates the large gains that can be achieved by detecting and recruiting developers with very high productivity. More research is needed to understand the mechanisms leading to the large productivity differences.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2023 |
Journal | Information and Software Technology |
Publisher | Elsevier |
Enabling Autonomous Teams and Continuous Deployment at Scale
IEEE IT Professional (2023).Status: Published
Enabling Autonomous Teams and Continuous Deployment at Scale
In this article, we give advice on transitioning to a more agile delivery model for large-scale agile development projects based on experience from the Parental Benefit Project of the Norwegian Labour and Welfare Administration. The project modernized a central part of the organization’s IT portfolio and included up to ten development teams working in parallel. The project successfully changed from using a delivery model which combined traditional project management elements and agile methods to a more agile delivery model with autonomous teams and continuous deployment. This transition was completed in tandem with the project execution. We identify key lessons learned which will be useful for other organizations considering similar changes and report how the new delivery model reduced risk and opened up a range of new possibilities for delivering the benefits of digitalization.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2023 |
Journal | IEEE IT Professional |
Publisher | IEEE |
Improved Measurement of Software Development Effort Estimation Bias
Information and software technology (2023).Status: Published
Improved Measurement of Software Development Effort Estimation Bias
Context: While prior software development effort estimation research has examined the properties of estimation error measures, there has not been much research on the properties of measures of estimation bias. Objectives: Improved measurement of software development effort estimation bias. Methods: Analysis of the extent to which measures of estimation bias meet the criterion that perfect estimates should result in zero bias. Results: Recommendations for measurement of estimation bias for estimates of the mean, median, and mode software development effort. The results include the recommendation to avoid a commonly used measure of effort estimation bias. Conclusion: Proper evaluation of estimation bias requires knowledge about the type of estimates evaluated, together with the selection of a measure of estimation bias that gives zero bias for perfect estimates of that type.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2023 |
Journal | Information and software technology |
Publisher | Elsevier |
Journal articles
A teamwork effectiveness model for agile software development
Empirical Software Engineering 27, no. 2 (2022).Status: Published
A teamwork effectiveness model for agile software development
Teamwork is crucial in software development, particularly in agile development teams which are cross-functional and where team members work intensively together to develop a cohesive software solution. Effective teamwork is not easy; prior studies indicate challenges with communication, learning, prioritization, and leadership. Nevertheless, there is much advice available for teams, from agile methods, practitioner literature, and general studies on teamwork to a growing body of empirical studies on teamwork in the specific context of agile software development. This article presents the agile teamwork effectiveness model (ATEM) for colocated agile development teams. The model is based on evidence from focus groups, case studies, and multi-vocal literature and is a revision of a general team effectiveness model. Our model of agile teamwork effectiveness is composed of shared leadership, team orientation, redundancy, adaptability, and peer feedback. Coordinating mechanisms are needed to facilitate these components. The coordinating mechanisms are shared mental models, communication, and mutual trust. We critically examine the model and discuss extensions for very small, multi-team, distributed, and safety-critical development contexts. The model is intended for re- searchers, team members, coaches, and leaders in the agile community.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | Empirical Software Engineering |
Volume | 27 |
Issue | 2 |
Date Published | Jan-03-2022 |
Publisher | Springer Nature |
ISSN | 1382-3256 |
Keywords | agiel teams, agile leadership, agile methods, agile teamwork model, big five model of teamwork, mutual performance monitoring, peer feedback, redundancy, scrum teams, teamwork model, teamwork theory |
URL | https://rdcu.be/cIINu |
DOI | 10.1007/s10664-021-10115-0 |
Quantifying means-end reasoning skills in simulation-based training: a logic-based approach
SIMULATION 98, no. 10 (2022): 933-957.Status: Published
Quantifying means-end reasoning skills in simulation-based training: a logic-based approach
We develop a logic-based approach for designing simulation-based training scenarios. Our methodology embodies a concise definition of the scenario concept and integrates the notions of training goals, acceptable versus unacceptable actions and performance scoring. The approach applies classical artificial intelligence (AI) planning to extract coherent plays from a causal description of the training domain. The domain- and task-specific parts are defined in a high-level action description language AL. Generic causal and temporal logic is added when the causal theory is compiled into the underlying Answer Set Programming (ASP) language. The ASP representation is used to derive a scoring function that reflects the quality of a play or training session, based on a distinction of states and actions into green (acceptable) and red (unacceptable) ones. To that end, we add to the casual theory a set of norms that specify an initial assignment of colors. The ASP engine uses these norms as axioms and propagates colors by consulting the causal theory. We prove that any set of such norms constitutes a conservative extension of the underlying causal theory. With this work, we hope to lay the foundation for the development of design and analysis tools for exercise managers. We envision a software system that lets an exercise manager view all plays of a tentative scenario design, with expediency information and scores for each possible play. Our approach is applicable to any domain in which means-ends reasoning is pertinent. We illustrate the approach in the domain of crisis response and management.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | SIMULATION |
Volume | 98 |
Issue | 10 |
Pagination | 933-957 |
Date Published | 05/2022 |
Publisher | SAGE journals |
Keywords | Answer Set Programming, automated scoring, deontic logic, means-end reasoning, Simulation-based training |
URL | https://doi.org/10.1177/00375497221095070 |
DOI | 10.1177/00375497221095070 |
When 2 + 2 should be 5: The summation fallacy in time prediction
Journal of Behavioral Decision Making 35, no. 3 (2022): e2265.Status: Published
When 2 + 2 should be 5: The summation fallacy in time prediction
Predictions of time (e.g., work hours) are often based on the aggregation of estimates of elements (e.g., activities, subtasks). The only types of estimates that can be safely aggregated by summation are those reflecting predicted average outcomes (expected values). The sums of other types of estimates, such as bounds of confidence intervals or estimates of the mode, do not have the same interpretation as their components (e.g., the sum of the 90% upper bounds is not the appropriate 90% upper bound of the sum). This can be a potential source of bias in predictions of time, as shown in Studies 1 and 2, where professionals with experience in estimation provided total estimates of time that were inconsistent with their estimates of individual tasks. Study 3 shows that this inconsistency can be attributed to improper aggregation of time estimates and demonstrates how this can produce both over- and underestimation—and also time prediction intervals that are far too wide. Study 4 suggests that the results may reflect a more general fallacy in the aggregation of probabilistic quantities. Our observations are consistent with that inconsistencies and biases are driven by a tendency towards applying a naïve summation (2+2=4) of probabilistic (stochastic) values, in situations where this is not appropriate. This summation fallacy may be in particular consequential in a context where informal estimation methods (expert-judgment based estimation) are used.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | Journal of Behavioral Decision Making |
Volume | 35 |
Issue | 3 |
Pagination | e2265 |
Publisher | Wiley |
When should we (not) use the mean magnitude of relative error (MMRE) as an error measure in software development effort estimation?
Information and Software Technology 143 (2022): 106784.Status: Published
When should we (not) use the mean magnitude of relative error (MMRE) as an error measure in software development effort estimation?
Context: The mean magnitude of relative error (MMRE) is an error measure frequently used to evaluate and compare the estimation performance of prediction models and software professionals.
Objective: This paper examines conditions for proper use of MMRE in effort estimation contexts.
Method: We apply research on scoring functions to identify the type of estimates that minimizes the expected value of the MMRE.
Results: We show that the MMRE is a proper error measure for estimates of the most likely (mode) effort, but not for estimates of the median or mean effort, provided that the effort usage is approximately log-normally distributed, which we argue is a reasonable assumption in many software development contexts. The relevance of the findings is demonstrated on real-world software development data.
Conclusion: MMRE is not a proper measure of the accuracy of estimates of the median or mean effort, but may be used for the accuracy evaluation of estimates of most likely effort.
Afilliation | Software Engineering |
Project(s) | Department of IT Management, EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | Information and Software Technology |
Volume | 143 |
Pagination | 106784 |
Date Published | 03/2022 |
Publisher | Elsevier |