Publications
Proceedings, refereed
Measurement of software development effort estimation bias: Avoiding biased measures of estimation bias
In 11th International Conference on Software Engineering and Applications (SEA 2022), 2022.Status: Accepted
Measurement of software development effort estimation bias: Avoiding biased measures of estimation bias
In this paper, we propose improvements in how estimation bias, e.g., the tendency towards under-estimating the effort, is measured. The proposed approach emphasizes the need to know what the estimates are meant to represent, i.e., the type of estimate we evaluate and the need for a match between the type of estimate given and the bias measure used. We show that even perfect estimates of the mean effort will not lead to an expectation of zero estimation bias when applying the frequently used bias measure: (actual effort – estimated effort)/actual effort. This measure will instead reward under-estimates of the mean effort. We also provide examples of bias measures that match estimates of the mean and the median effort, and argue that there are, in general, no practical bias measures for estimates of the most likely effort. The paper concludes with implications for the evaluation of bias of software development effort estimates.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | 11th International Conference on Software Engineering and Applications (SEA 2022) |
Journal Article
Realizing benefits in public IT projects: A multiple case study
a journal (2022).Status: Submitted
Realizing benefits in public IT projects: A multiple case study
IT investments in the public sector are large, and it is essential that they lead to benefits for the organizations themselves and for the wider society. While there is evidence suggesting a positive connection between the existence of benefits management practices and benefits realization, less is known about how to implement such practices effectively. The paper aims to provide insights into when benefits are most likely to be realized, and how benefits management practices and roles should be implemented in order to have a positive effect on the projects’ success in terms of realizing benefits. The authors collected data relating to ten Norwegian public IT projects. For each project, they collected data on benefits management from project documents, by interviewing the project owners and benefits owners, and follow-up surveys. The benefits internal to the organization were those with the highest degree of realization, while the societal benefits were those with the lowest degree. Projects assessed to have more specific, measurable, accountable, and realistically planned benefits were more successful in realizing benefits. Benefits owners were most effective when they were able to attract attention towards the benefits to be realized, had a strong mandate, and had domain expertise.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | a journal |
Publisher | x |
When 2 + 2 should be 5: The summation fallacy in time prediction
Journal of Behavioral Decision Making (2022).Status: Accepted
When 2 + 2 should be 5: The summation fallacy in time prediction
Predictions of time (e.g., work hours) are often based on the aggregation of estimates of elements (e.g., activities, subtasks). The only types of estimates that can be safely aggregated by summation are those reflecting predicted average outcomes (expected values). The sums of other types of estimates, such as bounds of confidence intervals or estimates of the mode, do not have the same interpretation as their components (e.g., the sum of the 90% upper bounds is not the appropriate 90% upper bound of the sum). This can be a potential source of bias in predictions of time, as shown in Studies 1 and 2, where professionals with experience in estimation provided total estimates of time that were inconsistent with their estimates of individual tasks. Study 3 shows that this inconsistency can be attributed to improper aggregation of time estimates and demonstrates how this can produce both over- and underestimation—and also time prediction intervals that are far too wide. Study 4 suggests that the results may reflect a more general fallacy in the aggregation of probabilistic quantities. Our observations are consistent with that inconsistencies and biases are driven by a tendency towards applying a naïve summation (2+2=4) of probabilistic (stochastic) values, in situations where this is not appropriate. This summation fallacy may be in particular consequential in a context where informal estimation methods (expert-judgment based estimation) are used.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | Journal of Behavioral Decision Making |
Publisher | Wiley |
When should we (not) use the mean magnitude of relative error (MMRE) as an error measure in software development effort estimation?
Information and Software Technology 143 (2022).Status: Accepted
When should we (not) use the mean magnitude of relative error (MMRE) as an error measure in software development effort estimation?
Context: The mean magnitude of relative error (MMRE) is an error measure frequently used to evaluate and compare the estimation performance of prediction models and software professionals.
Objective: This paper examines conditions for proper use of MMRE in effort estimation contexts.
Method: We apply research on scoring functions to identify the type of estimates that minimizes the expected value of the MMRE.
Results: We show that the MMRE is a proper error measure for estimates of the most likely (mode) effort, but not for estimates of the median or mean effort, provided that the effort usage is approximately log-normally distributed, which we argue is a reasonable assumption in many software development contexts. The relevance of the findings is demonstrated on real-world software development data.
Conclusion: MMRE is not a proper measure of the accuracy of estimates of the median or mean effort, but may be used for the accuracy evaluation of estimates of most likely effort.
Afilliation | Software Engineering |
Project(s) | Department of IT Management, EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | Information and Software Technology |
Volume | 143 |
Date Published | 03/2022 |
Publisher | Elsevier |
Journal Article
A field experiment on trialsourcing and the effect of contract types on outsourced software development
Information and Software Technology 134 (2021): 106559.Status: Published
A field experiment on trialsourcing and the effect of contract types on outsourced software development
Context: To ensure the success of software projects, it is essential to select skilled developers and to use suitable work contracts. Objective: This study tests two hypotheses: (i) the use of work-sample testing (trialsourcing) improves the selection of skilled software developers; and (ii) the use of contracts based on hourly payment leads to better software project outcomes than fixed-price contracts. Method: Fifty-seven software freelancers with relevant experience and good evaluation scores from previous clients were invited to complete a two-hour long trialsourcing task to qualify for a software development project. Thirty-six developers completed the trialsourcing task with acceptable performance, and, based on a stratified allocation process, were asked to give a proposal based on an hourly payment or a fixed-price contract. Eight hourly payment-based and eight fixed-priced proposals were accepted. The process and product characteristics of the completion of these 16 projects were collected and analysed. Results and Conclusion: Only partial support for our hypotheses was observed. While the use of trialsourcing may have prevented the selection of developers with insufficient skills, the performance on the trialsourcing task of the selected developers did not predict performance on the project. The use of hourly payments led to lower costs than fixed-price contracts, but not to improved processes or products. We plan to follow up these, to us unexpected, results with research on how to design more skill-predictive trialsourcing tasks, and when and why different project contexts give different contract consequences.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Information and Software Technology |
Volume | 134 |
Pagination | 106559 |
Date Published | June 2021 |
Publisher | Elsevier |
Backsourcing of Information Technology - A Systematic Literature Review
Information Systems Research (2021).Status: Submitted
Backsourcing of Information Technology - A Systematic Literature Review
Context: Backsourcing is the process of insourcing previously outsourced activities. When companies experience environmental or strategic changes, or challenges with outsourcing, backsourcing can be a viable alternative. While outsourcing and related processes have been extensively studied in information technology, few studies report experiences with backsourcing.
Objectives: We intend to summarize the results of the research literature on the backsourcing of IT, with a focus on software development. By identifying practical relevance experience, we aim to present findings that may help companies considering backsourcing. In addition, we aim to identify gaps in the current research literature and point out areas for future work.
Method: Our systematic literature review (SLR) started with a search for empirical studies on the backsourcing of IT. From each study we identified the contexts in which backsourcing occurs, the factors leading to the decision to backsource, the backsourcing process itself, and the outcomes of backsourcing. We employed inductive coding to extract textual data from the papers identified and qualitative cross-case analysis to synthesize the evidence from backsourcing experiences.
Results: We identified 17 papers that reported 26 cases of backsourcing, six of which were related to software development. The cases came from a variety of contexts. The most common reasons for backsourcing were improving quality, reducing costs, and regaining control of outsourced activities. The backsourcing process can be described as containing five sub-processes: change management, vendor relationship management, competence building, organizational build-up, and transfer of ownership. Furthermore, we identified 14 positive outcomes and nine negative outcomes of backsourcing. Finally, we aggregated the evidence and detailed three relationships of potential use to companies considering backsourcing.
Conclusion: The backsourcing of IT is a complex process; its implementation depends on the prior outsourcing relationship and other contextual factors. Our systematic literature review may contribute to a better understanding of this process by identifying its components and their relationships based on the peer-reviewed literature. Our results may also serve as a motivation and baseline for further research on backsourcing and may provide guidelines and process fragments from which practitioners can benefit when they engage in backsourcing.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector, Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Information Systems Research |
Publisher | Institute for Operations Research and the Management Sciences |
Keywords | backshoring, Backsourcing, information technology, software development, Software Engineering, systematic literature review |
Benefits management in software development: A systematic review of empirical studies
IET Software 15, no. 1 (2021): 1-24.Status: Published
Benefits management in software development: A systematic review of empirical studies
Considerable resources are wasted on software projects delivering less than the planned benefits. Herein, the objective is to synthesize empirical evidence of the adoption and impact of benefits management (BM) in software development, and to suggest directions for future research. A systematic review of the literature is performed and identified 4836 scientific papers of which the authors found 47 to include relevant research. While most organizations identify and structure benefits at the outset of a project, fewer organizations report implementing BM as a continuous process throughout the project lifecycle. Empirical evidence gives support for positive impact on project outcome from the following BM practices: identifying and structuring benefits, planning benefits realization, BM during project execution, benefits evaluation and the practice of having people responsible for benefits realization. The authors suggest four research directions to understand (1) why BM practices sometimes not are adopted, (2) BM in relation to other management practices, (3) BM in agile software development and (4) BM in the context of organizations' value creation logics.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | IET Software |
Volume | 15 |
Issue | 1 |
Pagination | 1-24 |
Publisher | IET |
Evaluation of Probabilistic Project Cost Estimates
IEEE Transactions on Engineering Management (2021): 1-16.Status: Published
Evaluation of Probabilistic Project Cost Estimates
Evaluation of cost estimates should be fair and give incentives for accuracy. These goals, we argue, are challenged by a lack of precision in what is meant by a cost estimate and the use of evaluation measures that do not reward the most accurate cost estimates. To improve the situation, we suggest the use of probabilistic cost estimates and propose guidelines on how to evaluate such estimates. The guidelines emphasize the importance of a match between the type of cost estimate provided by the estimators and the chosen cost evaluation measure, and the need for an evaluation of both the calibration and the informativeness of probabilistic cost estimates. The feasibility of the guidelines is exemplified in an analysis of a set of 69 large Norwegian governmental projects. The evaluation indicated that the projects had quite accurate and unbiased P50 estimates and that the prediction intervals were reasonably well calibrated. It also showed that the cost prediction intervals were non-informative with respect to differences in cost uncertainty and, consequently, not useful to identify projects with higher cost uncertainty. The results demonstrate the usefulness of applying the proposed cost estimation evaluation guidelines.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | IEEE Transactions on Engineering Management |
Pagination | 1-16 |
Date Published | 08/2021 |
Publisher | IEEE |
Jakten på nytte i offentlige IT-prosjekter
Stat & Styring 31, no. 3 (2021): 38-41.Status: Published
Jakten på nytte i offentlige IT-prosjekter
Det er økende oppmerksomhet på at offentlige tiltak skal ha nytte for brukere og samfunn – heldigvis. Det hjelper, for eksempel, lite at et nytt IT-system blir levert med spesifisert funksjonalitet, til avtalt tid og uten overskridelse av budsjettet dersom systemet ikke blir brukt og leverer nytte som forutsatt.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Stat & Styring |
Volume | 31 |
Issue | 3 |
Pagination | 38-41 |
Date Published | 10/2021 |
Publisher | Scandinavian University Press |
Relative estimates of software development effort: Are they more accurate or less time-consuming to produce than absolute estimates, and to what extent are they person-independent?
Information and Software Technology 143 (2021): 1-9.Status: Published
Relative estimates of software development effort: Are they more accurate or less time-consuming to produce than absolute estimates, and to what extent are they person-independent?
Context: Estimates of software development effort may be given as judgments of relationships between the use of efforts on different tasks – that is, as relative estimates. The use of relative estimates has increased with the introduction of story points in agile software development contexts.
Objective: This study examines to what extent relative estimates are likely to be more accurate or less time-consuming to produce than absolute software development effort estimates and to what extent relative estimates can be considered developer-independent.
Method: We conducted two experiments. In the first experiment, we collected estimates from 102 professional software developers estimating the same tasks and randomly allocated to providing relative estimates in story points or absolute estimates in work-hours. In the second experiment, we collected the actual efforts of 20 professional software developers completing the same 5 programming tasks and used these to analyse the variance in relative efforts.
Results: The results from the first experiment indicates that the relative estimates were less accurate than the absolute estimates, and that the time consumed completing the estimation work was higher for those using relative estimation, even when only considering developers with extensive prior experience in story point–based estimation for both tasks. The second experiment revealed that the relative effort was far from developer-independent, especially for the least productive developers. This suggests that relative estimates to a large extent are developer-dependent.
Conclusions: Although there may be good reasons for the continued use of relative estimates, we interpret our results as not supporting that the use of relative estimates is connected with higher estimation accuracy or less time consumed on producing the estimates. Neither do our results support a high degree of developer-independence in relative estimates.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Information and Software Technology |
Volume | 143 |
Pagination | 1-9 |
Date Published | 03/2021 |
Publisher | Wiley |
Miscellaneous
Hvordan lykkes med digitalisering? En undersøkelse av nyttestyring i IT-prosjekter i offentlig sektor (Concept-rapport nr. 64)
NTNU: Ex Ante Akademisk forlag, 2021.Status: Published
Hvordan lykkes med digitalisering? En undersøkelse av nyttestyring i IT-prosjekter i offentlig sektor (Concept-rapport nr. 64)
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Miscellaneous |
Year of Publication | 2021 |
Publisher | Ex Ante Akademisk forlag |
Place Published | NTNU |
ISBN Number | 978-82-8433-004-4 |
Proceedings, refereed
Relations Between Effort Estimates, Skill Indicators, and Measured Programming Skill
In ESEC/FSE. Journal First publication (IEEE TSE), 2021.Status: Published
Relations Between Effort Estimates, Skill Indicators, and Measured Programming Skill
There are large skill differences among software developers, and clients and managers will benefit from being able to identify those with better skill. This study examines the relations between low effort estimates, and other commonly used skill indicators, and measured programming skill. One hundred and four professional software developers were recruited. After skill-related information was collected, they were asked to estimate the effort for four larger and five smaller programming tasks. Finally, they completed a programming skill test. The lowest and most over-optimistic effort estimates for the larger tasks were given by those with the lowest programming skill, which is in accordance with the well-known Dunning-Kruger effect. For the smaller tasks, however, those with the lowest programming skill had the highest and most over-pessimistic estimates. The other programming skill indicators, such as length of experience, company assessed skill and self-assessed skill, were only moderately correlated with measured skill and not particularly useful in guiding developer skill identification. A practical implication is that for larger and more complex tasks, the use of low effort estimates and commonly used skill indicators as selection criteria leads to a substantial risk of selecting among the least skilled developers.
Afilliation | Software Engineering |
Project(s) | EDOS: Effective Digitalization of Public Sector |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | ESEC/FSE |
Publisher | Journal First publication (IEEE TSE) |
Relations Between Effort Estimates, Skill Indicators, and Measured Programming Skill (A Journal first conference publication)
In CM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). ACM, 2021.Status: Published
Relations Between Effort Estimates, Skill Indicators, and Measured Programming Skill (A Journal first conference publication)
There are large skill differences among software developers, and clients and managers will benefit from being able to identify those with better skill. This study examines the relations between low effort estimates, and other commonly used skill indicators, and measured programming skill. One hundred and four professional software developers were recruited. After skill-related information was collected, they were asked to estimate the effort for four larger and five smaller programming tasks. Finally, they completed a programming skill test. The lowest and most over-optimistic effort estimates for the larger tasks were given by those with the lowest programming skill, which is in accordance with the well-known Dunning-Kruger effect. For the smaller tasks, however, those with the lowest programming skill had the highest and most over-pessimistic estimates. The other programming skill indicators, such as length of experience, company assessed skill and self-assessed skill, were only moderately correlated with measured skill and not particularly useful in guiding developer skill identification. A practical implication is that for larger and more complex tasks, the use of low effort estimates and commonly used skill indicators as selection criteria leads to a substantial risk of selecting among the least skilled developers.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | CM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) |
Publisher | ACM |
Team resource management decisions in software development projects
In IEEE 23rd Conference on Business Informatics (CBI) . IEEE, 2021.Status: Published
Team resource management decisions in software development projects
Purpose: This paper examines to what extent resource managers are likely to make normatively correct decisions in complex, but realistic, software development team resource management situations.
Research design: Three scenarios were designed, all of them with a software development project consisting of a higher and a lower productivity team. The resource managers were asked to decide which team to increase (or decrease) the productivity by 10% in order to minimize the total effort or the total duration of the software project. Ninety-nine software professionals, most of them with substantial resource management experience, were randomly allocated one of the scenarios.
Findings: Most of the resource managers perceived their scenario as realistic and occurring in practice. The decisions tended, however, to be different from the normatively correct ones. In particular, when the scenario structure resembled that used to document a time-saving bias, the great majority of the managers made non-normative decisions. The findings suggest that the resource managers made decisions based on simple heuristics, often leading to non-normative decisions. When asked about the decisions they would make in practice, more of the resource managers gave normatively correct responses. Our findings suggest a cost-saving potential from more awareness of how to make team resource management decisions.
Originality: The study may be the first to document non-normative team resource decisions, including those related to the time-saving bias, in the context of project management.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | IEEE 23rd Conference on Business Informatics (CBI) |
Publisher | IEEE |
Talk, keynote
The 8th International Conference on Electrical Engineering and Informatics
In ICEEI2021 - The 8th International Conference on Electrical Engineering and Informatics, Malaysia (virtual conference), 2021.Status: Published
The 8th International Conference on Electrical Engineering and Informatics
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talk, keynote |
Year of Publication | 2021 |
Location of Talk | ICEEI2021 - The 8th International Conference on Electrical Engineering and Informatics, Malaysia (virtual conference) |
URL | www.ftsm.ukm.my/iceei2021 |
Proceedings, refereed
Benefits management and agile practices in software projects: how perceived benefits are impacted
In IEEE 22nd Conference on Business Informatics (CBI). Vol. 2. IEEE, 2020.Status: Published
Benefits management and agile practices in software projects: how perceived benefits are impacted
Considerable resources are wasted on projects that deliver few or no benefits. The main objective is to better understand the characteristics of projects that are successful in delivering good client benefits. We asked 71 Norwegian software professionals to report information about projects completed between 2016 and 2018. We found that both benefits management and agile practices have a significant relationship with perceived realisation of client benefits. This includes the benefits management practices of having a plan for benefits realisation, individuals with assigned responsibility for benefits realisation, benefits management during project execution, quantification of realised benefits, evaluation of realised benefits, re-estimation of benefits during project execution, and the agile practices of a flexible scope and frequent deliveries to production. The software projects that were successful in delivering client benefits adopted benefits management and agile practices to a larger extent than the less successful ones. Future studies are required to establish more comprehensive understanding of what distinguishes projects that deliver good client benefits from the rest, including studies of the realisation of client benefits in agile software projects.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | IEEE 22nd Conference on Business Informatics (CBI) |
Volume | 2 |
Pagination | 48-56 |
Publisher | IEEE |
Journal Article
How to pose for a professional photo: The effect of three facial expressions on perception of competence of a software developer
Australian Journal of Psychology 72 (2020): 257-266.Status: Published
How to pose for a professional photo: The effect of three facial expressions on perception of competence of a software developer
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Australian Journal of Psychology |
Volume | 72 |
Pagination | 257-266 |
Publisher | Wiley |
Practices connected to perceived client benefits of software projects
IET Software 14, no. 6 (2020): 677-683.Status: Published
Practices connected to perceived client benefits of software projects
It is well-documented that many software projects deliver fewer benefits than planned. However prior research has had a stronger focus on the ability to deliver within budget, on time and with the specified functionality, than on what to do to successfully deliver client benefits. The authors have conducted a survey collecting information about benefits management practices, agile practices, use of contracts, and the perceived success in delivery of client benefits. The authors received responses from 83 software professionals with information about 73 recent and 74 older software projects. There was no statistically significant improvement of the delivered client benefits from the older to the recent projects. Statistically significant findings, applying a general linear model-based analysis, include that the degree of success in delivering client benefits is connected to a project having: (i) a plan for how to realise the benefits, (ii) implemented practices for benefits management during project execution, (iii) frequent deliveries to production during the project execution, and (iv) a process for the evaluation of realised benefits after project completion. The authors argue that greater use of these practices represents a potential for organisations to increase their success in delivering benefits from software projects.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | IET Software |
Volume | 14 |
Issue | 6 |
Pagination | 677-683 |
Publisher | The Institution of Engineering and Technology |
Relations Between Effort Estimates, Skill Indicators, and Measured Programming Skill
Transactions on Software Engineering (2020).Status: Published
Relations Between Effort Estimates, Skill Indicators, and Measured Programming Skill
There are large skill differences among software developers, and clients and managers will benefit from being able to identify those with better skill. This study examines the relations between low effort estimates, and other commonly used skill indicators, and measured programming skill. One hundred and four professional software developers were recruited. After skill-related information was collected, they were asked to estimate the effort for four larger and five smaller programming tasks. Finally, they completed a programming skill test. The lowest and most over-optimistic effort estimates for the larger tasks were given by those with the lowest programming skill, which is in accordance with the well-known Dunning-Kruger effect. For the smaller tasks, however, those with the lowest programming skill had the highest and most over-pessimistic estimates. The other programming skill indicators, such as length of experience, company assessed skill and self-assessed skill, were only moderately correlated with measured skill and not particularly useful in guiding developer skill identification. A practical implication is that for larger and more complex tasks, the use of low effort estimates and commonly used skill indicators as selection criteria leads to a substantial risk of selecting among the least skilled developers.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Transactions on Software Engineering |
Date Published | February 2020 |
Publisher | IEEE |
Sequence effects in the estimation of software development effort
Journal of Systems and Software 159, no. January 2020 (2020): 110448.Status: Published
Sequence effects in the estimation of software development effort
Currently, little is known about how much the sequence in which software development tasks or projects are estimated affects judgment-based effort estimates. To gain more knowledge, we examined estimation sequence effects in two experiments. In the first experiment, 362 software professionals estimated the effort of three large tasks of similar sizes, whereas in the second experiment 104 software professionals estimated the effort of four large and five small tasks. The sequence of the tasks was randomised in both experiments. The first experiment, with tasks of similar size, showed a mean increase of 10% from the first to the second and a 3% increase from the second to the third estimate. The second experiment showed that estimating a larger task after a smaller one led to a mean decrease in the estimate of 24%, and that estimating a smaller task after a larger one led to a mean increase of 25%. There was no statistically significant reduction in the sequence effect with higher competence. We conclude that more awareness about how the estimation sequence affects the estimates may reduce potentially harmful estimation biases. In particular, it may reduce the likelihood of a bias towards too low effort estimates.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Journal of Systems and Software |
Volume | 159 |
Issue | January 2020 |
Pagination | 110448 |
Publisher | Elsevier |
Talks, invited
Agile software development and benefits management: A perfect match?
In Seminar on effective software development, Kathmandu, Nepal, 2019.Status: Published
Agile software development and benefits management: A perfect match?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2019 |
Location of Talk | Seminar on effective software development, Kathmandu, Nepal |
Conducting realistic, controlled experiments in software engineering
In Seminar for PhD-students at University of Manaus, Brazil, 2019.Status: Published
Conducting realistic, controlled experiments in software engineering
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2019 |
Location of Talk | Seminar for PhD-students at University of Manaus, Brazil |
Estimering av IT-utvikling
In Seminar, Knowit, Norway, 2019.Status: Published
Estimering av IT-utvikling
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2019 |
Location of Talk | Seminar, Knowit, Norway |
Hva kjennetegner IT -utvikling som lykkes?
In Seminar for Helsedirektoratet, Hønefoss, Norway, 2019.Status: Published
Hva kjennetegner IT -utvikling som lykkes?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2019 |
Location of Talk | Seminar for Helsedirektoratet, Hønefoss, Norway |
Hva kjennetegner IT-utvikling som lykkes?
In Presentasjon for Skatteetaten (intern workshop), Norway, 2019.Status: Published
Hva kjennetegner IT-utvikling som lykkes?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2019 |
Location of Talk | Presentasjon for Skatteetaten (intern workshop), Norway |
What makes software development projects successful, and what makes them fail? (and how to find out)
In Presentation at Simula Summer School, Berlin, Germany, 2019.Status: Published
What makes software development projects successful, and what makes them fail? (and how to find out)
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2019 |
Location of Talk | Presentation at Simula Summer School, Berlin, Germany |
Technical reports
Estimering av kostnader i store statlige prosjekter: Hvor gode er estimatene og usikkerhetsanalysene i KS2-rapportene?
In Concept-rapport nr. 59. Trondheim: Ex ante akademisk forlag, 2019.Status: Published
Estimering av kostnader i store statlige prosjekter: Hvor gode er estimatene og usikkerhetsanalysene i KS2-rapportene?
The external quality assurance scheme for large government investment projects (the QA scheme / the state project model) aims, among other things, to ensure that budgets are realistic and that the risk analyses of the cost estimates reflect real cost uncertainty. The extent to which budgets, estimates and risk analyses are realistic, and where there may be potentials for improvements, are the main themes of this study.
Chapter 1 describes the background and motivation for the study. The starting point is that the Concept research programme collects final costs in projects that have been through QA2 (quality assurance of cost estimate before the parliament’s investment decision). That provides a basis for studies of cost performance. As the sample of projects increase, more detailed studies of the estimates that formed the basis for the parliament’s investment decision becomes possible.
The study has three main topics. We look at:
The realism in the projects’ budgets
The realism in the point estimates in the QA2 reports, and
The realism and information value in the prediction intervals and estimation distributions.
Chapter 2 provides a review of previous studies of cost performance in projects that have been through QA2. They all show relatively good results both in terms of deviation from budgets and risk assessments. While average cost overruns reported in international studies typically have been around 30 per cent, Norwegian studies report average overruns of between two and six per cent. Other studies also typically report a strong underestimation of uncertainty. The P50 and P85 estimates from the QA2 reports on the other hand (that is, estimates that are not expected to be exceeded in 50 and 85 per cent of cases, respectively) seem to have been reasonably well calibrated. However, several authors have pointed out that the distribution of final costs to the budgets have been somewhat higher than assumed at the time of the investment decision.
The data used in the study, which is described in Chapter 3, is based on a larger sample of projects than previous studies. The analyses focus more on the estimates than previous studies have done. The analysis of the P50 and P85 estimates is based on samples of 83 and 85 projects respectively. Sufficient data for our analysis of the cost estimates were found for 70 of these projects.
In Chapter 4, we outline detailed research questions and the methodology for the analyses. In this, we motivate and indicate, based on the latest research on the area, how probability-based cost estimates should be evaluated.
We introduce an analysis of estimate deviations and estimation bias based on what is a reasonable "loss function", where the loss function is what we attempt to minimise in the estimates. We evaluate the extent to which we have been successful in estimating the real uncertainty of projects ex ante. We also assess how informative prediction intervals and estimate distributions have been. We argue that well-calibrated probability-based estimates (e.g., that 50 per cent of P50 estimates should not be exceeded) are not a sufficient evaluation criterion. In addition, we need indicators for how informative the probability-based estimates have been.
In Chapter 5, we find that the median deviations between actual costs and the P50, measured as absolute percentage deviation, is 10 per cent (mean = 12.5), and that the median deviation from the P85 is 1.5 per cent (average = 3.4). In other words, for all the projects, there is only a slight tendency for overruns, and much lower than what has been reported in international studies. Over time, however, there has been a somewhat worrying development. While there was a tendency for cost underruns in the past (an average of 6 per cent underruns of the P50 for projects with an investment decision between 2001 and 2003), there has been a tendency for cost overruns in the later years (an average of 12 per cent overruns in the period 2010-2012).
Given well-calibrated estimates, the actual cost should be below the P50 in about 50 per cent of the cases and below the P85 in about 85 per cent of the cases. However, we find that this only applies in 40 per cent of the cases for the P50 and 73 per cent for the P85. The shares have been declining over time. While in 2001-2003, 62 and 100 per cent were within the P50 and P85 respectively, in 2010-2012 there were only 21 and 43 per cent within, albeit based on a smaller sample than in the time-periods before. The reason why hit rates for the P50 and the P85 for all projects together are not so far from the intended targets is because we have gone from overestimation to underestimation. The tendency for underestimation should be reversed through better estimation and governance in future projects.
The analyses of the estimates in Chapter 6 find about the same degree of overruns and estimate deviations for the P50 and P85 estimates as those reported in Chapter 5. The P50 estimates showed a median estimate bias of -1 per cent (mean = 3 per cent). The median percentage deviation (regardless of sign) was 12 per cent (mean = 14 per cent). We calculated that the expected deviation from the P50 budget could not be less than 8-10 per cent, given some assumptions that the projects do not adapt deliveries to reduce deviations. Although the latter assumption hardly is met, this calculation suggests that the deviations are not particularly high.
We observe that there is typically a reduction from estimate to budget. The P50 budget was on average seven per cent lower than the P50 estimate and the P85 budget seven per cent lower than the P85 estimate. Although there were several projects that should have retained the original P50 and P85 estimates as P50 and P85 budget, respectively, we did not find that the adjustments overall reduced the realism. Many of the adjustments seem to be well justified.
The estimates in the QA2 reports include both point estimates, prediction intervals and estimate distributions (S-curves). Our analyses include all of these and have as their main findings are as follows:
The estimation distributions and prediction intervals are typically too narrow to reflect actual uncertainty. For example, as many as 19 per cent of the projects have a lower cost than the P10 estimate and 20 per cent more than the P90 estimate. Future estimation should take into account that the scope for project costs is broader than previously typically assumed.
Estimated cost uncertainty, estimated through the width of the prediction interval and estimate distribution, does not correlate with actual cost uncertainty, measured by cost deviations and overruns. This indicates a low ability to distinguish between projects with high and low cost uncertainty. If we become better at identifying the high-risk projects, we could potentially reduce the need for risk contingency without compromising cost performance and project execution. We show, given some assumptions, that the P85 could be 17 per cent lower if the ability to distinguish between low and high risk projects had been better. Measures to improve this capability should be given priority in the estimation work.
There are differences in estimation performance between agencies and between consultancies carrying out the external QA. Defence projects stand out by having a strong tendency to overestimate costs (their average underrun of the P50 estimate is 19 per cent) and overly narrow prediction intervals (29 per cent of projects within the 80 percent prediction range). The Norwegian Public Roads Administration also tends to estimate too narrow prediction intervals (57 per cent of projects within the 80 per cent prediction interval). Among the QA consultancies, there are no major differences in estimate deviations, but larger differences in how realistic the uncertainty is estimated. There may be differences in project complexity or other issues that explain these differences.
Given the inability to distinguish between low- and high-risk projects in the estimation work, a simple mechanical mark-up model could in theory do just as well as the more demanding QA2 estimation work. We investigated this, where the uplifts were based on historical estimate deviations, but found that the QA2 estimates did better. This indicates that the work done in the QA2 estimation provides added value, measured against simple mark-up models.
In Chapter 7, we summarize and discuss the findings. Overall, the main conclusions are that the QA2 framework is useful and that cost estimates appear to be realistic and reasonably well calibrated. However, developments over time are worrying and should lead to improvements in the estimation work. Two major areas of improvement are to specify broader estimate distributions, that is, to recognize that cost uncertainty is typically greater than that which has previously been identified in the estimation work, as well as to better distinguish between projects with low and high cost uncertainty.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Technical reports |
Year of Publication | 2019 |
Secondary Title | Concept-rapport nr. 59 |
Publisher | Ex ante akademisk forlag |
Place Published | Trondheim |
ISBN Number | 78-82-93253-81-5 |
ISSN Number | 0803-9763 |
Journal Article
Evaluating Probabilistic Software Development Effort Estimates Maximizing Informativeness Subject to Calibration
Information and Software Technology 115 (2019): 93-96.Status: Published
Evaluating Probabilistic Software Development Effort Estimates Maximizing Informativeness Subject to Calibration
Context: Probabilistic effort estimates inform about the uncertainty and may give useful input to plans, budgets and investment analyses. Objective & Method: This paper introduces, motivates and illustrates two principles on how to evaluate the accuracy and other performance criteria of probabilistic effort estimates in software development contexts. Results: The first principle emphasizes a consistency between the estimation error measure and the loss function of the chosen type of probabilistic single point effort estimates. The second principle points at the importance of not just measuring calibration, but also informativeness of estimated prediction intervals and distributions. The relevance of the evaluation principles is illustrated by a performance evaluation of estimates from twenty-eight software professionals using two different uncertainty assessment methods to estimate the effort of the same thirty software maintenance tasks.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Information and Software Technology |
Volume | 115 |
Pagination | 93-96 |
Publisher | Elsevier |
Relations between Project Size, Agile Practices and Successful Software Development
IEEE Software 36, no. 2 (2019): 39-43.Status: Published
Relations between Project Size, Agile Practices and Successful Software Development
The use of agile methods in the execution of large-scale software development is increasing. To find out more about the effect of this on project performance, information was collected about 196 Norwegian IT-projects. Increased project size was associated with decreased project performance for both agile and non-agile projects, but the projects using agile methods had better performance than the non-agile projects for all examined project size categories. Flexible scope, frequent deliveries to production, a high degree of requirement changes and more competent providers are candidates to explain the better performance of agile projects.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | IEEE Software |
Volume | 36 |
Issue | 2 |
Pagination | 39-43 |
Publisher | IEEE |
Public outreach
Forstyrrelser, avbrudd og produktivitet
Computerworld Norway, 2019.Status: Published
Forstyrrelser, avbrudd og produktivitet
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Public outreach |
Year of Publication | 2019 |
Publisher | Computerworld Norway |
I praksis riktig, men i teorien feil: Kostnadsbesparelser i team
Computerworld Norway, 2019.Status: Published
I praksis riktig, men i teorien feil: Kostnadsbesparelser i team
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Public outreach |
Year of Publication | 2019 |
Publisher | Computerworld Norway |
Sannsynlighetsvurderinger av kostnader
Computerworld Norway, 2019.Status: Published
Sannsynlighetsvurderinger av kostnader
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Public outreach |
Year of Publication | 2019 |
Publisher | Computerworld Norway |
Smidig og stor - en selvmotsigelse innen IT-utvikling?
Computerworld Norway, 2019.Status: Published
Smidig og stor - en selvmotsigelse innen IT-utvikling?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Public outreach |
Year of Publication | 2019 |
Publisher | Computerworld Norway |
Usannsynlige sannsynligheter
Computerworld Norway, 2019.Status: Published
Usannsynlige sannsynligheter
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Public outreach |
Year of Publication | 2019 |
Publisher | Computerworld Norway |
Talk, keynote
From myths and fashions to evidence-based software engineering
In Seminar on effective software development, Kathmandu, Nepal, 2019.Status: Published
From myths and fashions to evidence-based software engineering
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talk, keynote |
Year of Publication | 2019 |
Location of Talk | Seminar on effective software development, Kathmandu, Nepal |
What can - and should - software engineering learn from psychology?(with a focus on research method)
In ICSIE, Cairo, Egypt, 2019.Status: Published
What can - and should - software engineering learn from psychology?(with a focus on research method)
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talk, keynote |
Year of Publication | 2019 |
Location of Talk | ICSIE, Cairo, Egypt |
Proceedings, non-refereed
Predicting software development skill from effort predictions (Un)skilled and unware of it?
In International symposium on forecasting (ISF), 2019.Status: Published
Predicting software development skill from effort predictions (Un)skilled and unware of it?
If software developers’ estimates of how much effort they would need to complete tasks were strongly correlated with their actual use of efforts, we could safely select among the developers with the lowest effort estimates and predict that they will be among the most skilled ones. Unfortunately, this is not necessarily the case. As documented in studies in many domains, those with lower skill tend to know less about how little they know and, for this reason, give over-optimistic estimates of their own performance.
This effect is termed the Dunning-Kruger effect. Does this mean that effort estimates are useless as predictors of software development skill?
To find out more on this we requested 104 software developers to estimate four larger and five smaller software development tasks and measured their programming skills. We assessed the connection between lower estimates and higher skill through rank correlations and hit rates, where the hit rate measures the frequency of selecting the most skilled out of two developers when selecting the one with the lower effort estimate on the same task.
The results were as predicted by the Dunning-Kruger effect for the larger tasks. The developers in the lowest skill quartile had on average lower estimates than those in the highest skill quartile. To predict relative programming skill based on these estimates would be very inaccurate. The correlations between task estimates and skill were between -0.15 and 0.01 and the hit rates between 44% and 50%.
The developers’ effort estimates on the smaller tasks were, however, much better connected with their measured programming skills. The rank correlations were between 0.29 and 0.50 and the hit rates between 62% and 68%. Clustering the estimates of all the smaller tasks into the same factor gave a rank correlation of 0.45 between this factor and the measured skill and a hit rate of 70%. While not very strong, we show that the estimates on these tasks is better connected with measured skill than those in use in software development contexts, i.e., company-assesses skill category, length of experience, self-assessed skill and confidence in knowing how to solve the tasks. These indicators had rank correlations between 0.14 and 0.29, and hit rates between 57% and 65%.
A possible explanation of why lower skill was connected with higher effort estimates for the smaller, but not for the larger tasks, is that lower skill in solving a task can both lead to lower skill in identifying the complexity and the simplicity of a task. In the situation with the simpler tasks, we propose, those with better skill were also better in identifying how easily the tasks could be solved at the time of estimation. This explanation suggests that we may be able to design estimation tasks even better at separating those with high and low task completion skill than in our study, when focusing on hidden simplicity, i.e., simplicity that the more skilled are more likely to identify than the less skilled ones.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, non-refereed |
Year of Publication | 2019 |
Conference Name | International symposium on forecasting (ISF) |
Talks, invited
Agile software development and benefits management: A perfect match
In PMI/Prosjekt Norge Workshop, Oslo, Norway, 2018.Status: Published
Agile software development and benefits management: A perfect match
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2018 |
Location of Talk | PMI/Prosjekt Norge Workshop, Oslo, Norway |
Alltid smidig når du går?
In Smidig-konferansen: Smidig i offentlig sektor, Oslo, Norway, 2018.Status: Published
Alltid smidig når du går?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2018 |
Location of Talk | Smidig-konferansen: Smidig i offentlig sektor, Oslo, Norway |
Cost and benefits of software development in a uncertain, skewed world
In Software Analytics and its Impact on Industry Delft Data Science Seminar, Delft, The Netherlands, 2018.Status: Published
Cost and benefits of software development in a uncertain, skewed world
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2018 |
Location of Talk | Software Analytics and its Impact on Industry Delft Data Science Seminar, Delft, The Netherlands |
Hva kjennetegner IT-prosjekter som lykkes?
In Seminar Forsvaret, Kolsås, Norway, 2018.Status: Published
Hva kjennetegner IT-prosjekter som lykkes?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2018 |
Location of Talk | Seminar Forsvaret, Kolsås, Norway |
Hvordan få tak i reell usikkerhet av kost-nytte i en skjev verden?
In Prosjekt Norges årlige prosjektlederkonferanse, Norway, 2018.Status: Published
Hvordan få tak i reell usikkerhet av kost-nytte i en skjev verden?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2018 |
Location of Talk | Prosjekt Norges årlige prosjektlederkonferanse, Norway |
Usikkerhetsanalyse
In HIT-seminar, Oslo, Norway, 2018.Status: Published
Usikkerhetsanalyse
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2018 |
Location of Talk | HIT-seminar, Oslo, Norway |
What makes software projects successful?
In Workshop with Government of Bangladesh Delegation (ECNEC), Oslo, Norway, 2018.Status: Published
What makes software projects successful?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talks, invited |
Year of Publication | 2018 |
Location of Talk | Workshop with Government of Bangladesh Delegation (ECNEC), Oslo, Norway |
Public outreach
Alltid smidig når du går! Men hva med oppstartsfasen og forretningssiden?
In Computerworld (Norge). IDG, 2018.Status: Published
Alltid smidig når du går! Men hva med oppstartsfasen og forretningssiden?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Public outreach |
Year of Publication | 2018 |
Secondary Title | Computerworld (Norge) |
Publisher | IDG |
De som vet lite vet også lite om hvor lite de vet
In Computerworld (Norge). IDG, 2018.Status: Published
De som vet lite vet også lite om hvor lite de vet
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Public outreach |
Year of Publication | 2018 |
Secondary Title | Computerworld (Norge) |
Publisher | IDG |
Digitalisering = produktivitetsvekst?
In Computerworld (Norge). IDG, 2018.Status: Published
Digitalisering = produktivitetsvekst?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Public outreach |
Year of Publication | 2018 |
Secondary Title | Computerworld (Norge) |
Publisher | IDG |
Hvordan velge å velge - og litt om duer
In Computerworld (Norge). IDG, 2018.Status: Published
Hvordan velge å velge - og litt om duer
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Public outreach |
Year of Publication | 2018 |
Secondary Title | Computerworld (Norge) |
Publisher | IDG |
Sannheten om uærlighet
In Computerworld (Norge). IDG, 2018.Status: Published
Sannheten om uærlighet
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Public outreach |
Year of Publication | 2018 |
Secondary Title | Computerworld (Norge) |
Publisher | IDG |
Proceedings, refereed
An Experimental Evaluation of a De-biasing Intervention for Professional Software Developers
In SAC 2018: Symposium on Applied Computing. New York: ACM, 2018.Status: Published
An Experimental Evaluation of a De-biasing Intervention for Professional Software Developers
CONTEXT: The role of expert judgement is essential in our quest to improve software project planning and execution. However, its accuracy is dependent on many factors, not least the avoidance of judgement biases, such as the anchoring bias, arising from being influenced by initial information, even when it's misleading or irrelevant. This strong effect is widely documented. OBJECTIVE: We aimed to replicate this anchoring bias using professionals and, novel in a software engineering context, explore de-biasing interventions through increasing knowledge and awareness of judgement biases. METHOD: We ran two series of experiments in company settings with a total of 410 software developers. Some developers took part in a workshop to heighten their awareness of a range of cognitive biases, including anchoring. Later, the anchoring bias was induced by presenting low or high productivity values, followed by the participants' estimates of their own project productivity. Our hypothesis was that the workshop would lead to reduced bias, i.e., work as a de-biasing intervention. RESULTS: The anchors had a large effect (robust Cohen's d=1.19) in influencing estimates. This was substantially reduced in those participants who attended the workshop (robust Cohen's d=0.72). The reduced bias related mainly to the high anchor. The de-biasing intervention also led to a threefold reduction in estimate variance. CONCLUSIONS: The impact of anchors upon judgement was substantial. Learning about judgement biases does appear capable of mitigating, although not removing, the anchoring bias. The positive effect of de-biasing through learning about biases suggests that it has value.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | SAC 2018: Symposium on Applied Computing |
Publisher | ACM |
Place Published | New York |
Do Agile Methods Work for Large Software Projects?
In 19th International Conference on Agile Software Development (XP 2018). Springer, 2018.Status: Published
Do Agile Methods Work for Large Software Projects?
Some people claim that agile methods do not scale well and are mainly useful for smaller software projects. Others claim that it is particularly in the context of large, complex software projects that the use of agile methods is likely to make the difference between success and failure. To find out more about whether or not agile methods work for larger projects, we conducted a questionnaire-based survey collecting information about 122 Norwegian software projects. Project success was measured as the combined performance of the project regarding delivered client benefits, cost control, and time control. We found that that projects using agile methods performed on average much better than those using non-agile methods for medium and large software projects, but not so much for smaller projects. This result gives support for the claim that agile methods are more rather than less successful compared to traditional methods when project size increases. There may consequently be more reasons to be concerned about how non-agile, rather than how agile methods, scale.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 19th International Conference on Agile Software Development (XP 2018) |
Pagination | 179-190 |
Publisher | Springer |
Looking back on previous estimation error as a method to improve the uncertainty assessment of benefits and costs of software development projects
In The 9th International Workshop on Empirical Software Engineering in Practice (IWESEP2018). IEEE, 2018.Status: Published
Looking back on previous estimation error as a method to improve the uncertainty assessment of benefits and costs of software development projects
Knowing the uncertainty of estimates of benefits and costs is useful when planning, budgeting and pricing projects. The traditional method for assessing such uncertainty is based on prediction intervals, e.g., asking for minimum and maximum values believed to be 90% likely to include the actual outcome. Studies report that the traditional method typically results in too narrow intervals and intervals that are too symmetric around the estimated most likely outcome when compared with the actual uncertainty of outcomes. We examine whether an uncertainty assessment method based on looking back on the previous estimation error of similar projects leads to wider and less symmetric prediction intervals. Sixty software professionals, with experience from estimating software project costs and benefits, were randomly divided into a group with a traditional or a group with a looking back-based uncertainty assessment method. We found that those using the looking back-based method had much wider prediction intervals for both costs and benefits. The software professionals of both groups provided uncertainty assessment values suggesting a left-skewed distribution for benefits and a right-skewed distribution for cost, but with much more skew among those using the looking back-based method. We argue that a looking back-based method is promising for improved realism in uncertainty assessment of benefits and costs of software development projects.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | The 9th International Workshop on Empirical Software Engineering in Practice (IWESEP2018) |
Pagination | 19-24 |
Publisher | IEEE |
Scope Creep or Embrace Change? A Survey of the Connections Between Requirement Changes, Use of Agile, and Software Project Success
In 12th International Conference on Project Management (ProMac). Tokyo, Japan: The Society of Project Management, 2018.Status: Published
Scope Creep or Embrace Change? A Survey of the Connections Between Requirement Changes, Use of Agile, and Software Project Success
Traditionally, a high degree of requirement change has been considered harmful for the success of software projects. Software professionals who use agile software development methods tend to view this topic differently. They tend to view requirement changes more as opportunities, which should be welcomed. Possibly, both views are correct but valid in different software development contexts. This paper aims at increasing the understanding of the connections between the degree of requirement change, choice of development method, and project success. Seventy software professionals were asked to provide information about their last software project. A higher degree of requirement changes, here defined as more than 30% of the requirements added, deleted, or changed during the project’s execution, was connected with a higher proportion of successful projects in an agile development context, but only when this included frequent deliveries to production. Our results consequently support that the agile claim of “embrace change” has merit, but only in agile contexts.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 12th International Conference on Project Management (ProMac) |
Pagination | 673-689 |
Publisher | The Society of Project Management |
Place Published | Tokyo, Japan |
Journal Article
Combining Data Analytics with Team Feedback to Improve the Estimation Process in Agile Software Development
Foundations of Computing and Decision Sciences 43, no. 4 (2018).Status: Published
Combining Data Analytics with Team Feedback to Improve the Estimation Process in Agile Software Development
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | Foundations of Computing and Decision Sciences |
Volume | 43 |
Issue | 4 |
Number | 305-334 |
Publisher | De Gruyter |
Talk, keynote
From myths and fashions to evidence-based software engineering
In ICSIE, Cairo, Egypt, 2018.Status: Published
From myths and fashions to evidence-based software engineering
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talk, keynote |
Year of Publication | 2018 |
Location of Talk | ICSIE, Cairo, Egypt |
Milliardinvesteringer i digitalisering. Hva gir det oss?
In Software 2018, DnD, Norway, 2018.Status: Published
Milliardinvesteringer i digitalisering. Hva gir det oss?
Privat og offentlig sektor investerer milliarder av kroner på å utvikle, videreutvikle og erstatte digitale løsninger med det formål å forbedre og effektivisere.
Hvor godt, og når, lykkes slike investeringene?
Bør vi roe ned eller heller øke tempoet i digitaliseringen?
Har investeringene total sett gitt mer effektivitet og bedre liv?
Det er overraskende vanskelig å kunne besvare disse spørsmålene godt. Noe vet vi imidlertid, og presentasjonen vil oppsummere kunnskap om dette basert på internasjonale evalueringer, egen og andres forskning på IT-utvikling, analyser av sammenheng mellom digitaliseringsinvesteringer og produktivitet, erfaringer fra gjennomganger i digitaliseringsrådet og andre kilder.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talk, keynote |
Year of Publication | 2018 |
Location of Talk | Software 2018, DnD, Norway |
Type of Talk | Keynote |
What can - and should - empirical software engineering learn from empirical studies in psychology?
In 12th International Symposium on
Empirical Software Engineering and Measurement, Oulu, Finland, 2018.Status: Published
What can - and should - empirical software engineering learn from empirical studies in psychology?
Software development is about people solving problems. They do this alone or in groups, for themselves or for others, with knowledge, experiences and biases. People solving problems have been empirically studied for more than two hundreds years in psychology, affecting not only treatment processes, but also economics, management, marketing, teaching and numerous other disciplines. What can we learn from how psychology researchers do their empirical studies and succeed in achieving useful results and affect practice? In this talk I will examine similarities and differences in use of empirical methods in psychology and software engineering and summarize this in what I argue has the potential of improving the quality and impact of our empirical software engineering studies.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talk, keynote |
Year of Publication | 2018 |
Location of Talk | 12th International Symposium on Empirical Software Engineering and Measurement, Oulu, Finland |
What makes software projects successful?
In ICSSE, Prague, Czech Republic, 2018.Status: Published
What makes software projects successful?
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talk, keynote |
Year of Publication | 2018 |
Location of Talk | ICSSE, Prague, Czech Republic |
When is agile better? How the use of agile and autonomous teams affect success differently in different contexts (and other results)
In A-teams (First International Workshop on Autonomous Agile Teams), XP-workshop, Porto, Portugal, 2018.Status: Published
When is agile better? How the use of agile and autonomous teams affect success differently in different contexts (and other results)
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Talk, keynote |
Year of Publication | 2018 |
Location of Talk | A-teams (First International Workshop on Autonomous Agile Teams), XP-workshop, Porto, Portugal |
Book
Time predictions: Understanding and avoiding unrealism in project planning and everyday life
In Simula SpringerBriefs on Computing. Switzerland: Springer, 2018.Status: Published
Time predictions: Understanding and avoiding unrealism in project planning and everyday life
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Book |
Year of Publication | 2018 |
Secondary Title | Simula SpringerBriefs on Computing |
Series Volume | 5 |
Number of Pages | 110 |
Publisher | Springer |
Place Published | Switzerland |
Proceedings, refereed
Connections between Contract Type, Project Size, Benefits Management, Agile Practices and Success of Software Development Projects
In FITAT 2017. Ulan Bator, Mongolia: Mongolia University, 2017.Status: Published
Connections between Contract Type, Project Size, Benefits Management, Agile Practices and Success of Software Development Projects
The study examines connections between the outcome of software development projects and contract type, project size, benefits management practices, and agile practices. The collected information relates to 146 software projects—half from 2007-2012 and the other half from 2015-2017. Binary logistic models built to explain successful and problematic software development projects revealed that the use of time and materials contracts, as opposed to fixed price contracts, was associated with fourteen times higher likelihood of project success. Other practices with a strong connection to project outcome included frequent delivery to production and benefit management during project execution. We found that the explanatory strength of a practice sometimes differs substantially in models of successful and in models of problematic projects, as well as over time.
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | FITAT 2017 |
Publisher | Mongolia University |
Place Published | Ulan Bator, Mongolia |
Software development contracts: The impact of the provider’s risk of financial loss on project success
In ICSE-workshop: CHASE. IEEE Press, 2017.Status: Published
Software development contracts: The impact of the provider’s risk of financial loss on project success
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | ICSE-workshop: CHASE |
Pagination | 30-35 |
Publisher | IEEE Press |
Working with Industry
In 5th IEEE/ACM International Workshop on Conducting Empirical Studies in Industry, CESI@ICSE 2017. IEEE Press, 2017.Status: Published
Working with Industry
The software engineering industry should be the laboratory of much, perhaps most, of the empirical software engineering research. Not only would this create a more realistic context and higher external validity of the empirical research, it would also ease the result transfer and make the results more convincing for the industry. Unfortunately, this is currently not the case. About 90% of software engineering experiments are, for example, conducted with students instead of software professionals as subjects. One reason for the lack of industry studies may be that an efficient and sustainable give-and-take-based collaboration between research and industry can be difficult to establish. The collaborations are frequently fragile, end before the research is completed, and lead to a waste of resources for both the researchers and the industrial partners. This paper presents stories and lessons learned from failed and successful research-industry collaborations. It has a focus on experience with the use of non-traditional collaboration types, such as payment to get industry participation in experiments, trade-based collaboration, lightweight collaborations at industry venues, and network-based collaborations. It is argued that empirical software engineering research should more often consider the use of alternative types of research–industry collaborations than those traditionally chosen.
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | 5th IEEE/ACM International Workshop on Conducting Empirical Studies in Industry, CESI@ICSE 2017 |
Pagination | 46-52 |
Publisher | IEEE Press |
DOI | 10.1109/CESI.2017.3 |
Journal Article
Direct and indirect connections between type of contract and software project outcome
International Journal of Project Management 35, no. 8 (2017).Status: Published
Direct and indirect connections between type of contract and software project outcome
This paper reports two empirical studies on how the use of different contract types affects the outcomes of software projects. The first study evaluates the effect of contract type on project failure using information from a large international dataset of small-scale, outsourced software projects and tasks. The second study proposes and tests how the use of contracts is connected with project outcome using information about Norwegian software projects with a public client. Both studies find that the use of fixed price contracts is connected with a higher risk of project failure compared to time and materials types of contracts. The results from the second study suggest that different project outcomes with different contract types is explained by differences in how the provider is selected, how the client is involved in the project, the use of agile practices and the use of benefit management during project execution.
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | International Journal of Project Management |
Volume | 35 |
Issue | 8 |
Number | 1573-1586 |
Publisher | Elsevier |
DOI | 10.1016/j.ijproman.2017.09.003 |
Guest editorial for special section on success and failure in software engineering
Empirical Software Engineering 22, no. 5 (2017).Status: Published
Guest editorial for special section on success and failure in software engineering
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | Empirical Software Engineering |
Volume | 22 |
Issue | 5 |
Number | 2281-2297 |
Publisher | Elsevier |
What Contributes to the Success of IT Projects? An Empirical Study of IT Projects in the Norwegian Public Sector
Journal of Software 12, no. 9 (2017): 751-758.Status: Published
What Contributes to the Success of IT Projects? An Empirical Study of IT Projects in the Norwegian Public Sector
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | Journal of Software |
Volume | 12 |
Issue | 9 |
Pagination | 751-758 |
Publisher | IAP |
Talks, invited
Empirical methods and evidence-based decisions in software engineering
In Seminar at Innsbruck University, Austria, 2017.Status: Published
Empirical methods and evidence-based decisions in software engineering
Afilliation | Software Engineering |
Project(s) | No Simula project |
Publication Type | Talks, invited |
Year of Publication | 2017 |
Location of Talk | Seminar at Innsbruck University, Austria |
Evaluering av digitalisering i offentlig sektor. Hvor gode er vi? Evaluerer vi det som er viktig?
In Evalueringskonferansen, Oslo, Norway, 2017.Status: Published
Evaluering av digitalisering i offentlig sektor. Hvor gode er vi? Evaluerer vi det som er viktig?
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Talks, invited |
Year of Publication | 2017 |
Location of Talk | Evalueringskonferansen, Oslo, Norway |
How to succeed and avoid failing with software projects
In Seminar "International Trends in Software Development", Kathmandu, Nepal, 2017.Status: Published
How to succeed and avoid failing with software projects
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Talks, invited |
Year of Publication | 2017 |
Location of Talk | Seminar "International Trends in Software Development", Kathmandu, Nepal |
Hva kjennetegner IT-prosjekter som lykkes? (Resultater fra SMIOS-prosjektet)
In Eleven presentations: 1) Oslo Kommune (Oslo), 2) Skatteetaten (Tønsberg), 3) SINTEF (Trondheim), 4) SPK (Oslo), 5) Sparebank 1 (Oslo), 6) Miles (Oslo), 7) Posten (Oslo), 8) Vinmonopolet (Oslo), 9) FHI (Oslo), 10) Knowit (Oslo), 11) Eika IT (Oslo) , Norway, 2017.Status: Published
Hva kjennetegner IT-prosjekter som lykkes? (Resultater fra SMIOS-prosjektet)
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Talks, invited |
Year of Publication | 2017 |
Location of Talk | Eleven presentations: 1) Oslo Kommune (Oslo), 2) Skatteetaten (Tønsberg), 3) SINTEF (Trondheim), 4) SPK (Oslo), 5) Sparebank 1 (Oslo), 6) Miles (Oslo), 7) Posten (Oslo), 8) Vinmonopolet (Oslo), 9) FHI (Oslo), 10) Knowit (Oslo), 11) Eika IT (Oslo) , Norway |
Nyttestyring og viktigheten av den gode kunde
In Prosjekt 2017 (Prosjekt Norges årlige prosjektlederkonferanse), Oslo, Norway, 2017.Status: Published
Nyttestyring og viktigheten av den gode kunde
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Talks, invited |
Year of Publication | 2017 |
Location of Talk | Prosjekt 2017 (Prosjekt Norges årlige prosjektlederkonferanse), Oslo, Norway |
Usikkerhetsanalyse med avhengigheter i en høyreskjev verden
In HIT-seminar, Oslo, Norway, 2017.Status: Published
Usikkerhetsanalyse med avhengigheter i en høyreskjev verden
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Talks, invited |
Year of Publication | 2017 |
Location of Talk | HIT-seminar, Oslo, Norway |
Public outreach
Evidensbasert praksis og misvisende nyheter
Article in Computerworld, 2017.Status: Published
Evidensbasert praksis og misvisende nyheter
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Public outreach |
Year of Publication | 2017 |
Publisher | Article in Computerworld |
Femti, feit og ferdig, eller stadig mer produktiv?
Article in Computerworld, 2017.Status: Published
Femti, feit og ferdig, eller stadig mer produktiv?
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Public outreach |
Year of Publication | 2017 |
Publisher | Article in Computerworld |
Frederick Winslow Taylor: Prosessforbedring industrien
Article in Computerworld, 2017.Status: Published
Frederick Winslow Taylor: Prosessforbedring industrien
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Public outreach |
Year of Publication | 2017 |
Publisher | Article in Computerworld |
Husk å glemme
Article in Computerworld, 2017.Status: Published
Husk å glemme
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Public outreach |
Year of Publication | 2017 |
Publisher | Article in Computerworld |
Kravendringer i IT-prosjekter: trussel eller mulighet?
Article in Computerworld, 2017.Status: Published
Kravendringer i IT-prosjekter: trussel eller mulighet?
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Public outreach |
Year of Publication | 2017 |
Publisher | Article in Computerworld |
Når regnereglene du lærte på skolen gir feil svar
Article in Computerworld, 2017.Status: Published
Når regnereglene du lærte på skolen gir feil svar
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Public outreach |
Year of Publication | 2017 |
Publisher | Article in Computerworld |
Poster
What contributes to the success of IT projects?: success factors, challenges and lessons learned from an empirical study of software projects in the Norwegian public sector
In ICSE. Buenos Aires, Argentina: IEEE Press, 2017.Status: Published
What contributes to the success of IT projects?: success factors, challenges and lessons learned from an empirical study of software projects in the Norwegian public sector
Context. Each year the public sector invests large amounts of money in the development and modifications of their software systems. These investments are not always successful and many public sector software projects fail to deliver the expected benefits. Goal. This study aims at reducing the waste of resources on failed software projects through better understanding of the success factors and challenges. Method.Thirty-five completed software projects in 11 organizations in the public sector of Norway were analyzed. For each project, representatives from the project owners, project management and the user organization were interviewed. Results. Small and large software projects reported different challenges, especially related to project priority. Taking advantage of agile practices such as flexible scope and frequent delivery increased the success rate of the projects. Projects with time and material contracts and involved clients during execution were more successful than other projects. The respondents experienced that extensive involvement and good competence of the client, high priority of the project, good dialogue between client and provider and appliance of agile practices were main success factors. Main challenges were related to technical issues, project planning and management, transition of the product to the user organization, involvement and competence of the client, and benefit management. Conclusions. Success factors tend to focus on human factors, e.g., involvement, competence and collaboration. Challenges focus on human factors as well as issues of technical nature. Both aspects need to be addressed to enable successful and avoid failed software projects. Competence, client involvement and benefit management are among factors that the public sector should focus on for realizing client benefits.
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Poster |
Year of Publication | 2017 |
Secondary Title | ICSE |
Publisher | IEEE Press |
Place Published | Buenos Aires, Argentina |
What it actually is a working hour?
15th European Congress of Psychology, Amsterdam, the Netherlands, 2017.Status: Published
What it actually is a working hour?
Afilliation | Software Engineering |
Project(s) | No Simula project |
Publication Type | Poster |
Year of Publication | 2017 |
Place Published | 15th European Congress of Psychology, Amsterdam, the Netherlands |
Talk, keynote
What makes software projects successful?
In ICSIE, Singapore. Singapore, 2017.Status: Published
What makes software projects successful?
Numerous research studies and consultancy reports make claims about how often, or rather how seldom, software projects are successful, why so many of them fail, and how to succeed more often. These studies and reports have reported very much the same success and failure factors and the same advices since the 1960s. If we already know how to make a successful software project, why is the proportion of failed software projects about the same as earlier? Are software professionals ignorant of the published knowledge or are there other reasons? One reason may be that previous studies have had very little focus on the most important success dimension, i.e., client benefits, contain very little practical advice on how to succeed, and have not managed to include the context-dependency and complexity of the connections between process choices and outcome. In this keynote I present recent attempts to better define and operationalize project success and better analyse and describe the context-dependent and probabilistic network of connections between essential choices made early and the outcome of software projects. The hope is that this may inspire other researchers to emphasise research on software project management that results in more context-aware results of practical use for software professionals.
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Talk, keynote |
Year of Publication | 2017 |
Location of Talk | ICSIE, Singapore |
Place Published | Singapore |
Working with industry to conduct empirical software engineering research: Patterns of successful and failed collaborations
In ICSE-workshop: CESI, Buenos Aires, Argentina. IEEE Press, 2017.Status: Published
Working with industry to conduct empirical software engineering research: Patterns of successful and failed collaborations
The industry should be the laboratory of a large part of empirical SE research. Not only does this create a more realistic context for the empirical research, it also eases the result transfer and makes them more convincing for the industry. Unfortunately, this is currently not the case. About 90% of SE experiments are, for example, conducted with students instead of software professionals as subjects. One reason for the lack of industry studies is that a good and sustainable “give-and-take”-based collaboration with industry can be difficult to establish. The collaborations are frequently fragile, end before the research is completed, and lead to a waste of resources for both the researchers and the industrial partners. This keynote presents experience from different, failed and successful, academia-industry collaborations with the goal of identifying approaches that are more likely to give the researchers insight and opportunities to evaluate methods and tools, and the industry benefits that make the collaborations worthwhile for them. A special focus will be on experiences with the use of “lightweight” collaborations, collaboration where the researchers use part of their budget to create monetary incentives for the industry to collaborate, and the use of evidence-based SE as a supporting tool for industry collaborations.
Afilliation | Software Engineering |
Project(s) | SMIOS: Successful ICT solutions in the public sector |
Publication Type | Talk, keynote |
Year of Publication | 2017 |
Location of Talk | ICSE-workshop: CESI, Buenos Aires, Argentina |
Publisher | IEEE Press |
Journal Article
A Survey on the Characteristics of Projects with Success in Delivering Client Benefits
Information and Software Technology 78 (2016): 83-94.Status: Published
A Survey on the Characteristics of Projects with Success in Delivering Client Benefits
Context: A large waste of resources in software development projects currently results from being unable to produce client benefits.
Objective: The main objective is to better understand the characteristics of successful software projects and contribute to software projects that are more likely to produce the planned client benefits.
Method: We asked 63 Norwegian software professionals, representing both the client and the provider role, to report information about their last completed project. In a follow-up survey with 64 Norwegian software professionals, we addressed selected findings from the first survey.
Results: The analysis of the project information showed the following: i) The project management triangle criteria of being on time, on budget, and having the specified functionality are poor correlates of the essential success dimension client benefits. ii) Benefit management planning before the project started and benefit management activities during project execution were connected with success in delivering client benefits. iii) Fixed-price projects and projects in which the selection of providers had a strong focus on low price were less successful in delivering project benefits than other projects. iv) Agile projects were in general more successful than other projects, but agile projects without flexible scope to reflect changed user needs and learning, or without frequent delivery to the client, had less than average success in delivering client benefits.
Conclusions: The software projects that were successful in delivering client benefits differed from the less successful ones in several ways. In particular, they applied benefit management practices during project execution, they avoided fixed-price contracts, they had less focus on low price in the selection of providers, and they applied the core agile practices - frequent delivery to the client and scope flexibility.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | Information and Software Technology |
Volume | 78 |
Pagination | 83-94 |
Publisher | Elsevier |
Better Selection of Software Providers Through Trialsourcing
IEEE Software 33, no. Sept-Oct (2016): 48-53.Status: Published
Better Selection of Software Providers Through Trialsourcing
In this article we show that differences between software providers in terms of productivity and quality can be very large and that traditional means of evaluating software providers, such as CVs and evaluations by reference clients, fail to separate the competent from the incompetent ones. We argue and motivate through analyses of empirical data that software clients would benefit from using work sample-based tests of software development providers' performance (trialsourcing) to guide their selection of providers. We discuss potential challenges of trialsourcing and provide recommendations on what to consider when designing proper trialsourcing processes.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | IEEE Software |
Volume | 33 |
Issue | Sept-Oct |
Pagination | 48-53 |
Publisher | Elsevier |
Cultural characteristics and their connection to increased risk of software project failure
Journal of Software 11 (2016): 606-614.Status: Published
Cultural characteristics and their connection to increased risk of software project failure
Abstract—Offshoring software projects to low cost countries has the potential of reducing the cost of software development and increasing the availability of competent development resources. It has, however, also been documented that it can increase the risk of project failure. In particular, the cultural differences between software client and provider countries are believed to increase the risk of project failure. The main goal of our study is to empirically examine how cultural characteristics of and cultural differences between software client and provider countries are connected with an increased risk of project failure. We analyze a large data set of small software projects with providers and clients from various countries. For each provider and client country, and each combination of provider and client country, we calculate the project failure rate. We use Hofstede’s culture dimensions power distance, individualism, masculinity and uncertainty avoidance together with Hall’s concept of high and low-context cultures (communication style) to measure cultural characteristics and differences. We found a statistically significant increase in failure rate with increased difference in the provider and client country’s communication style, but no connection between cultural differences and project failure rate using Hofstede’s cultural characteristics. Provider countries, and in particular, client countries with a low-context culture had lower proportions of project failures than countries with high-context cultures. A decrease in power distance and increase in individualism were both connected with a decrease in project failure rate. In total, the cultural differences between countries seem to matter less than cultural characteristics of the countries. In particular, some of the cultural characteristics of the client country were strongly connected with project failure rates. Software offshoring clients may benefit from an awareness of the importance of their role in avoiding project failures and consider adopting the cultural characteristics of the most successful client offshoring countries. They may also benefit from selecting software providers from low-context cultures.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | Journal of Software |
Volume | 11 |
Number | 6 |
Pagination | 606-614 |
Publisher | International Academy Publisher |
Incorrect Results in Software Engineering Experiments: How to Improve Research Practices
Journal of Systems and Software 117 (2016): 274-281.Status: Published
Incorrect Results in Software Engineering Experiments: How to Improve Research Practices
Context: The trustworthiness of research results is a growing concern in many empirical disciplines. Aim: The goals of this paper are to assess how much the trustworthiness of results reported in software engineering experiments is affected by researcher and publication bias and to suggest improved research practices. Method: First, we conducted a small-scale survey to document the presence of researcher and publication biases in software engineering experiments. Then, we built a model that estimates the proportion of correct results for different levels of researcher and publication bias. A review of 150 randomly selected software engineering experiments published in the period 2002-2012 was conducted to provide input to the model. Results: The survey indicates that researcher and publication bias is quite common. This finding is supported by the observation that the actual proportion of statistically significant results reported in the reviewed papers was about twice as high as the one expected assuming no researcher and publication bias. Our models suggest a high proportion of incorrect results even with quite conservative assumptions. Conclusion: Research practices must improve to increase the trustworthiness of software engineering experiments. A key to this improvement is to avoid conducting studies with unsatisfactory low statistical power.
Afilliation | Software Engineering |
Project(s) | No Simula project |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | Journal of Systems and Software |
Volume | 117 |
Pagination | 274-281 |
Publisher | Elsevier |
Numerical anchors and their strong effects on software development effort estimates
Journal of Systems and Software 116 (2016): 49-56.Status: Published
Numerical anchors and their strong effects on software development effort estimates
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | Journal of Systems and Software |
Volume | 116 |
Pagination | 49-56 |
Publisher | Elsevier |
Unit effects in software project effort estimation: Work-hours gives lower effort estimates than workdays
Journal of Systems and Software 117 (2016): 274-281.Status: Published
Unit effects in software project effort estimation: Work-hours gives lower effort estimates than workdays
Software development effort estimates are typically expert judgment-based and too low to reflect the actual use of effort. Our goal is to understand how the choice of effort unit affects expert judgement-based effort estimates, and to use this knowledge to increase the realism of effort estimates. We conducted two experiments where the software professionals were randomly instructed to estimate the effort of the same projects in work-hours or in workdays. In both experiment, the software professionals estimating in work-hours had much lower estimates (on average 33%–59% lower) than those estimating in workdays. We argue that the unitosity effect—i.e., that we tend to infer information about the quantity from the choice of unit—is the main explanation for the large difference in effort estimates. A practical implication of the unit effect is that, in contexts where there is a tendency toward effort under estimation, the instruction to estimate in higher granularity effort units, such as workdays instead of work-hours, is likely to lead to more realistic effort estimates.
Afilliation | Software Engineering |
Project(s) | No Simula project |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | Journal of Systems and Software |
Volume | 117 |
Pagination | 274-281 |
Publisher | Elsevier |
Public outreach
Enkelt er bra, uforståelig er bedre?
In Article in Computerworld Norway, 2016.Status: Published
Enkelt er bra, uforståelig er bedre?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2016 |
Secondary Title | Article in Computerworld Norway |
Date Published | 12/2016 |
Klarer vi å kontrollere kunstig superintelligens?
In Computerworld, 2016.Status: Published
Klarer vi å kontrollere kunstig superintelligens?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2016 |
Secondary Title | Computerworld |
Kulturelle egenskaper og offshoring av IT-utvikling
Article in Computerworld, 2016.Status: Published
Kulturelle egenskaper og offshoring av IT-utvikling
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2016 |
Publisher | Article in Computerworld |
Kunde eller leverandørstyrt bemanning av IT-prosjekter
Article in Computerworld, 2016.Status: Published
Kunde eller leverandørstyrt bemanning av IT-prosjekter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2016 |
Publisher | Article in Computerworld |
Suksess som indikator for fremtidige skuffelser
Article in Computerworld, 2016.Status: Published
Suksess som indikator for fremtidige skuffelser
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2016 |
Publisher | Article in Computerworld |
Talks, invited
Estimering av IT-utvikling
In KnowIT, Estimation seminar, Oslo, 2016.Status: Published
Estimering av IT-utvikling
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2016 |
Location of Talk | KnowIT, Estimation seminar, Oslo |
Evidence-based software engineering: A framework for collaboration between researchers and software professionals
In Amsterdam. Seminar at Centrum voor Wiskunde en Informatica (CWI), 2016.Status: Published
Evidence-based software engineering: A framework for collaboration between researchers and software professionals
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2016 |
Location of Talk | Amsterdam |
Publisher | Seminar at Centrum voor Wiskunde en Informatica (CWI) |
How to identify risky IT projects and avoid them turning into black swans
In Ernst & Young: Nordic Advisory Learning Weekend, Riga, 2016.Status: Published
How to identify risky IT projects and avoid them turning into black swans
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2016 |
Location of Talk | Ernst & Young: Nordic Advisory Learning Weekend, Riga |
Hva kjennetegner IT-prosjekter som lykkes
In Oslo. Seminars/conferences organised by LO-IT, Econa and Tekna (Det digitale skiftet), 2016.Status: Published
Hva kjennetegner IT-prosjekter som lykkes
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2016 |
Location of Talk | Oslo |
Publisher | Seminars/conferences organised by LO-IT, Econa and Tekna (Det digitale skiftet) |
Hva kjennetegner IT-prosjekter som lykkes (Resultater fra SMIOS-prosjektet)
In Oslo. Presentations at: HIT-seminar, NSB-seminar, nav-seminar, spk-seminar, Lyse-seminar, Thales-days, Digitaliseringsrådet, Finansdepartementet, KMD, transportkomiteen, 2016.Status: Published
Hva kjennetegner IT-prosjekter som lykkes (Resultater fra SMIOS-prosjektet)
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2016 |
Location of Talk | Oslo |
Publisher | Presentations at: HIT-seminar, NSB-seminar, nav-seminar, spk-seminar, Lyse-seminar, Thales-days, Digitaliseringsrådet, Finansdepartementet, KMD, transportkomiteen |
Hva skiller IT-prosjekter som lykkes fra de som mislykkes? Hva vil det si å lykkes?
In Seminar at Fornebu Consulting, 2016.Status: Published
Hva skiller IT-prosjekter som lykkes fra de som mislykkes? Hva vil det si å lykkes?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2016 |
Location of Talk | Seminar at Fornebu Consulting |
Suksess med offentlige IT-prosjekter. Hva er det og hvordan får vi det til?
In Trondheim. Hemit-konferansen, 2016.Status: Published
Suksess med offentlige IT-prosjekter. Hva er det og hvordan får vi det til?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2016 |
Location of Talk | Trondheim |
Publisher | Hemit-konferansen |
Vurderinger og beslutninger. Hvor rasjonelle er vi?
In DND's Software Conference, 2016.Status: Published
Vurderinger og beslutninger. Hvor rasjonelle er vi?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2016 |
Location of Talk | DND's Software Conference |
Talk, keynote
Fixed Price without Fixed Specifications
In Gøteborg. Vol. Keynote at REFSQ (Requirement Engineering Foundation for Software Engieering), 2016.Status: Published
Fixed Price without Fixed Specifications
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2016 |
Location of Talk | Gøteborg |
Volume | Keynote at REFSQ (Requirement Engineering Foundation for Software Engieering) |
Hva skiller IT-prosjekter som lykkes fra de som mislykkes? Hva vil det si å lykkes?
In Computas brukerforum, 2016.Status: Published
Hva skiller IT-prosjekter som lykkes fra de som mislykkes? Hva vil det si å lykkes?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2016 |
Location of Talk | Computas brukerforum |
Judgment and decision-making in software engineering: How rational are we?
In Tokyo, Japan. ICSIE, 2016.Status: Published
Judgment and decision-making in software engineering: How rational are we?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2016 |
Location of Talk | Tokyo, Japan |
Publisher | ICSIE |
Successful IT projects: The role of the contract
In Zürich. Seminar organized by the Swiss Association for Quality, 2016.Status: Published
Successful IT projects: The role of the contract
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2016 |
Location of Talk | Zürich |
Publisher | Seminar organized by the Swiss Association for Quality |
Proceedings, refereed
The Use of Precision of Software Development Effort Estimates to Communicate Uncertainty
In Software Quality Days. The Future of Systems-and Software Development. Springer International Publishing, 2016.Status: Published
The Use of Precision of Software Development Effort Estimates to Communicate Uncertainty
The precision of estimates may be applied to communicate the
uncertainty of required software development effort. The effort estimates 1000
and 975 work-hours, for example, communicate different levels of expected
estimation accuracy. Through observational and experimental studies we found
that software professionals (i) sometimes, but not in the majority of the
examined projects, used estimate precision to convey effort uncertainty, (ii)
tended to interpret overly precise, inaccurate effort estimates as indicating low
developer competence and low trustworthiness of the estimates, while too
narrow effort prediction intervals had the opposite effect. This difference
remained even when the actual effort was known to be outside the narrow effort
prediction interval. We identified several challenges related to the use of the
precision of single value estimates to communicate effort uncertainty and
recommend that software professionals use effort prediction intervals, and not
the preciseness of single value estimates, to communicate effort uncertainty.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Software Quality Days. The Future of Systems-and Software Development |
Pagination | 156-168 |
Publisher | Springer International Publishing |
Talks, invited
Å (mis)lykkes med IT-prosjekter
In Presentation at an IKT Norge-seminar, 2015.Status: Published
Å (mis)lykkes med IT-prosjekter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Presentation at an IKT Norge-seminar |
Den gode kunde: Kompetanse, involvering og kultur
In HIT-seminar, Oslo, 2015.Status: Published
Den gode kunde: Kompetanse, involvering og kultur
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | HIT-seminar, Oslo |
Empirical Methods and Evidence-Based Decisions in Software Engineering
In Oulu, Finland, 2015.Status: Published
Empirical Methods and Evidence-Based Decisions in Software Engineering
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Oulu, Finland |
Type of Talk | Seminar at the Software Engineering Department |
Estimering av IT-prosjekter: Hvordan bli bedre til å treffe med estimater, planer og budsjetter
In Politiet, Oslo, 2015.Status: Published
Estimering av IT-prosjekter: Hvordan bli bedre til å treffe med estimater, planer og budsjetter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Politiet, Oslo |
From fashion-based to evidence-based software engineering
In Teleplan, Oslo & Wroclaw, Poland, 2015.Status: Published
From fashion-based to evidence-based software engineering
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Teleplan, Oslo & Wroclaw, Poland |
From myths and fashions to evidence-based software engineering
In Stockholm, SICS seminar, 2015.Status: Published
From myths and fashions to evidence-based software engineering
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Stockholm, SICS seminar |
Type of Talk | Seminar talk at software engineering department |
Hva skal til for å lykes i IT-prosjekter? Hvor mye og hvordan kan man lære av andres suksesser og fiaskoer?
In Høgskolen i Hamar, 2015.Status: Published
Hva skal til for å lykes i IT-prosjekter? Hvor mye og hvordan kan man lære av andres suksesser og fiaskoer?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Høgskolen i Hamar |
Hva skal til for å lykkes i IT-prosjekter? Hvor mye og hvordan kan man lære av andres suksesser og fiaskoer?
In DnD's Software Conference, Oslo, 2015.Status: Published
Hva skal til for å lykkes i IT-prosjekter? Hvor mye og hvordan kan man lære av andres suksesser og fiaskoer?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | DnD's Software Conference, Oslo |
Type of Talk | DnD Software Conference |
Hvilke IT-prosjekter lykkes best
In Trondheim, NOKIOS, 2015.Status: Published
Hvilke IT-prosjekter lykkes best
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Trondheim, NOKIOS |
Hvilken betydning har kontrakten for suksess i IT-prosjekter?
In Presentation at a DnD seminar, 2015.Status: Published
Hvilken betydning har kontrakten for suksess i IT-prosjekter?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Presentation at a DnD seminar |
Ingen flere store offentlige IT-prosjekter?
In Presentation at Difi seminar, 2015.Status: Published
Ingen flere store offentlige IT-prosjekter?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Presentation at Difi seminar |
Offshoring av IT-utvikling: Hva er viktig for å lykkes
In Digin-seminar, Kristiansand, 2015.Status: Published
Offshoring av IT-utvikling: Hva er viktig for å lykkes
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Digin-seminar, Kristiansand |
Software development effort estimation: How to improve it
In Deerwalk seminar, Kathmandu, Nepal, 2015.Status: Published
Software development effort estimation: How to improve it
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Deerwalk seminar, Kathmandu, Nepal |
Suksess og fiasko i offentlige IKT-prosjekter
In Presentation at a seminar organized by the Ministry of Local Government and Modernisation, 2015.Status: Published
Suksess og fiasko i offentlige IKT-prosjekter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Presentation at a seminar organized by the Ministry of Local Government and Modernisation |
The world is skewed
In CREST-seminar, London. CREST, 2015.Status: Published
The world is skewed
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | CREST-seminar, London |
Publisher | CREST |
Talks, contributed
Are most published research findings in empirical software engineering wrong or with exaggerated effect sizes? How to improve?
In ISERN-meeting, Beijing, China, 2015.Status: Published
Are most published research findings in empirical software engineering wrong or with exaggerated effect sizes? How to improve?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2015 |
Location of Talk | ISERN-meeting, Beijing, China |
Proceedings, refereed
Believing is Seeing: Confirmation Bias Studies in Software Engineering
In 41st Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Madeira. Vol. IEEE, 2015.Status: Published
Believing is Seeing: Confirmation Bias Studies in Software Engineering
Abstract—Confirmation bias is the human tendency to search for, collect, interpret, analyse, or recall information in a way that confirms one’s prior beliefs or preferences. In this paper, we review previous research and demonstrate confirmation bias and its effect in two software engineering contexts. The first study documents that managers bias their interpretation of randomly generated project data towards confirmation of their preferred contract type. The second study reports from an analysis of the results of 35 published comparisons of regression and analogy-based cost estimation models. Twenty of these comparisons evaluate the performance of a self-developed analogy-based estimation model relative to a regression-based model and may be biased towards finding evidence confirming a better accuracy of their own model. A statistical meta-analysis of all 35 comparisons showed that the analogy-based models were more accurate than the regression-based models, whereas removing the 20 comparisons where the researchers evaluated their own models gave the opposite result. Our results support the idea that it is important to consider the presence and degree of confirmation bias when collecting information about topics where there are prior preferences, beliefs, or vested interests to avoid misleading conclusions.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | 41st Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Madeira |
Volume | |
Pagination | 92-95 |
Publisher | IEEE |
The Effect of the Time Unit on Software Development Effort Estimates
In 9th International Conference on Software, Knowledge, Information Management & Applications (SKIMA), Kathmandu, Nepal. IEEE, 2015.Status: Published
The Effect of the Time Unit on Software Development Effort Estimates
Estimates of software development effort are frequently inaccurate and over-optimistic. In this paper we describe how changes in the granularity of the unit of estimation, e.g., work-days instead of work-hours, affects the effort estimates. We describe four psychological mechanisms, how they interact and discuss the expected total effect of higher granularity units on effort estimates. We argue that the mechanisms in general imply that higher granularity effort units will result in higher effort estimates, e.g., that estimating software development work in work-days or weeks will lead to higher estimates than when estimating in work-hours. A possible implication of this predicted effect is that, in contexts where there is a tendency towards under-estimation, estimation in work-days or weeks instead of work-hours leads to more realistic estimates.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | 9th International Conference on Software, Knowledge, Information Management & Applications (SKIMA), Kathmandu, Nepal |
Pagination | 178-182 |
Publisher | IEEE |
Public outreach
Er tiden for de store IT-anskaffelsene nå over?
In Article in Computerworld Norge, 2015.Status: Published
Er tiden for de store IT-anskaffelsene nå over?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2015 |
Secondary Title | Article in Computerworld Norge |
Med høy presisjon
In Article in Computerworld Norge, 2015.Status: Published
Med høy presisjon
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2015 |
Secondary Title | Article in Computerworld Norge |
Sitte i celle eller i åpent kontorlandskap?
In Article in Computerworld Norge, 2015.Status: Published
Sitte i celle eller i åpent kontorlandskap?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2015 |
Secondary Title | Article in Computerworld Norge |
Tallverdieffekten: Vi rekker mer på tolv måneder enn ett år
In Article in Computerworld Norge, 2015.Status: Published
Tallverdieffekten: Vi rekker mer på tolv måneder enn ett år
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2015 |
Secondary Title | Article in Computerworld Norge |
Trenger vi mer lokal eksperimentering?
In Article in Computerworld Norway, 2015.Status: Published
Trenger vi mer lokal eksperimentering?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2015 |
Secondary Title | Article in Computerworld Norway |
Vellykkede IT-prosjekter krever gode kunder
In Article in Computerworld Norway, 2015.Status: Published
Vellykkede IT-prosjekter krever gode kunder
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2015 |
Secondary Title | Article in Computerworld Norway |
Verden er stort sett ikke normal, men høyreskjev
In Article in Computerworld Norge, 2015.Status: Published
Verden er stort sett ikke normal, men høyreskjev
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2015 |
Secondary Title | Article in Computerworld Norge |
Talk, keynote
Hva vet vi om IT-bransjens evne til å levere nyttige løsninger med god kvalitet?
In Keynote at Computerworld Norges Round Table seminar, 2015.Status: Published
Hva vet vi om IT-bransjens evne til å levere nyttige løsninger med god kvalitet?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2015 |
Location of Talk | Keynote at Computerworld Norges Round Table seminar |
The use and misuse of statistics in studies on software development. Things you should know about statistics that you didn't learn in school
In Keynote at ICSIE, Dubai, UAE, 2015.Status: Published
The use and misuse of statistics in studies on software development. Things you should know about statistics that you didn't learn in school
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2015 |
Location of Talk | Keynote at ICSIE, Dubai, UAE |
Proceedings, refereed
Communication of Software Cost Estimates
In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering. : ACM, 2014.Status: Published
Communication of Software Cost Estimates
The meaning of an effort or cost estimate should be understood and communicated consistently and clearly to avoid planning and budgeting mistakes. Results from two studies, one of 42 software companies and one of 423 individual software developers, suggest that this is far from being the case. In both studies we found a large variety in what was meant by an effort estimate and that the meaning was frequently not communicated. To improve the planning and budgeting of software projects we recommend that the meaning of effort estimates is understood and communicated using a probability-based terminology.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering |
Pagination | 241-245 |
Publisher | ACM |
Place Published |
Fallacies and Biases When Adding Effort Estimates
In Proceedings at Euromicro/SEEA. : IEEE, 2014.Status: Published
Fallacies and Biases When Adding Effort Estimates
Software professionals do not always clarify what they mean by their effort estimates. Knowing what is meant by an estimate is, however, essential when adding individual effort estimates from a work breakdown structure to find the estimated total effort. Adding the most likely instead of the mean effort of a set of cost elements may result in substantial underestimation of the total effort. In a survey of forty-four software companies we found only two companies that clarified the meaning of their estimates and had a proper method for adding these estimates. The other companies typically added single point estimates without clarifying what they added or with types of estimates likely to give too low estimates of the total effort. We examine the effect of improper addition of estimates and find, for the studied contexts, that summing the most likely effort estimates would lead to a substantial under-estimation of the most likely total effort. We also find that the use of the PERT-method, which provides a proper statistical basis for adding effort estimates and is used by many software companies, is likely to underestimate the effort in software development contexts. This is caused by, we argue and illustrate with empirical data, people's tendency towards providing too narrow minimum and maximum effort intervals. We outline a method that, we believe, better ensures that proper estimates of the total effort are produced.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings at Euromicro/SEEA |
Pagination | 277-284 |
Publisher | IEEE |
Place Published |
The Ignorance of Confidence Levels in Minimum-Maximum Software Development Effort Intervals
In ICSIE, Dubai. Vol. Lecture Notes on Software Engineering, 2(4). : , 2014.Status: Published
The Ignorance of Confidence Levels in Minimum-Maximum Software Development Effort Intervals
Software professionals are frequently asked to provide minimum-maximum effort intervals for a given confidence level. They may for example be asked to provide minimum-maximum intervals where it is 90% likely to include the actual use of effort. If their response is the interval from 800 to 1200 work-hours this should correspond to that it is 90% likely that the actual effort will be more than 800 and less than 1200 work-hours. This effort interval information is, amongst others, used in the planning and budgeting of software projects. In this paper we show that software professionals tend to ignore the confidence levels connected with the minimum-maximum effort intervals. As a consequence, the meaning of minimum-maximum effort interval is unclear and the use of such intervals questionable. The experiment used to document the ignorance of the confidence level is based on requesting one group of software developers to be 98% confident, and another group to be 80% confident when providing their effort intervals. In spite of a difference in confidence levels that should generate quite difference effort intervals, the actual intervals were almost the same. This finding challenge commonly recommended effort uncertainty assessment practices, e.g., those implemented in the PERT method, which are based on the assumption that software professionals are able to provide minimum-maximum effort that reflect the stated confidence levels. The finding does also challenge the explanation typically given for too narrow confidence intervals, i.e., that people are over-confident. Instead, we propose that a more likely explanation is that people ignore the confidence level when setting the minimum and maximum values.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | ICSIE, Dubai |
Volume | Lecture Notes on Software Engineering, 2(4) |
Pagination | 327-340 |
Publisher | |
Place Published |
Talks, invited
Den Gode Kunden - Viktigere Enn Du Kanskje Tror
In Difi-seminar + NSB seminar, 2014.Status: Published
Den Gode Kunden - Viktigere Enn Du Kanskje Tror
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | Difi-seminar + NSB seminar |
Erfaringer fra it-prosjekter. Fokus på kompetanse og kvalitet, som basis for å levere et forventet resultat
In Invited talk: Computerworld Round Table, 2014.Status: Published
Erfaringer fra it-prosjekter. Fokus på kompetanse og kvalitet, som basis for å levere et forventet resultat
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | Invited talk: Computerworld Round Table |
Estimering av IT-prosjekter: Hva vet vi virker?
In Presentation at Thales seminar, 2014.Status: Published
Estimering av IT-prosjekter: Hva vet vi virker?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | Presentation at Thales seminar |
Estimering av IT-prosjekter: Hvordan bli bedre til å treffe med estimater, planer, budsjetter og pris?
In Presentation at Teleplan-seminar, 2014.Status: Published
Estimering av IT-prosjekter: Hvordan bli bedre til å treffe med estimater, planer, budsjetter og pris?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | Presentation at Teleplan-seminar |
Fra myter og moter til evidensbasert IT-utvikling
In Presentation at TEKNA-seminar, 2014.Status: Published
Fra myter og moter til evidensbasert IT-utvikling
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | Presentation at TEKNA-seminar |
From Myths, Fashion, Over-Simplifications and Non-Validated Claims to Evidence-Based Software Engineering
In Distinguished Lecturers Series Leon the Mathematician, University of Thessaloniki, 2014.Status: Published
From Myths, Fashion, Over-Simplifications and Non-Validated Claims to Evidence-Based Software Engineering
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | Distinguished Lecturers Series Leon the Mathematician, University of Thessaloniki |
Hvordan anslå usikkerhet til estimater
In Presentation at JavaZone (repeated for NTNU-students as JavaZone Academy presentation). : , 2014.Status: Published
Hvordan anslå usikkerhet til estimater
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | Presentation at JavaZone (repeated for NTNU-students as JavaZone Academy presentation) |
Publisher | |
Place Published |
Hvordan Unngå Skuffelser i IT-Prosjekter og Ellers i Livet
In Miles seminar, 2014.Status: Published
Hvordan Unngå Skuffelser i IT-Prosjekter og Ellers i Livet
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | Miles seminar |
Ten years with Evidence-Based Software Engineering. What is it? Has it had any impact? What’s next?
In Hong Kong. Hong Kong: Impact paper award talk at: FSE, 2014.Status: Published
Ten years with Evidence-Based Software Engineering. What is it? Has it had any impact? What’s next?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | Hong Kong |
Publisher | Impact paper award talk at: FSE |
Place Published | Hong Kong |
Usikkerhetsestimering av IT-prosjekter
In Presentation at seminar for Kongsberg FMC Subsea, 2014.Status: Published
Usikkerhetsestimering av IT-prosjekter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | Presentation at seminar for Kongsberg FMC Subsea |
Journal Article
Failure Factors of Small Software Projects at a Global Outsourcing Marketplace
Journal of Systems and Software 92 (2014): 157-169.Status: Published
Failure Factors of Small Software Projects at a Global Outsourcing Marketplace
The presented study aims at a better understanding of when and why software projects fail. The analysis is based on a data set of 785,325 small-scale outsourced software projects. A binary logistic regression model relying solely on information known at the time of a project's start-up correctly predicted 74% of the project failures and 67% of the non-failures. The model-predicted failure probability corresponded well with the actual frequencies of failures for most levels of failure risk. The model suggests that the factors connected to the strongest reduction in the risk of failure are related to previous collaboration between the client and the provider and a low failure rate of previous projects completed by the provider. We found the characteristics of the client to be almost as important as those of the provider in explaining project failures and that the risk of project failure increased with an increased client emphasis on low price and with an increased project size. The identified relationships seem to be stable across the studied project size categories, which indicate that the results may potentially be applicable to larger projects than the small-scale projects dominating this data set.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2014 |
Journal | Journal of Systems and Software |
Volume | 92 |
Number | x |
Pagination | 157-169 |
Keywords | Conference, Internal Seminar, University of Oslo, Workshop |
What We Do and Don't Know About Software Development Effort Estimation
IEEE Software 31, no. 2 (2014): 37-40.Status: Published
What We Do and Don't Know About Software Development Effort Estimation
OVERWHELMING EVIDENCE DOCUMENTS a tendency toward cost and effort overruns in software projects. On average, this overrun seems to be around 30 percent.1 Furthermore, comparing the estimation accuracy of the 1980s with that reported in more recent surveys suggests that the estimation accuracy hasn't changed much since then. Estimation methods haven't changed much either. In spite of an extensive research on formal estimation models, the dominating estimation method is still expert estimation. An apparent lack of improvement in estimation accuracy doesn't mean that we don't know more about effort estimation than before. In this article, I try to summarize some of the knowledge I believe we've gained. Some of this knowledge has the potential of improving the estimation accuracy, some is about what most likely will not lead to improvements, and some is about what we know we don't know about effort estimation.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2014 |
Journal | IEEE Software |
Volume | 31 |
Issue | 2 |
Number | March/April |
Pagination | 37-40 |
Talk, keynote
From Myths and Fashions to Evidence-Based Software Engineering
In Keynote at: Lindholmen Software Day, Gothenburg, Sweeden, 2014.Status: Published
From Myths and Fashions to Evidence-Based Software Engineering
The software engineering discipline contains numerous myths and over-simplifications. Some of them may be harmless, but others may lead to inefficient practices and contribute to a fashion- and myth-based software engineering discipline. In this presentation I give examples of software engineering myths and over-simplifications, discuss how they are created and spread and illustrate how it is possible to base important software engineering decision and practice on available evidence from research, practice and own empirical studies. A move towards evidence-based software engineering requires that software professionals become more critical towards claims, know how to formulate answerable questions, collect and evaluate evidence and use evidence to guide important decisions. This requires training and, not least, a change in mindset. Results on how to do this are presented.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2014 |
Location of Talk | Keynote at: Lindholmen Software Day, Gothenburg, Sweeden |
Myths, Over-Simplifications and Unconfirmed Claims in Software Engineering
In Keynote at: EUROMICRO DSD/SEAA, Verona, Italy, 2014.Status: Published
Myths, Over-Simplifications and Unconfirmed Claims in Software Engineering
The software engineering discipline contains numerous myths, over-simplifications and unconfirmed claims. Some of them may be harmless, but others may hamper the use of good practices. Examples of myths, over-simplifications and unconfirmed claims are presented to demonstrate the mechanisms contributing to the creation, spread and survival of them. Ten years ago we (Barbara Kitchenham, Tore Dybå and myself) introduced evidence-based software engineering. The goal was to support a move towards a discipline with practices based on valid scientific and experience-based evidence, i.e., to reduce the influence from myths, over-simplifications and unconfirmed claims. I report from my experience in providing training in the use of evidence-based practices to software professionals and computer science students. The experience includes the need to emphasize and train software professionals in the collection and evaluation of practice-based experience and in the design and completion of studies within their own context, not so much in collecting and evaluating research-based evidence.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2014 |
Location of Talk | Keynote at: EUROMICRO DSD/SEAA, Verona, Italy |
Skumle IT-Prosjekter Og Sorte Svaner
In Keynote at CIO Forum on Project Management, 2014.Status: Published
Skumle IT-Prosjekter Og Sorte Svaner
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2014 |
Location of Talk | Keynote at CIO Forum on Project Management |
Ten Years With Evidence-Based Software Engineering. What Is It? Has It Had Any Impact? What's Next?
In Keynote on 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 2014.Status: Published
Ten Years With Evidence-Based Software Engineering. What Is It? Has It Had Any Impact? What's Next?
An evidence-based software engineer is one who is able to: 1) Formulate a question, related to a decision or judgment, so that it can be answered by the use of evidence, 2) Collect, critically evaluate and summarise relevant evidence from research, practise and local studies, 3) Apply the evidence, integrated with knowledge about the local context, to guide decisions and judgments. The keynote reflects on what it in practise means to be evidence-based in software engineering contexts, where the number of different contexts is high and the research-based evidence sparse, and why there is a need for more evidence-based practises. We summarise our experience from ten years of Evidence-Based Software Engineering in the context of university courses, training of software engineers and systematic literature reviews of software engineering research. While there are challenges in training people in evidence-based practise, our experience suggest that it is feasible and that the training can make an important difference in terms of quality of software engineering judgment and decisions. Based on our experience we suggest changes in how evidence-based software engineering should be presented and taught, and how we should ease the transfer of research results into evidence-based practises.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2014 |
Location of Talk | Keynote on 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering |
Publisher | ACM |
Things You Were Never Told, Did Not Understand, Forgot, Or Chose to Ignore in Statistics
In Keynote at 1st Conference on Advanced Probability and Statistics in Information Systems (APSIS), 2014.Status: Published
Things You Were Never Told, Did Not Understand, Forgot, Or Chose to Ignore in Statistics
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2014 |
Location of Talk | Keynote at 1st Conference on Advanced Probability and Statistics in Information Systems (APSIS) |
Public outreach
Hva er et kostnadsestimat?
, 2014.Status: Published
Hva er et kostnadsestimat?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2014 |
Issue | February |
Publisher | |
Type of Work | Article in Computerworld Norge |
Hvor ofte og når lykkes IT-prosjekter?
, 2014.Status: Published
Hvor ofte og når lykkes IT-prosjekter?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2014 |
Issue | September |
Publisher | |
Type of Work | Article in Computerworld Norge |
Hvordan gjøre statistikk mer overbevisende?
, 2014.Status: Published
Hvordan gjøre statistikk mer overbevisende?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2014 |
Issue | March |
Publisher | |
Type of Work | Article in Computerworld Norge |
Kommunikasjon av kostnadsestimater
, 2014.Status: Published
Kommunikasjon av kostnadsestimater
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2014 |
Issue | January |
Publisher | |
Type of Work | Article in Computerworld Norge |
Lønner det seg med mer utdannelse?
, 2014.Status: Published
Lønner det seg med mer utdannelse?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2014 |
Issue | April |
Publisher | |
Type of Work | Article in Computerworld Norge |
Når vi har bestemt oss, ser mye annerledes ut
, 2014.Status: Published
Når vi har bestemt oss, ser mye annerledes ut
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2014 |
Issue | June |
Publisher | |
Type of Work | Article in Computerworld Norge |
Offshoring av IT-utvikling: Hva er viktig for å lykkes?
, 2014.Status: Published
Offshoring av IT-utvikling: Hva er viktig for å lykkes?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2014 |
Publisher | |
Type of Work | Presentation at DIGIN-seminar |
Praedico ergo sum
, 2014.Status: Published
Praedico ergo sum
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2014 |
Issue | October |
Publisher | |
Type of Work | Article in Computerworld Norge |
Prosjektstyring: Ikke mye nytt siden pyramidene
, 2014.Status: Published
Prosjektstyring: Ikke mye nytt siden pyramidene
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2014 |
Issue | December |
Publisher | |
Type of Work | Article in Computerworld Norge |
Visdommen til små grupper
, 2014.Status: Published
Visdommen til små grupper
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2014 |
Publisher | |
Type of Work | Article in Computerworld Norge |
Poster
The Effect of Facial Expression on Evaluation of Competence: Keep Smiling!
In Poster at: 17th General Meeting of the European Association of Social Psychology, The Netherlands, Amsterdam, 2014.Status: Published
The Effect of Facial Expression on Evaluation of Competence: Keep Smiling!
Afilliation | Software Engineering, Software Engineering |
Publication Type | Poster |
Year of Publication | 2014 |
Secondary Title | Poster at: 17th General Meeting of the European Association of Social Psychology, The Netherlands, Amsterdam |
Date Published | July |
Proceedings, refereed
A Strong Focus on Low Price When Selecting Software Providers Increases the Likelihood of Failure in Software Outsourcing Projects
In 7th International Conference on Evaluation and Assessment in Software Engineering, 2013.Status: Published
A Strong Focus on Low Price When Selecting Software Providers Increases the Likelihood of Failure in Software Outsourcing Projects
Context: Bidding rounds are frequently used to select competent and cost-efficient providers for software projects. Objective: We hypothesize that emphasizing low price when selecting software providers in such bidding rounds substantially increases the likelihood the project will fail. Method: The hypothesis is tested by analyzing a dataset of 4,791,067 bids for 785,326 small-scale projects registered at a web-based marketplace connecting software clients and providers. Results: We find evidence supporting our hypothesis. For example, selecting providers with bids 25% lower than the average bid is connected to a 9% increase in the frequency of project failures for the same level of provider skill. In addition, we found that clients emphasizing a low price, on average, selected providers with lower skill levels. This decrease in provider skill level further strengthened the negative effect of a strong focus on low price on project failures. For example, selecting a provider with a 15% failure rate for previous projects instead of 5% increased the failure rate by 33%. Conclusion: We interpret the findings to suggest that a client may substantially reduce the likelihood of project failure by reducing the emphasis on low price when selecting a provider.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | 7th International Conference on Evaluation and Assessment in Software Engineering |
Pagination | 220-227 |
Poster
Cognitive Bias Remains After De-Biasing Information
2013.Status: Published
Cognitive Bias Remains After De-Biasing Information
Context: The role of humans is becoming increasingly appreciated in our quest to improve software project prediction. The accuracy of the predictions is dependent on many factors, not least cognitive biases, which influence thinking and can often lead to poor decisions. Objective: The strong impact of the anchoring bias is well known. This study investigated the impact of anchoring on professional software engineers' estimation accuracy after they had attended a workshop specifically aimed at addressing this bias. We hypothesised the bias would remain despite the workshop and that location could be a confound. Method: We ran four studies with a total of 118 software engineers to explore the impact of the anchoring bias on the accuracy of productivity estimations made by professional software engineers. The participants attended a workshop introducing the concept of cognitive bias and the influence on decision making and were introduced to strategies for de-biasing. Immediately after the workshop, participants were randomly allocated to a high or a low anchor group and asked to estimate their productivity as lines of code per hour (LOC{-}hr) on a recently completed project. Results: The anchor had a large effect (partial η2 \sim 0.22; p < 0.0001) in influencing estimates. The median estimate of hourly productivity for the high anchor was 30 LOC{-}hr and 7 LOC{-}hr for the low anchor. Lower values might reasonably be interpreted as more realistic estimates of hourly productivity. Conclusion: The impact of bias upon expert judgement is substantial, seemingly ubiquitous and difficult to eradicate. This study demonstrates the pervasive nature of the anchoring bias despite de-biasing information.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Poster |
Year of Publication | 2013 |
Date Published | July |
Keywords | Conference |
Public outreach
Det økonomisk Mest Fordelaktige Tilbudet
2013.Status: Published
Det økonomisk Mest Fordelaktige Tilbudet
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2013 |
Er ”kopier Og Lim”-Programmering Skadelig?
2013.Status: Published
Er ”kopier Og Lim”-Programmering Skadelig?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2013 |
Lederkompetanse Evaluert På 100 Millisekunder
2013.Status: Published
Lederkompetanse Evaluert På 100 Millisekunder
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2013 |
Oppsiktsvekkende Resultater
2013.Status: Published
Oppsiktsvekkende Resultater
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2013 |
Presten Bayes, Kravspesifikasjoner Og Skandalejournalisme
2013.Status: Published
Presten Bayes, Kravspesifikasjoner Og Skandalejournalisme
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2013 |
Problemer Med Offshoring Av IT. Kultur- Eller Kvalitetsproblemer?
2013.Status: Published
Problemer Med Offshoring Av IT. Kultur- Eller Kvalitetsproblemer?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2013 |
Ustrukturerte Ansettelsesintervjuer
2013.Status: Published
Ustrukturerte Ansettelsesintervjuer
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2013 |
Talks, invited
Dødssynder Innen Kostnadsestimering
In Presentation at SMIDIG 2013, 2013.Status: Published
Dødssynder Innen Kostnadsestimering
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2013 |
Location of Talk | Presentation at SMIDIG 2013 |
Keywords | Conference |
Estimater, Usikkerhet, Kommunikasjon. Hva Er Et Estimat? Hvordan Be Om Og Kommunisere Usikkerhet? Effekt Av Ulike Avtaletyper.
In Presentation at internal seminar at Creuna, 2013.Status: Published
Estimater, Usikkerhet, Kommunikasjon. Hva Er Et Estimat? Hvordan Be Om Og Kommunisere Usikkerhet? Effekt Av Ulike Avtaletyper.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2013 |
Location of Talk | Presentation at internal seminar at Creuna |
Keywords | Workshop |
Estimering Av IT-Prosjekter: Hvorfor Bommer Vi Og Hvordan Kan Vi Bli Bedre?
In Presentation at five company internal seminars (KnowIT, Bouvet, Mesan, Politiet, Storebrand), 2013.Status: Published
Estimering Av IT-Prosjekter: Hvorfor Bommer Vi Og Hvordan Kan Vi Bli Bedre?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2013 |
Location of Talk | Presentation at five company internal seminars (KnowIT, Bouvet, Mesan, Politiet, Storebrand) |
Keywords | Workshop |
From Myths and Fashions to Evidence-Based Software Engineering
In Presented at internal seminars at OneCall, Deerwalk (Nepal), Thales and at Certus workshop (Simula), 2013.Status: Published
From Myths and Fashions to Evidence-Based Software Engineering
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2013 |
Location of Talk | Presented at internal seminars at OneCall, Deerwalk (Nepal), Thales and at Certus workshop (Simula) |
Keywords | Workshop |
How Much Is a Great Developer Worth?
In Presentation at Norwegian Developer Conference (NDC) 2013, 2013.Status: Published
How Much Is a Great Developer Worth?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2013 |
Location of Talk | Presentation at Norwegian Developer Conference (NDC) 2013 |
Keywords | Conference |
How to Avoid Disappointments in Software Projects (and Life)
In Presentation at Gilb Week (London), 2013.Status: Published
How to Avoid Disappointments in Software Projects (and Life)
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2013 |
Location of Talk | Presentation at Gilb Week (London) |
Keywords | Workshop |
Hvorfor (ikke) Fastpris? Vinnerens Forbannelse, Informasjonsasymmetri, Utvalgsrisiko, Opportunistisk Adferd Og IT-Kontrakter
In Presented at internal seminar at Norsvin, 2013.Status: Published
Hvorfor (ikke) Fastpris? Vinnerens Forbannelse, Informasjonsasymmetri, Utvalgsrisiko, Opportunistisk Adferd Og IT-Kontrakter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2013 |
Location of Talk | Presented at internal seminar at Norsvin |
Keywords | Workshop |
Offshoring Av IT-Utvikling: Hva Er Viktig for å Lykkes?
In Presented at internal seminar at Telenor, 2013.Status: Published
Offshoring Av IT-Utvikling: Hva Er Viktig for å Lykkes?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2013 |
Location of Talk | Presented at internal seminar at Telenor |
Keywords | Workshop |
Talk, keynote
Finding Results Where There Are None
In Keynote at: IWSM-Mensura, Ankara, Turkey, 2013.Status: Published
Finding Results Where There Are None
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2013 |
Location of Talk | Keynote at: IWSM-Mensura, Ankara, Turkey |
Judgement and Decision-Making in Software Engineering
In Keynote at: QSIC, Nanjing, china, 2013.Status: Published
Judgement and Decision-Making in Software Engineering
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2013 |
Location of Talk | Keynote at: QSIC, Nanjing, china |
Date Published | July |
Keywords | Conference |
Journal Article
Relative Estimation of Software Development Effort: It Matters With What and How You Compare
IEEE Software (2013): 74-79.Status: Published
Relative Estimation of Software Development Effort: It Matters With What and How You Compare
Software development effort estimation is frequently based on comparing the effort of one task relative to that of another. We present empirical results that show how relative estimation may result in biased assessments of similarity and over-optimistic effort estimates. We observe that tasks tend to be assessed as more similar than they in reality are when compared with each other, that the similarity of two tasks depends on the direction of the comparison and that it matters whether the comparison is based on difference in work-hours or as the ratio. We use the observations and other evidence to suggest ways of improving the accuracy of relative estimation.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2013 |
Journal | IEEE Software |
Number | March |
Pagination | 74-79 |
The Influence of Selection Bias on Effort Overruns in Software Development Projects
Information and Software Technology 55 (2013): 1640-1650.Status: Published
The Influence of Selection Bias on Effort Overruns in Software Development Projects
Context: A potentially important, but neglected, reason for effort overruns in software projects is related to selection bias. Selection bias-induced effort overruns occur when proposals are more likely to be accepted and lead to actual projects when based on effort estimates that are too low rather than on realistic estimates or estimates that are too high. The effect of this bias may be particularly important in bidding rounds, but is potentially relevant in all situations where there is effort or cost-based selection between alternatives. Objective: To better understand the relevance and management of selection bias effects in software development contexts. Method: First, we present a statistical model illustrating the relation between selection bias in bidding and other contexts and effort overruns. Then, we examine this relation in an experiment with software professionals who estimated and completed a set of development tasks and examine relevant field study evidence. Finally, we use a selection bias scenario to assess awareness of the effect of selection bias among software providers. Results: The results from the statistical model and the experiment demonstrated that selection bias is capable of explaining much of the effort overruns. The field evidence was also consistent with a substantial effect of selection bias on effort overruns, although there are alternative explanations for the findings. We found a low awareness of selection bias among the software providers. Conclusion: Selection bias is likely to be an important source of effort overruns and should be addressed to reduce problems related to over-optimistic effort estimates.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2013 |
Journal | Information and Software Technology |
Volume | 55 |
Number | 9 |
Pagination | 1640-1650 |
Proceedings, refereed
Does the Use of Fibonacci Numbers in Planning Poker Affect Effort Estimates?
In 16th Int. Conference on Evaluation & Assessment in Software Engineering. IET, 2012.Status: Published
Does the Use of Fibonacci Numbers in Planning Poker Affect Effort Estimates?
Background: The estimation technique Planning Poker is common in agile software development. The cards used to propose an estimate in Planning Poker do not include all numbers, but for example only the numbers 0, ½, 1, 2, 3, 5, 8, 13, 20, 40 and 100. We denote this, somewhat inaccurately, a Fibonacci scale in this paper. In spite of the widespread use of the Fibonacci scale in agile estimation, we do not know much about how this scale influences the estimation process. Aim: Better understanding of the effect of going from a linear scale to a Fibonacci scale in effort estimation. Method: We conducted two empirical studies. In the first study, we gave computer science students the same estimation task. Half of the students estimated the task using the Fibonacci scale and the other half a linear scale. The second study included four estimation teams, each composed of four software professionals, estimating the effort to complete the same ten tasks. Two of the teams estimated the first five tasks using the Fibonacci scale and the last five using the linear scale. The two other teams used the scales in the opposite sequence. Results: We found a median decrease in the effort estimates of 60% (first study) and 26% (second study) when using a Fibonacci scale instead of the traditional linear scale. The scale difference in the effort estimates decreased as the developers' skill increased. Conclusion: The use of a Fibonacci scale, and possibly other non-linear scales, is likely to affect the effort estimates towards lower values compared to linear scales. A possible explanation for this scale-induced effect is that people tend to be biased towards toward the middle of the provided scale, especially when the uncertainty is substantial. The middle value is likely to be perceived as lower for the Fibonacci than for the linear scale.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | 16th Int. Conference on Evaluation & Assessment in Software Engineering |
Pagination | 228-232 |
Publisher | IET |
First Impressions in Software Development Effort Estimation: Easy to Create and Difficult to Neutralize
In 16th Int. Conference on Evaluation & Assessment in Software Engineering. IET, 2012.Status: Published
First Impressions in Software Development Effort Estimation: Easy to Create and Difficult to Neutralize
The four studies included in this paper examine the strength of the first impression in the estimation of work effort. The studied context is mainly one where the first impressions of software development effort is manipulated (biased) through comparisons with either much too low or too high reference effort values, e.g., by responding to a question from the client whether one believes that the effort will be less than 10 work-hours when the likely usage of effort typically would be in the range of 100-300 work-hours. Then, the software developers are exposed to a subsequent comparison with reference effort values in the opposite direction, e.g., by responding to a question from the client whether one believes that the effort would be less than 800 work-hours. The results from the four studies suggest a strong first impression effect, but also a noticeable effect from the subsequent comparisons. We also observe that the instruction to “forget” the first impression seems to have the opposite effect, i.e., it seems to increase the strength of the first impression. A practical implication of the results is that it is essential that software professionals ensure that their first impression of a project's development effort is based on comparisons with representative reference values and objects. First impressions in software development seem to be easy to manipulate with misleading reference values and difficult to replace.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | 16th Int. Conference on Evaluation & Assessment in Software Engineering |
Pagination | 216-222 |
Publisher | IET |
Myths and Over-Simplifications in Software Engineering
In Proceedings of ICSCT. Lecture Notes on Software Engineering, 2012.Status: Published
Myths and Over-Simplifications in Software Engineering
The software engineering discipline contains numerous myths and over-simplifications. Some of them may be harmless, but others may hamper evidence-based practices and contribute to a fashion- and myth-based software engineering discipline. In this article we give examples of software engineering myths and over-simplifications and discuss how they are created and spread. One essential mechanism of the creation and spread of myths and over-simplifications are, we argue, people's tendency towards searching for confirming and neglecting disconfirming evidence. We report from a study examining this tendency. The study demonstrated that the developers who believed in a positive effect of agile methods tended to interpret randomly generated (neutral) project data as evidence confirming the benefit of agile methods. For the purpose of supporting evidence-based practice and avoiding unwanted influence from myths and over-simplifications, we provide a checklist to be used to evaluate the validity of software engineering claims.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | Proceedings of ICSCT |
Publisher | Lecture Notes on Software Engineering |
Talks, invited
Fra Myter Til Evidens
In Invited talk (TAD-seminar), 2012.Status: Published
Fra Myter Til Evidens
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2012 |
Location of Talk | Invited talk (TAD-seminar) |
Relative Estimation of Software Development Effort: It Matter With What and How You Compare
In Presentation at JavaZone, 2012.Status: Published
Relative Estimation of Software Development Effort: It Matter With What and How You Compare
Afilliation | Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2012 |
Location of Talk | Presentation at JavaZone |
Software Development Effort Estimation: Why It Fails and How to Improve It
In Steria-seminar, 2012.Status: Published
Software Development Effort Estimation: Why It Fails and How to Improve It
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2012 |
Location of Talk | Steria-seminar |
Journal Article
From Origami to Software Development: a Review of Studies on Judgment-Based Predictions of Performance Time
Psychological Bulletin 138 (2012): 238-271.Status: Published
From Origami to Software Development: a Review of Studies on Judgment-Based Predictions of Performance Time
Afilliation | Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2012 |
Journal | Psychological Bulletin |
Volume | 138 |
Number | 2 |
Pagination | 238-271 |
How Does Project Size Affect Cost Estimation Error? Statistical Artifacts and Methodological Challenges
International Journal of Project Management 30 (2012): 751-862.Status: Published
How Does Project Size Affect Cost Estimation Error? Statistical Artifacts and Methodological Challenges
Empirical studies differ in what they report as the underlying relation between project size and percent cost overrun. As a consequence, the studies also differ in their project management recommendations. We show that studies with a project size measure based on the actual cost systematically report an increase in percent cost overrun with increased project size, whereas studies with a project size measure based on the estimated cost report a decrease or no change in percent cost overrun with increased project size. The observed pattern is, we argue, to some extent a statistical artifact caused by imperfect correlation between the estimated and the actual cost. We conclude that the previous observational studies cannot be considered as providing reliable evidence in favor of an underlying project size related cost estimation bias. The more robust evidence from controlled experiments, limited to small tasks, suggests an increase in underestimation with increased project size.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2012 |
Journal | International Journal of Project Management |
Volume | 30 |
Number | 7 |
Pagination | 751-862 |
Interpretation Problems Related to the Use of Regression Models to Decide on Economy of Scale in Software Development
Journal of Systems and Software 85 (2012): 2494-2503.Status: Published
Interpretation Problems Related to the Use of Regression Models to Decide on Economy of Scale in Software Development
Many research studies report an economy of scale in software development, i.e., an increase in productivity with increasing project size. Several software practitioners seem, on the other hand, to believe in a diseconomy of scale, i.e., a decrease in productivity with increasing project size. In this paper we argue that violations of essential regression model assumptions in the research studies to a large extent may explain this disagreement. Particularly illustrating is the finding that the use of the production function (Size = a\bulletEffortb), instead of the factor input model (Effort=a\bulletSizeb), would most likely have led to the opposite result, i.e., a tendency towards reporting diseconomy of scale in the research studies. We conclude that there are good reasons to warn against the use of regression analysis parameters to investigate economies of scale and to look for other analysis methods when studying economy of scale in software development contexts.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2012 |
Journal | Journal of Systems and Software |
Volume | 85 |
Number | 11 |
Pagination | 2494-2503 |
Software Development Estimation Biases: the Role of Interdependence
IEEE Transactions on Software Engineering 38 (2012): 677-693.Status: Published
Software Development Estimation Biases: the Role of Interdependence
Software development effort estimates are frequently too low, which may lead to poor project plans and project failures. One reason for this bias seems to be that the effort estimates produced by software developers are affected by information that has no relevance for the actual use of effort. We attempted to acquire a better understanding of the underlying mechanisms and the robustness of this type of estimation bias. For this purpose, we hired 374 software developers working in outsourcing companies to participate in a set of three experiments. The experiments examined the connection between estimation bias and developer dimensions: Self-construal (how one sees oneself), thinking style, nationality, experience, skill, education, sex, and organizational role. We found that estimation bias was present along most of the studied dimensions. The most interesting finding may be that the estimation bias increased significantly with higher levels of interdependence, i.e., with stronger emphasis connectedness, social context and relationships. We propose that this connection may be enabled by an activation of one's self-construal when engaging in effort estimation, and, a connection between a more interdependent self-construal and increased search for indirect messages, lower ability to ignore irrelevant context, and a stronger emphasis on socially desirable responses.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2012 |
Journal | IEEE Transactions on Software Engineering |
Volume | 38 |
Number | 3 |
Pagination | 677-693 |
Public outreach
Hvor Gode Er Vi Til å Forutse Kriser?
2012.Status: Published
Hvor Gode Er Vi Til å Forutse Kriser?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2012 |
Hvor Mye Er En Svært Dyktig IT-Utvikler Verdt?
2012.Status: Published
Hvor Mye Er En Svært Dyktig IT-Utvikler Verdt?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2012 |
Jo Mer Usikker, Jo Mer Middels
2012.Status: Published
Jo Mer Usikker, Jo Mer Middels
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2012 |
Lønnsdiskriminering I IT-Bransjen
2012.Status: Published
Lønnsdiskriminering I IT-Bransjen
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2012 |
Smått Er Godt, Eller Jo Større Jo Bedre
2012.Status: Published
Smått Er Godt, Eller Jo Større Jo Bedre
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2012 |
Vakker Programvarekode
2012.Status: Published
Vakker Programvarekode
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2012 |
Vil Vi Noen Gang Få Feilfri Programvare?
2012.Status: Published
Vil Vi Noen Gang Få Feilfri Programvare?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2012 |
Talk, keynote
Things aren’t always what they seem: Misinterpretation of observational data in software engineering
In Keynote at Euromicro, CESME, 2012.Status: Published
Things aren’t always what they seem: Misinterpretation of observational data in software engineering
Afilliation | Software Engineering |
Project(s) | No Simula project |
Publication Type | Talk, keynote |
Year of Publication | 2012 |
Location of Talk | Keynote at Euromicro, CESME |
Date Published | August |
Keywords | Conference |
Public outreach
Å Svare På Hvor Fornøyd Du Er Gjør Deg Mindre Fornøyd
2011.Status: Published
Å Svare På Hvor Fornøyd Du Er Gjør Deg Mindre Fornøyd
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2011 |
Du Er Mer Lik Meg Enn Jeg Er Lik Deg
2011.Status: Published
Du Er Mer Lik Meg Enn Jeg Er Lik Deg
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2011 |
FLOSS, Flott Eller Flopp
2011.Status: Published
FLOSS, Flott Eller Flopp
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2011 |
Huleboere Bodde Ikke I Huler Men I Telt
2011.Status: Published
Huleboere Bodde Ikke I Huler Men I Telt
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2011 |
IT-Fiaskoer: Djevelen Er I Detaljene
2011.Status: Published
IT-Fiaskoer: Djevelen Er I Detaljene
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2011 |
Lønner Det Seg å Følge IT-Moten?
2011.Status: Published
Lønner Det Seg å Følge IT-Moten?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2011 |
Testere Bør Være Destruktive
2011.Status: Published
Testere Bør Være Destruktive
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2011 |
Journal Article
Contrasting Ideal and Realistic Conditions As a Means to Improve Judgment-Based Software Development Effort Estimation
Information and Software Technology 53 (2011): 1382-1390.Status: Published
Contrasting Ideal and Realistic Conditions As a Means to Improve Judgment-Based Software Development Effort Estimation
Context: The effort estimates of software development work are on average too low. A possible reason for this tendency is that software developers, perhaps unconsciously, assume ideal conditions when they estimate the most likely use of effort. In this article, we propose and evaluate a two-step estimation process that may induce more awareness of the difference between idealistic and realistic conditions and as a consequence more realistic effort estimates. The proposed process differs from traditional judgment-based estimation processes in that it starts with an effort estimation that assumes ideal conditions before the most likely use of effort is estimated. Objective: The objective of the paper is to examine the potential of the proposed method to induce more realism in the judgment-based estimates of work effort. Method: Three experiments with software professionals as participants were completed. In all three experiments there was one group of participants which followed the proposed and another group which followed the traditional estimation process. In one of the experiments there was an additional group which started with a probabilistically defined estimate of minimum effort before estimating the most likely effort. Results: We found, in all three experiments, that estimation of most likely effort seems to assume rather idealistic assumptions and that the use of the proposed process seems to yield more realistic effort estimates. In contrast, starting with an estimate of the minimum effort, rather than an estimate based on ideal conditions, did not have the same positive effect on the subsequent estimate of the most likely effort. Conclusion: The empirical results from our studies together with similar results from other domains suggest that the proposed estimation process is promising for the improvement of the realism of software development effort estimates.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2011 |
Journal | Information and Software Technology |
Volume | 53 |
Number | 12 |
Pagination | 1382-1390 |
The Impact of Irrelevant and Misleading Information on Software Development Effort Estimates: a Randomized Controlled Field Experiment
IEEE Transactions on Software Engineering 37 (2011): 695-707.Status: Published
The Impact of Irrelevant and Misleading Information on Software Development Effort Estimates: a Randomized Controlled Field Experiment
Several studies have reported that software development effort estimates can be strongly affected by effort-irrelevant and misleading information without the estimators being aware of this effect. These studies were conducted in laboratory (artificial) estimation contexts. To increase our knowledge about the importance of these effects in field settings, we paid 46 outsourcing companies from Eastern European and East Asian countries to estimate the required effort of the same five software development projects. The companies were allocated randomly to either the original requirement specification or a manipulated version of the original requirement specification. The manipulations were as follows: i) reduced length of requirement specification with no change of content, ii) information about the low effort spent on the development of the old system to be replaced, iii) information about the client's unrealistic expectations about low cost, and iv) a restriction of a short development period with start up a few months ahead (which should, rationally speaking, lead to an increase in effort). All manipulations led to decreased median effort estimates, but only manipulation iv) led to a large, statistically significant decrease. A comparison of the effects of similar types of irrelevant and misleading information in laboratory and field settings suggests that the effect of manipulations i), ii) and iii) where much lower in field settings than in laboratory settings, while the effect of manipulation iv) was almost at the same level. We conclude that the tendency towards a smaller effect in field settings means that laboratory studies are frequently only useful for demonstrating the existence of a software engineering phenomenon, or for understanding it better, and that we need field studies to analyze its importance.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2011 |
Journal | IEEE Transactions on Software Engineering |
Volume | 37 |
Number | 5 |
Pagination | 695-707 |
To Read Two Pages, I Need 5 Minutes, But Give Me 5 Minutes and I Will Read Four: How to Change Productivity Estimates by Inverting the Question
Applied Cognitive Psychology 25 (2011): 314-323.Status: Published
To Read Two Pages, I Need 5 Minutes, But Give Me 5 Minutes and I Will Read Four: How to Change Productivity Estimates by Inverting the Question
Past research has shown that people underestimate the time they need to complete large tasks, whereas completion times for smaller tasks are often overestimated, suggesting higher productivity estimates for larger than for smaller tasks. By replacing the traditional question about how much time a given work will take with a question about how much work can be completed within a given amount of time, we found the opposite pattern. This could reflect a general tendency to underestimate large amounts relatively to small ones both for durations and for amounts of work. We explored this idea in two studies where students estimated reading tasks, a third where IT-professionals estimated software projects, and a fourth where participants imagined a familiar walk, divided into time segments or part distances of varying lengths.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2011 |
Journal | Applied Cognitive Psychology |
Volume | 25 |
Number | 2 |
Pagination | 314-323 |
Talks, invited
Du Er Mer Lik Meg Enn Jeg Er Lik Deg: Asymmetri I Relativ Estimering
In XP-meeting, 2011.Status: Published
Du Er Mer Lik Meg Enn Jeg Er Lik Deg: Asymmetri I Relativ Estimering
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2011 |
Location of Talk | XP-meeting |
Estimering Av IT-Prosjekter: Hva Vet Vi? Hvordan Bli Bedre?
In Sparebank 1-seminar, 2011.Status: Published
Estimering Av IT-Prosjekter: Hva Vet Vi? Hvordan Bli Bedre?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2011 |
Location of Talk | Sparebank 1-seminar |
How We Know What Isn't So: Common Myths in Daily Life and Software Development. Why Do We Believe in Myths?
In Chalmers-seminar, 2011.Status: Published
How We Know What Isn't So: Common Myths in Daily Life and Software Development. Why Do We Believe in Myths?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2011 |
Location of Talk | Chalmers-seminar |
Human Judgment in Planning and Estimation of Software Projects
In ICKE 2011 (India), 2011.Status: Published
Human Judgment in Planning and Estimation of Software Projects
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2011 |
Location of Talk | ICKE 2011 (India) |
Identification and Management of IT-Projects With High Risk of Cost Overrun
In ABB seminar, 2011.Status: Published
Identification and Management of IT-Projects With High Risk of Cost Overrun
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2011 |
Location of Talk | ABB seminar |
Identification and Management of IT-Projects With High Risk of Cost Overrun
In Bearing Point-seminar, 2011.Status: Published
Identification and Management of IT-Projects With High Risk of Cost Overrun
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2011 |
Location of Talk | Bearing Point-seminar |
Software Development Effort Estimation: Why It Fails and How to Improve It
In Seminar held for various companies in Ukraine, Mexico and Nepal, 2011.Status: Published
Software Development Effort Estimation: Why It Fails and How to Improve It
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2011 |
Location of Talk | Seminar held for various companies in Ukraine, Mexico and Nepal |
Vinnerens Forbannelse, Informasjonsasymmetri, Utvalgsrisiko, Moralsk Risiko Og IT-Kontrakter
In Confex Seminar, 2011.Status: Published
Vinnerens Forbannelse, Informasjonsasymmetri, Utvalgsrisiko, Moralsk Risiko Og IT-Kontrakter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2011 |
Location of Talk | Confex Seminar |
Talk, keynote
From Fashion and Opinion-Based to Evidence-Based Software and Knowledge Engineering
In Keynote at: ICKE (India), 2011.Status: Published
From Fashion and Opinion-Based to Evidence-Based Software and Knowledge Engineering
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2011 |
Location of Talk | Keynote at: ICKE (India) |
Lønner Det Seg å Investere I IT-Prosjekter? Hva Vet Vi Om ROI-Prediksjoner?
In Keynote at: IDG, Oslo, 2011.Status: Published
Lønner Det Seg å Investere I IT-Prosjekter? Hva Vet Vi Om ROI-Prediksjoner?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2011 |
Location of Talk | Keynote at: IDG, Oslo |
Book Chapter
Overconfidence in the Accuracy of Own Work Effort Predictions: the Role of Interval Width
In Perspectives on thinking, judging, and decision making, 47-56. Universitetsforlaget, 2011.Status: Published
Overconfidence in the Accuracy of Own Work Effort Predictions: the Role of Interval Width
Afilliation | Software Engineering, Software Engineering |
Publication Type | Book Chapter |
Year of Publication | 2011 |
Book Title | Perspectives on thinking, judging, and decision making |
Chapter | 3 |
Pagination | 47-56 |
Publisher | Universitetsforlaget |
ISBN Number | 978-82-15-01878-2 |
Public outreach
Blir Vi Mer Og Mer Middelmådige?
In Computerworld, 2010.Status: Published
Blir Vi Mer Og Mer Middelmådige?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2010 |
Secondary Title | Computerworld |
Hva Skjer Når Budsjettet Eller Leveransedato Presses?
In Computerworld, 2010.Status: Published
Hva Skjer Når Budsjettet Eller Leveransedato Presses?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2010 |
Secondary Title | Computerworld |
Offshoring: Vær En God Kunde - Og Du Vil Lykkes
In Dagens Næringsliv, 2010.Status: Published
Offshoring: Vær En God Kunde - Og Du Vil Lykkes
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2010 |
Secondary Title | Dagens Næringsliv |
Små gaver er også egnet til å påvirke
In Computerworld, 2010.Status: Published
Små gaver er også egnet til å påvirke
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2010 |
Secondary Title | Computerworld |
Vår Medfødte Tallforståelse
In Computerworld, 2010.Status: Published
Vår Medfødte Tallforståelse
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2010 |
Secondary Title | Computerworld |
Ønsketenkning
In Computerworld, 2010.Status: Published
Ønsketenkning
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2010 |
Secondary Title | Computerworld |
”Det Meste Av Kommunikasjon Er Ikke-Verbal" Og Annen Misbruk Av Forskning
In Computerworld, 2010.Status: Published
”Det Meste Av Kommunikasjon Er Ikke-Verbal" Og Annen Misbruk Av Forskning
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2010 |
Secondary Title | Computerworld |
Talks, contributed
Estimering Av IT-Prosjekter: Hva Vet Vi? Hvordan Bli Bedre?
In Presentation at Computas seminar, 2010.Status: Published
Estimering Av IT-Prosjekter: Hva Vet Vi? Hvordan Bli Bedre?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Presentation at Computas seminar |
How to Become an Excellent IT-Research Institute
In Seminar at University of Auckland, 2010.Status: Published
How to Become an Excellent IT-Research Institute
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Seminar at University of Auckland |
How We Know What Isn't So. Common Myths in Business, Health and Daily Life. Why Do We Believe in Myths?
In Presentation at IT-Fornebu seminar, 2010.Status: Published
How We Know What Isn't So. Common Myths in Business, Health and Daily Life. Why Do We Believe in Myths?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Presentation at IT-Fornebu seminar |
Hvordan Estimering Av Ideell Tid Gjør Deg Mer Realistisk (med Innlagt NM I Estimering)
In Presentation at JavaZone, 2010.Status: Published
Hvordan Estimering Av Ideell Tid Gjør Deg Mer Realistisk (med Innlagt NM I Estimering)
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Presentation at JavaZone |
Hvordan Forbedre Estimering Av Tid Og Kostnader I IT-Prosjekter
In Presentation at TrygVesta seminar, 2010.Status: Published
Hvordan Forbedre Estimering Av Tid Og Kostnader I IT-Prosjekter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Presentation at TrygVesta seminar |
Identification and Management of IT-Projects With High Risk of Cost Overrun
In Presentation at Prepare's seminar for the IT-industry (and at an internal Bearing Point seminar), 2010.Status: Published
Identification and Management of IT-Projects With High Risk of Cost Overrun
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Presentation at Prepare's seminar for the IT-industry (and at an internal Bearing Point seminar) |
Jeg Ser Det Når Jeg Tror Det! Moter, Retorikk Og Systemutviklingsmetoder
In Seminar for Telenor, 2010.Status: Published
Jeg Ser Det Når Jeg Tror Det! Moter, Retorikk Og Systemutviklingsmetoder
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Seminar for Telenor |
Jeg Vet Ikke Hva Et ”story Point” Er, Men Det Virker Bra ...
In Presentation at DnD Software conference, 2010.Status: Published
Jeg Vet Ikke Hva Et ”story Point” Er, Men Det Virker Bra ...
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Presentation at DnD Software conference |
Research Quality. What Is It? How to Achieve It?
In Presentation at Telenor seminar, 2010.Status: Published
Research Quality. What Is It? How to Achieve It?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Presentation at Telenor seminar |
Software Development Effort Estimation: Why It Fails and How to Improve It
In Presentations at seminars at Orient, VietSoft and IMT, Vietnam, 2010.Status: Published
Software Development Effort Estimation: Why It Fails and How to Improve It
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Presentations at seminars at Orient, VietSoft and IMT, Vietnam |
Software Development Effort Estimation: Why It Fails and How to Improve It
In Seminar for the IT-industry in Auckland, organized by University of Auckland, 2010.Status: Published
Software Development Effort Estimation: Why It Fails and How to Improve It
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Seminar for the IT-industry in Auckland, organized by University of Auckland |
The Value of Empirical Software Engineering Research
In Presentation at Chalmers University, 2010.Status: Published
The Value of Empirical Software Engineering Research
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2010 |
Location of Talk | Presentation at Chalmers University |
Talks, invited
How to Become an Excellent IT-Research Institute (with Substantial Impact on the IT-Intensive Industry)
In Invited presentation at the Omar Dengo fundacion seminar: Research, Software Development, and Strategic Investments, Costa Rica, 2010.Status: Published
How to Become an Excellent IT-Research Institute (with Substantial Impact on the IT-Intensive Industry)
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2010 |
Location of Talk | Invited presentation at the Omar Dengo fundacion seminar: Research, Software Development, and Strategic Investments, Costa Rica |
Hvem Passer Offshoring For? Hva Er Viktig for å Lykkes?
In Presentation at CIO Forum (IDG), 2010.Status: Published
Hvem Passer Offshoring For? Hva Er Viktig for å Lykkes?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2010 |
Location of Talk | Presentation at CIO Forum (IDG) |
Identification and Management of IT-Projects With High Risk of Cost Overrun
In Presentation at Prepare's seminar for the IT-industry (and at an internal Bearing Point seminar), 2010.Status: Published
Identification and Management of IT-Projects With High Risk of Cost Overrun
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2010 |
Location of Talk | Presentation at Prepare's seminar for the IT-industry (and at an internal Bearing Point seminar) |
The Economic Benefits of Publicly Funded IT-Research. How Research and Education on Software Engineering Can Contribute to Costa Rica's ICT Capability
In Invited presentation at the Omar Dengo fundacion seminar: Software Engineering and Emerging Markets, Costa Rica, 2010.Status: Published
The Economic Benefits of Publicly Funded IT-Research. How Research and Education on Software Engineering Can Contribute to Costa Rica's ICT Capability
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2010 |
Location of Talk | Invited presentation at the Omar Dengo fundacion seminar: Software Engineering and Emerging Markets, Costa Rica |
Journal Article
Identification of More Risks Can Lead to Increased Over-Optimism of and Over-Confidence in Software Development Effort Estimates
Information and Software Technology 52 (2010): 506-516.Status: Published
Identification of More Risks Can Lead to Increased Over-Optimism of and Over-Confidence in Software Development Effort Estimates
Context: Software professionals are, on average, over-optimistic about the required effort usage and over-confident about the accuracy of their effort estimates. Objective: A better understanding of the mechanisms leading to the over-optimism and over-confidence may enable better estimation processes and, as a consequence, better managed software development projects. Method: We hypothesize that there are situations where more work on risk identification leads to increased over-optimism and over-confidence in software development effort estimates, instead of the intended improvement of realism. Four experiments with software professionals are conducted to test the hypothesis. Results: All four experiments provide results in support of the hypothesis. Possible explanations of the counter-intuitive finding relate to results from cognitive science on “illusion-of-control”, “cognitive accessibility”, “the peak-end rule” and “risk as feeling.” Conclusions: Thorough work on risk identification is essential for many purposes and our results should not lead to less emphasis on this activity. Our results do, however, suggest that it matters how risk identification and judgment-based effort estimation processes are combined. A simple approach for better combination of risk identification work and effort estimation is suggested.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2010 |
Journal | Information and Software Technology |
Volume | 52 |
Number | 5 |
Pagination | 506-516 |
Selection of Effort Estimation Strategies
Journal of Systems and Software 83 (2010): 1039-1050.Status: Published
Selection of Effort Estimation Strategies
We currently know little about the factors that motivate the selection and change of estimation strategy in judgment-based effort estimation context. A better understanding of these issues may lead to more accurate judgment-based effort estimates and motivates the four experiments reported in this paper. The experiments' two main results are the identification of the importance of “estimation surprises” (large estimation errors) to motivate estimation strategy change and the large individual variation in the initial choice of estimation strategy. The individual variation seems not only to be a result of differences in previous experiences, but also a result of differences in the mental “accessibility” of the strategies. We found, for example, that the use of a strategy was increased when we instructed a developer to use the same type of strategy on unrelated tasks immediately before. The laboratory contexts of the studies means that the results should be interpreted as a first step towards more knowledge about expert estimation strategies and that there is a strong need for more studies, preferably in field situations, before recommending actions on the basis of the findings.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2010 |
Journal | Journal of Systems and Software |
Volume | 83 |
Number | 6 |
Pagination | 1039-1050 |
The Effects of Request Formats on Judgment-Based Effort Estimation
Journal of Systems and Software 83 (2010): 29-36.Status: Published
The Effects of Request Formats on Judgment-Based Effort Estimation
In this paper we study the effects of a change from the traditional request “How much effort is required to complete X?” to the alternative “How much can be completed in Y work-hours?”. Studies 1 and 2 report that software professionals receiving the alternative format provided much lower, and presumably more optimistic, effort estimates of the same software development work than those receiving the traditional format. Studies 3 and 4 suggest that the effect belongs to the family of anchoring effects. An implication of our results is that project managers and clients should avoid the alternative estimation request format.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2010 |
Journal | Journal of Systems and Software |
Volume | 83 |
Number | 1 |
Pagination | 29-36 |
Talk, keynote
Moter, Myter Og Overforenklinger I IT-Bransjen
In Keynote at: Norsk informatikk-konferanse (NIK), 2010.Status: Published
Moter, Myter Og Overforenklinger I IT-Bransjen
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2010 |
Location of Talk | Keynote at: Norsk informatikk-konferanse (NIK) |
Journal Article
A Preliminary Study of Sequence Effects in Judgment-Based Software Development Work- Effort Estimation
IET Software 3 (2009): 435-441.Status: Published
A Preliminary Study of Sequence Effects in Judgment-Based Software Development Work- Effort Estimation
Context: Software development effort estimates are often inaccurate, and this inaccuracy cause problems for the clients as well as the providers. Consequently, we need more knowledge about the estimation processes, so that we can improve them. Objective: This study investigates how initial judgment-based estimation of work effort in software development affects subsequent, unrelated estimation work. Method: Fifty-six software professionals from the same company were allocated randomly to two groups. One group estimated the most likely effort required to complete a small software development task, while the other group estimated the effort required to complete a large task. After that, all the subjects estimated the effort required to complete the same medium-sized task. We replicated the experiment in another company (with 17 software professionals). Results: We found that sequence effects may have a strong impact on judgment-based effort estimates. Both in the first experiment and in the replication, the subsequent estimates were assimilated towards the subjects' initial estimate, i.e., the group that began with a small task supplied, on average, lower estimates of the medium-sized task than the group that began with the large task. Conclusion: Our findings suggest that knowledge about sequence effects may be important in order to improve estimation processes. However, currently we have a quite incomplete understanding of how, when and how much sequence effects affect effort estimation. Consequently, further research is needed.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2009 |
Journal | IET Software |
Volume | 3 |
Number | 5 |
Pagination | 435-441 |
DOI | 10.1049/iet-sen.2008.0110 |
How to Avoid Selecting Providers With Bids Based on Over-Optimistic Cost Estimates
IEEE Software Magazine 26 (2009): 79-84.Status: Published
How to Avoid Selecting Providers With Bids Based on Over-Optimistic Cost Estimates
It is well known that software development companies tend to produce over-optimistic cost estimates and that this over-optimism may lead to delivery problems for the clients as well as the providers. In this paper we summarize evidence suggesting that the clients can reduce the likelihood of selecting providers with bids based on over-optimistic cost estimates through their control of the bidding processes. Important means for this purpose include: avoid inviting many bidders when price is an important criterion for selection; avoid using price as an important criterion for selection when the ability to assess provider competence is low; apply bidding processes that ensure that the provider understands the complexity of the project; avoid budget or price expectation related information in the bidding material and avoid a negotiation process in which you ask for bid updates on reduced functionality. The evidence presented in this paper can also be used by software providers to identify bidding rounds where it is likely that they win only when strongly over-optimistic about the cost.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2009 |
Journal | IEEE Software Magazine |
Volume | 26 |
Number | 3 |
Pagination | 79-84 |
Software Development Effort Estimation: Formal Models Or Expert Judgment? (Viewpoints Article)
IEEE Software (2009): 14-19.Status: Published
Software Development Effort Estimation: Formal Models Or Expert Judgment? (Viewpoints Article)
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2009 |
Journal | IEEE Software |
Number | March/April |
Pagination | 14-19 |
The Impact of Lessons-Learned Sessions on Effort Estimation and Uncertainty Assessments
IEEE Transactions on Software Engineering 35 (2009): 368-383.Status: Published
The Impact of Lessons-Learned Sessions on Effort Estimation and Uncertainty Assessments
Inaccurate estimates of software development effort is a frequently reported cause of IT-project failures. We report results from a study that investigated the effect of introducing lessons learned sessions on estimation accuracy and the assessment of uncertainty. Twenty software professionals were randomly allocated to a Learning group or a Control group and instructed to estimate and complete the same five development tasks. Those in the Learning group, but not those in the Control group, were instructed to spend at least 30 minutes on identifying, analyzing, and summarizing their effort estimation and uncertainty assessment experience after completing each task. We found that the estimation accuracy and the realism of the uncertainty assessment were not better in the Learning group than in the Control group. A follow-up study with 83 software professionals was completed to better understand this lack of improvement from lessons-learned sessions. The follow-up study found that receiving feedback about other software professionals' estimation performance led to more realistic uncertainty assessments than receiving the same feedback of one's own estimates. Lessons-learned sessions, we argue, have to be carefully designed to avoid wasting resources on learning processes that stimulate rather than reduce learning biases related to assessment of own estimation performance.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2009 |
Journal | IEEE Transactions on Software Engineering |
Volume | 35 |
Number | 3 |
Pagination | 368-383 |
Public outreach
Åtti Prosent Av Systemutviklere Er Bedre Enn Gjennomsnittet
In Computerworld, 2009.Status: Published
Åtti Prosent Av Systemutviklere Er Bedre Enn Gjennomsnittet
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2009 |
Secondary Title | Computerworld |
Bindersmyten Og ”45% Av All Funksjonalitet Brukes Aldri”
In Computerworld, 2009.Status: Published
Bindersmyten Og ”45% Av All Funksjonalitet Brukes Aldri”
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2009 |
Secondary Title | Computerworld |
Jeg Ser Det Når Jeg Tror Det
In Computerworld, 2009.Status: Published
Jeg Ser Det Når Jeg Tror Det
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2009 |
Secondary Title | Computerworld |
Jeg Ser Det Når Jeg Tror Det! Moter, Retorikk Og Systemutviklingsmetoder
In Computerworld, 2009.Status: Published
Jeg Ser Det Når Jeg Tror Det! Moter, Retorikk Og Systemutviklingsmetoder
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2009 |
Secondary Title | Computerworld |
Jo Mer Jeg Tenker På Det, Desto Sikrere Blir Jeg
In Computerworld, 2009.Status: Published
Jo Mer Jeg Tenker På Det, Desto Sikrere Blir Jeg
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2009 |
Secondary Title | Computerworld |
Så Du Gorillaen?
In Computerworld, 2009.Status: Published
Så Du Gorillaen?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2009 |
Secondary Title | Computerworld |
Spillet Som Kun Har Tapere
In Computerworld, 2009.Status: Published
Spillet Som Kun Har Tapere
Afilliation | Software Engineering, Software Engineering |
Publication Type | Public outreach |
Year of Publication | 2009 |
Secondary Title | Computerworld |
Talks, contributed
Er IT-Utviklere Historieløse? Gjør Det Noe?
In Article in Computerworld Norway, 2009.Status: Published
Er IT-Utviklere Historieløse? Gjør Det Noe?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2009 |
Location of Talk | Article in Computerworld Norway |
Feillæring Skaper Overtro På Smidige Metoder
In Presentation at Smidig, 2009.Status: Published
Feillæring Skaper Overtro På Smidige Metoder
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2009 |
Location of Talk | Presentation at Smidig |
Hvordan Tilrettelegge for Realisme I Estimater Av Arbeidsmengde I IT-Prosjekter
In Presentation at LinPro seminar, 2009.Status: Published
Hvordan Tilrettelegge for Realisme I Estimater Av Arbeidsmengde I IT-Prosjekter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2009 |
Location of Talk | Presentation at LinPro seminar |
Myter Og Empiri (og Hvorfor Vi Aldri Vil Få Svaret På Om Smidige Metoder Virker)
In Presentation at a Bouvet seminar, 2009.Status: Published
Myter Og Empiri (og Hvorfor Vi Aldri Vil Få Svaret På Om Smidige Metoder Virker)
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2009 |
Location of Talk | Presentation at a Bouvet seminar |
Myter Og Empiri Innen Systemutvikling
In Presentation at DnD's Software Conference, 2009.Status: Published
Myter Og Empiri Innen Systemutvikling
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2009 |
Location of Talk | Presentation at DnD's Software Conference |
Realistisk Estimering Av Prosjekt, Release Og Iterasjon
In Presentation at an Ergo seminar, 2009.Status: Published
Realistisk Estimering Av Prosjekt, Release Og Iterasjon
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2009 |
Location of Talk | Presentation at an Ergo seminar |
Software Development Effort Estimation: What Do We Know About It?
In Presentation at PGS (Wroclaw) seminar, 2009.Status: Published
Software Development Effort Estimation: What Do We Know About It?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2009 |
Location of Talk | Presentation at PGS (Wroclaw) seminar |
Talk, keynote
How to Impact Software Engineering Practice Through Empirical Research
In Keynote at: EASE, 2009.Status: Published
How to Impact Software Engineering Practice Through Empirical Research
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2009 |
Location of Talk | Keynote at: EASE |
Book Chapter
Software Development Effort Estimation: Demystifying and Improving Expert Estimation
In Simula Research Laboratory - by thinking constantly about it, 381-404. Heidelberg: Springer, 2009.Status: Published
Software Development Effort Estimation: Demystifying and Improving Expert Estimation
Research background: Inaccurate estimation of software development effort is one of the most important reasons of IT-project failures. While too low effort estimates may lead to project management problems, delayed deliveries, budget overruns and low software quality, too high effort estimates may lead to lost business opportunities and inefficient use of resources. These IT-project problems motivated the BEST (Better Estimation of Software Tasks) project at Simula Research Laboratory to conduct research with the goal of improving effort estimation methods. Scientific challenges: The project's main focus is to improve judgment-based effort estimation (expert estimation), which is the estimation approach most frequently used by the software industry. We argue that a better understanding of the mental steps involved in expert estimation is necessary to achieve robust improvement of software development estimation processes. A great challenge when studying expert judgment is to understand the unconscious steps involved. To study these steps, we make use of multidisciplinary competencies, especially psychology and software engineering, and financial resources enabling studies in realistic software development effort estimation contexts. Obtained and expected results: In this section, we present the main results of the project from its beginnings in 2001 until today and describe how the project has benefited from being an integrated part of Simula Research Laboratory, especially through the opportunity to conduct large-scale, controlled experiments in field settings. The section includes practical results on how to improve estimation methods, scientific results leading to a better understanding of the mental processes involved in judgment-based effort estimation, and our innovative research methods in the field of empirical software engineering. Results of the BEST project are currently in use by several software companies, as well as by researchers in forecasting and psychology, and they are also included in project management and software engineering textbooks. The BEST project plans to continue its multidisciplinary effort with the goal of constructing and evaluating better models of the mental steps involved in judgment-based effort estimation. An improved model of these steps will enhance our ability to accurately estimate software development effort and to predict when different types of estimation methods can be expected to deliver accurate effort estimates. This will in turn lead to a reduction of the number of IT-project failures and make better use of scarce financial and human resources.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Book Chapter |
Year of Publication | 2009 |
Book Title | Simula Research Laboratory - by thinking constantly about it |
Chapter | 26 |
Pagination | 381-404 |
Publisher | Springer |
Place Published | Heidelberg |
ISBN Number | 978-3642011559 |
Proceedings, non-refereed
The Role of Judgment in Software Estimation
In ICSE 2009 Panel, 2009.Status: Accepted
The Role of Judgment in Software Estimation
The consequences of poor (or good) judgment in es-timation are reflected in software quality, cost, time-to-market, and operational reliability. While judgment plays a prominent role in aviation and medicine - the same care arguably needs to be taken in making trade-offs impacting software development. This panel will bring together practitioners to discuss aspects of judgment in software estimation including a discussion of current practices and influences from other disci-plines.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, non-refereed |
Year of Publication | 2009 |
Conference Name | ICSE 2009 Panel |
The role of judgment in software estimation
In International Conference on Software Engineering. Vol. Companion Volume. IEEE, 2009.Status: Published
The role of judgment in software estimation
The consequences of poor (or good) judgment in estimation are reflected in software quality, cost, time-tomarket, and operational reliability. While judgment plays a prominent role in aviation and medicine–the same care arguably needs to be taken in making tradeoffs impacting software development. This panel will bring togetherpractitioners to discuss aspects of judgment in software estimation including a discussion of current practices and influences from other disciplines.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, non-refereed |
Year of Publication | 2009 |
Conference Name | International Conference on Software Engineering |
Volume | Companion Volume |
Pagination | 13-17 |
Publisher | IEEE |
Talks, invited
Utvikling Av Programvare I Lavkostland
In Presentation at DnD seminar on outsourcing, 2009.Status: Published
Utvikling Av Programvare I Lavkostland
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, invited |
Year of Publication | 2009 |
Location of Talk | Presentation at DnD seminar on outsourcing |
Proceedings, refereed
A Preliminary Study of Sequence Effects in Judgment-Based Software Development Work- Effort Estimation
In Proceedings, 12th International Conference on Evaluation and Assessment in Software Engineering. University of Bari, Italy, 2008.Status: Published
A Preliminary Study of Sequence Effects in Judgment-Based Software Development Work- Effort Estimation
Context: Software development effort estimates are often inaccurate, and this inaccuracy cause problems for the clients as well as the providers. Consequently, we need more knowledge about the estimation processes, so that we can improve them. Objective: This study investigates how initial judgment-based estimation of work effort in software development affects subsequent, unrelated estimation work. Method: Fifty-six software professionals from the same company were allocated randomly to two groups. One group estimated the most likely effort required to complete a small software development task, while the other group estimated the effort required to complete a large task. After that, all the subjects estimated the effort required to complete the same medium-sized task. We replicated the experiment in another company (with 17 software professionals). Results: We found that sequence effects may have a strong impact on judgment-based effort estimates. Both in the first experiment and in the replication, the subsequent estimates were assimilated towards the subjects' initial estimate, i.e., the group that began with a small task supplied, on average, lower estimates of the medium-sized task than the group that began with the large task. Conclusion: The results of this study suggest that knowledge about sequence effects may be important in order to improve estimation processes. However, currently we have a quite incomplete understanding of how, when and how much sequence effects affect effort estimation. Consequently, further research is needed.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2008 |
Conference Name | Proceedings, 12th International Conference on Evaluation and Assessment in Software Engineering |
Publisher | University of Bari, Italy |
Judgment-Updating Among Software Professionals
In The 2nd international conference on software knowledge information management and applications (SKIMA). SKIMA conference organising committee, 2008.Status: Published
Judgment-Updating Among Software Professionals
Initial judgments related to key decisions in software projects are often based on one-sided or misleading information. The initial assessment of the benefits of introducing a new development tool may for example be based a vendor's sales demonstration or a reference client's favorable description. In this paper we study software professionals' abilities to adjust their early, biased judgments when receiving contradicting or less biased information. The first study, involving 160 software professionals, found a strong under-adjustment for the impact of misleading information and one-sided argument. A follow-up two weeks later found that this under-adjustment was not removed over time. The second study, involving 65 software professionals, found that the ability to update biased judgments may sometimes be quite good, but that it is hard to predict when. A practical consequence of our results is that software professionals should strongly emphasize the avoidance of biased and potentially misleading information and not trust that they are able to adjust their judgments and beliefs when more reliable and unbiased information are available.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2008 |
Conference Name | The 2nd international conference on software knowledge information management and applications (SKIMA) |
Pagination | 62-67 |
Publisher | SKIMA conference organising committee |
ISBN Number | 9781851432516 |
Talks, contributed
Avoiding Impact From Irrelevant and Misleading Information
In Presentation at Chalmers University, Gøteborg, 2008.Status: Published
Avoiding Impact From Irrelevant and Misleading Information
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at Chalmers University, Gøteborg |
Best for Hvem
In Article in Computerworld Norge, 2008.Status: Published
Best for Hvem
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Article in Computerworld Norge |
Hjelper Smidige Metoder?
In Article in Computerworld Norge, 2008.Status: Published
Hjelper Smidige Metoder?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Article in Computerworld Norge |
How to Avoid Impact From Irrelevant and Misleading Information on Your Cost Estimates
In Presentation at Itestra Seminar (München), 2008.Status: Published
How to Avoid Impact From Irrelevant and Misleading Information on Your Cost Estimates
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at Itestra Seminar (München) |
Human Biases in Forecasting of Work-Effort: Differences in Effect Sizes in Laboratory and Field Settings
In Presentation at International Symposium on Forecasting, 2008.Status: Published
Human Biases in Forecasting of Work-Effort: Differences in Effect Sizes in Laboratory and Field Settings
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at International Symposium on Forecasting |
Hvorfor Læring Av Erfaring Er Vanskelig Og Hvordan Bli Bedre
In Presentation at JavaZone, 2008.Status: Published
Hvorfor Læring Av Erfaring Er Vanskelig Og Hvordan Bli Bedre
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at JavaZone |
Kritisk Tenkning
In Article in Computerworld Norge, 2008.Status: Published
Kritisk Tenkning
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Article in Computerworld Norge |
Kulturforskjeller Og Offshoring-Prosjekter
In Article in Computerworld Norge, 2008.Status: Published
Kulturforskjeller Og Offshoring-Prosjekter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Article in Computerworld Norge |
Kulturforskjeller Og Outsourcing Av IT-Prosjekter Inni Er Vi Like - Eller Er Vi Egentlig Det?
In Presentation at Simula's Industry Seminar, 2008.Status: Published
Kulturforskjeller Og Outsourcing Av IT-Prosjekter Inni Er Vi Like - Eller Er Vi Egentlig Det?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at Simula's Industry Seminar |
Læring Av Erfaring
In Presentation at Nordnet (project management conference), 2008.Status: Published
Læring Av Erfaring
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at Nordnet (project management conference) |
Peer, Du Lyver! Nei, Jeg Gjør Ei!
In Article in Computerworld Norge, 2008.Status: Published
Peer, Du Lyver! Nei, Jeg Gjør Ei!
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Article in Computerworld Norge |
Priming
In Article in Computerworld Norge, 2008.Status: Published
Priming
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Article in Computerworld Norge |
Prinsipper for Estimering Av Utviklingskostnader I IT-Prosjekter
In Keynote at Geiloseminaret 2008, 2008.Status: Published
Prinsipper for Estimering Av Utviklingskostnader I IT-Prosjekter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Keynote at Geiloseminaret 2008 |
Rasjonalitetens Makt Og Maktens Rasjonalitet
In Presentation at Objectnet seminar (Dublin), 2008.Status: Published
Rasjonalitetens Makt Og Maktens Rasjonalitet
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at Objectnet seminar (Dublin) |
Sequence Effects in Judgment-Based Software Development Work-Effort Forecasting
In Proceedings of the 28th International Symposium on Forecasting, 2008.Status: Published
Sequence Effects in Judgment-Based Software Development Work-Effort Forecasting
Accurate forecasts of work-effort are essential to enable successful software development projects. Such forecasts are typically judgment-based and inaccurate. One important reason for the inaccuracy may be due to the “sequence effect”, i.e., that initial software development work-effort forecasts may affect subsequent, even unrelated, judgment-based forecasting work. If, for example, the previously forecasted development task was small, there seems to be a tendency to under-estimate the work-effort of the following task. We report sequence effects found in the laboratory and in the field. In a laboratory-based experiment, we randomly divided 56 software professionals into two groups. One group started with work-effort forecasts of a small and the other of a larger software development task. Subsequently, all the software professionals were asked to forecast the work-effort of the same medium-sized task. We found that the forecasts of the medium-sized tasks were assimilated towards the initial forecasts, i.e., the group that initially forecasted a small task submitted, on average, lower forecasts of the medium-sized task than the group that initially forecasted a larger task. We found similar results in a field experiment. Forty software companies from Asia and Eastern Europe were hired to forecast the work-effort of the same five software development projects. The companies that started with forecasts of the largest projects submitted, on average, the highest forecasts of the other projects. The typical approach for software development work-effort forecasting is based on a decomposition of projects into sub-tasks. A possible consequence of our results is that software professionals should start with forecasts of medium complex, medium sized sub-tasks of the project, or, with large and complex task if there is a strong tendency towards over-optimistic forecasts.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Proceedings of the 28th International Symposium on Forecasting |
Notes | Talk at ISF 2008, Nice, France (June 2008) |
Software Development Effort Estimation: Why It Fails and How to Improve It
In Presentation at D2Hawkeye (Nepal) seminar, 2008.Status: Published
Software Development Effort Estimation: Why It Fails and How to Improve It
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at D2Hawkeye (Nepal) seminar |
Software Development Effort Estimation: Why It Fails and How to Improve It
In Presentation at Silverstone (India) Seminar, 2008.Status: Published
Software Development Effort Estimation: Why It Fails and How to Improve It
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at Silverstone (India) Seminar |
Software Development Effort Estimation: Why It Fails and How to Improve It
In Presenation at TMA Solution (Vietnam) seminar, 2008.Status: Published
Software Development Effort Estimation: Why It Fails and How to Improve It
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presenation at TMA Solution (Vietnam) seminar |
Software Project Planning and Management: Why It Fails and How to Improve It
In Presentation at Kathmandu University, 2008.Status: Published
Software Project Planning and Management: Why It Fails and How to Improve It
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at Kathmandu University |
The Impact of Irrelevant and Misleading Information on Software Development Effort Estimation: a Randomized, Controlled Field Experiment of Forty-Six Outsourcing Companies
In Presentation at IT University Copenhagen, 2008.Status: Published
The Impact of Irrelevant and Misleading Information on Software Development Effort Estimation: a Randomized, Controlled Field Experiment of Forty-Six Outsourcing Companies
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at IT University Copenhagen |
When Should We Trust Expert Judgment in Software Development?
In Presentation at Technische Universität München, 2008.Status: Published
When Should We Trust Expert Judgment in Software Development?
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Presentation at Technische Universität München |
Økt Psykologisk Distanse Fører Til økt Optimisme
In Article in Computerworld Norge, 2008.Status: Published
Økt Psykologisk Distanse Fører Til økt Optimisme
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talks, contributed |
Year of Publication | 2008 |
Location of Talk | Article in Computerworld Norge |
Journal Article
Avoiding Irrelevant and Misleading Information When Estimating Development Effort
IEEE Software (2008): 78-83.Status: Published
Avoiding Irrelevant and Misleading Information When Estimating Development Effort
Software development effort estimates are reported to be highly inaccurate and systematically over-optimistic. We provide empirical evidence that suggests that this problem is caused, to some extent, by the influence of irrelevant and misleading information, e.g., information about the client's budget, present in the estimation material. The only really effective means of eliminating this influence is to avoid exposure to this type of information. Other means, such as more use of formal effort estimation models, improved analysis of requirement specifications, and better selection of estimators, had a positive effect but did not eliminate the influence. We propose process elements that are designed to avoid irrelevant and misleading information and illustrate how this process may lead to more realism in effort estimation.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2008 |
Journal | IEEE Software |
Number | May/June |
Pagination | 78-83 |
The Impact of Irrelevant Information on Estimates of Software Development Effort
Journal of Systems and Software (2008).Status: Submitted
The Impact of Irrelevant Information on Estimates of Software Development Effort
Software professionals typically estimate software development effort on the basis of a requirement specification. Parts of this specification frequently contain information that is irrelevant to the estimation of the actual effort involved in the development of software. We hypothesize that effort-irrelevant information sometimes has a strong impact on effort estimates. To test this hypothesis, we conducted three controlled experiments with software professionals. In each of the experiments, the software professionals received specifications describing the same requirements. However, we gave one group of the software professionals a version of the requirement specification where we had included additional, effort-irrelevant, information. In all three experiments, we observed that the estimates of most likely effort increased when the estimates were based on requirement specifications that contained the information irrelevant to development effort. The results suggest that when estimation-irrelevant information is included as input to expert judgment-based estimation processes, the estimators find it difficult to ignore it. The results also show that it is difficult to predict the impact of estimation-irrelevant information and that software professionals seem to be unaware of the impact. One possible (and advisable) course of action, given our findings, would be to remove estimation-irrelevant information from the requirement specification prior to the use of it as input to estimation work.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2008 |
Journal | Journal of Systems and Software |
The Role of Artificial Design Elements in Software Engineering Experiments
IEEE Transactions on Software Engineering 34 (2008): 242-259.Status: Published
The Role of Artificial Design Elements in Software Engineering Experiments
Increased realism in software engineering experiments is often promoted as an important means to increase generalizability and industrial relevance. In this context, artificiality, e.g., the use of constructed tasks in place of realistic tasks, is seen as a threat. In this article, we examine the opposite view, that deliberately introduced artificial design elements may increase knowledge gain and enhance both generalizability and relevance. In the first part of the article, we identify and evaluate arguments and examples in favor of, and against, deliberately introducing artificiality into software engineering experiments. In the second part of the article, we summarize a content analysis of articles reporting software engineering experiments published over the ten-year period 1993-2002. The analysis reveals a striving for realism and external validity, but little awareness of for what and when, various degrees of artificiality and realism are appropriate. We conclude that an increased awareness and deliberation in these respects is essential. However, arguments in favor of artificial design elements should not be used to justify studies that are badly designed or that have research questions of low relevance.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2008 |
Journal | IEEE Transactions on Software Engineering |
Volume | 34 |
Number | 2 |
Pagination | 242-259 |
Date Published | March/April |
The Role of Outcome Feedback in Improving the Uncertainty Assessment of Software Development Effort Estimates
ACM Transactions on Software Engineering and Methodology 17 (2008): 20-35.Status: Published
The Role of Outcome Feedback in Improving the Uncertainty Assessment of Software Development Effort Estimates
To enable properly-sized software project budgets and plans it is important to be able to assess the uncertainty of the estimates of the most likely effort required to complete the projects. Previous studies show that software professionals tend to be too optimistic about the uncertainty of their effort estimates, i.e., they tend to be overconfident. This paper reports the results from a study on the role of outcome feedback in the uncertainty assessment learning process. Software developers were given repeated and immediate outcome feedback about the discrepancy between the estimated most likely effort and the actual effort. We found that one condition for the improvement of uncertainty assessments of effort estimates may be the use of explicitly-formulated strategies for uncertainty assessment. By contrast, intuition-based uncertainty assessment strategies may lead to no or little improvement. This has the implication that we cannot expect much improvement in uncertainty assessment unless we train and instruct software developers to use the previous estimation accuracy experience as input to an explicitly defined uncertainty assessment process.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Journal Article |
Year of Publication | 2008 |
Journal | ACM Transactions on Software Engineering and Methodology |
Volume | 17 |
Number | 4 |
Pagination | 20-35 |
Talk, keynote
Prinsipper for estimering av utviklingskostnader i IT-prosjekter
In Geiloseminaret, Geilo, 2008.Status: Published
Prinsipper for estimering av utviklingskostnader i IT-prosjekter
Afilliation | Software Engineering, Software Engineering |
Publication Type | Talk, keynote |
Year of Publication | 2008 |
Location of Talk | Geiloseminaret, Geilo |
Proceedings, refereed
A Critique of How We Measure and Interpret the Accuracy of Software Development Effort Estimation
In 1st International Workshop on Software Productivity Analysis and Cost Estimation. Information Processing Society of Japan, 2007.Status: Published
A Critique of How We Measure and Interpret the Accuracy of Software Development Effort Estimation
This paper criticizes current practice regarding the measurement and interpretation of the accuracy of software development effort estimation. The shortcomings we discuss are related to: 1) the meaning of {`}effort estimate', 2) the meaning of {`}estimation accuracy', 3) estimation of moving targets, and 4) assessment of the estimation process, and not only the discrepancy between the estimated and the actual effort, to evaluate estimation skill. It is possible to correct several of the discussed shortcomings by better practice. However, there are also inherent problems related to both laboratory and field analyses of the accuracy of software development effort estimation. It is essential that both software researchers and professionals are aware of these problems and their implications for the analysis of the measurement of effort estimation accuracy.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2007 |
Conference Name | 1st International Workshop on Software Productivity Analysis and Cost Estimation |
Pagination | 15-22 |
Publisher | Information Processing Society of Japan |
ISBN Number | ISBN 978-4-915256-72-1 C3040 |