Predictions of time (e.g., work hours) are often based on the aggregation of estimates of elements (e.g., activities, subtasks). The only types of estimates that can be safely aggregated by summation are those reflecting predicted average outcomes (expected values). The sums of other types of estimates, such as bounds of confidence intervals or estimates of the mode, do not have the same interpretation as their components (e.g., the sum of the 90% upper bounds is not the appropriate 90% upper bound of the sum). This can be a potential source of bias in predictions of time, as shown in Studies 1 and 2, where professionals with experience in estimation provided total estimates of time that were inconsistent with their estimates of individual tasks. Study 3 shows that this inconsistency can be attributed to improper aggregation of time estimates and demonstrates how this can produce both over- and underestimation—and also time prediction intervals that are far too wide. Study 4 suggests that the results may reflect a more general fallacy in the aggregation of probabilistic quantities. Our observations are consistent with that inconsistencies and biases are driven by a tendency towards applying a naïve summation (2+2=4) of probabilistic (stochastic) values, in situations where this is not appropriate. This summation fallacy may be in particular consequential in a context where informal estimation methods (expert-judgment based estimation) are used.

Context: To ensure the success of software projects, it is essential to select skilled developers and to use suitable work contracts. Objective: This study tests two hypotheses: (i) the use of work-sample testing (trialsourcing) improves the selection of skilled software developers; and (ii) the use of contracts based on hourly payment leads to better software project outcomes than fixed-price contracts. Method: Fifty-seven software freelancers with relevant experience and good evaluation scores from previous clients were invited to complete a two-hour long trialsourcing task to qualify for a software development project. Thirty-six developers completed the trialsourcing task with acceptable performance, and, based on a stratified allocation process, were asked to give a proposal based on an hourly payment or a fixed-price contract. Eight hourly payment-based and eight fixed-priced proposals were accepted. The process and product characteristics of the completion of these 16 projects were collected and analysed. Results and Conclusion: Only partial support for our hypotheses was observed. While the use of trialsourcing may have prevented the selection of developers with insufficient skills, the performance on the trialsourcing task of the selected developers did not predict performance on the project. The use of hourly payments led to lower costs than fixed-price contracts, but not to improved processes or products. We plan to follow up these, to us unexpected, results with research on how to design more skill-predictive trialsourcing tasks, and when and why different project contexts give different contract consequences.