In this paper, we propose improvements in how estimation bias, e.g., the tendency towards under-estimating the effort, is measured. The proposed approach emphasizes the need to know what the estimates are meant to represent, i.e., the type of estimate we evaluate and the need for a match between the type of estimate given and the bias measure used. We show that even perfect estimates of the mean effort will not lead to an expectation of zero estimation bias when applying the frequently used bias measure: (actual effort – estimated effort)/actual effort. This measure will instead reward under-estimates of the mean effort. We also provide examples of bias measures that match estimates of the mean and the median effort, and argue that there are, in general, no practical bias measures for estimates of the most likely effort. The paper concludes with implications for the evaluation of bias of software development effort estimates.

Background: In order to judge evidence it is important to be able to assess study quality. Checklists are means to objectify the assessment. In an earlier study we proposed and evaluated a checklist for surveys, which was assessed by experts.

Objective: (1) To assess whether the use of the checklist enables students with limited experience in research to consistently and accurately assess the quality of a research paper. (2) To elicit qualitative feedback to identify improvements to the checklist.

Method: The students reviewed a survey in a one-group posttest-only quasi-experiment using the checklist. In total 13 students participated in the context of the course Evidence-based software engineering as part of the study program Information Systems at Flensburg University of Applied Sciences.

Results: In total the students achieved 74% percent of agreement among each other. However, the Kappa values indicated mostly a poor level of agreement considering the classification by Fleiss. In addition, the students were quite inaccurate assessing the questions. Though, they performed well on questions for research objectives and the identification of population.

Conclusion: Findings indicate that students do not assess reliably. However, further investigations are needed to substantiate the findings.

Considerable resources are wasted on projects that deliver few or no benefits. The main objective is to better understand the characteristics of projects that are successful in delivering good client benefits. We asked 71 Norwegian software professionals to report information about projects completed between 2016 and 2018. We found that both benefits management and agile practices have a significant relationship with perceived realisation of client benefits. This includes the benefits management practices of having a plan for benefits realisation, individuals with assigned responsibility for benefits realisation, benefits management during project execution, quantification of realised benefits, evaluation of realised benefits, re-estimation of benefits during project execution, and the agile practices of a flexible scope and frequent deliveries to production. The software projects that were successful in delivering client benefits adopted benefits management and agile practices to a larger extent than the less successful ones. Future studies are required to establish more comprehensive understanding of what distinguishes projects that deliver good client benefits from the rest, including studies of the realisation of client benefits in agile software projects.