Publications
Journal Article
Determining a Core View of Research Quality in Empirical Software Engineering
Empirical Software Engineering (2021).Status: Submitted
Determining a Core View of Research Quality in Empirical Software Engineering
Context: Research quality is intended to appraise the design and reporting of studies. It comprises a set of concepts such as methodological rigor, practical relevance, and conformance to ethical standards. Depending on the perspective, different views of importance are given to the conceptual dimensions of research quality.
Objective: We intend to assess the level of alignment between researchers with regard to a conceptual model of research quality. This includes aligning the definition of research quality and reasoning on the relative importance of quality characteristics.
Method: We conducted a mixed-methods approach with two distinct group perspectives: (i) a research group; and (ii) the empirical software engineering research community. Our data collection approach comprised a questionnaire survey and a complementary focus group. We carried out a hierarchical voting prioritization to collect relative values for importance.
Results: In the context of this research, ‘internally valid’, ‘relevant research idea’, and ‘applicable results’ are perceived as the core dimensions of quality. The alignment at the research group level was higher compared to that at the community level.
Conclusion: The interdisciplinary model of research quality was seen to express fairly the quality of research in the software engineering context. It presented limitations regarding its structure and components' description, which resulted in an updated model.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Empirical Software Engineering |
Publisher | Springer |
Experiential Learning Approach for Software Engineering Courses at Higher Education Level
Journal of Computing in Higher Education (2021).Status: Submitted
Experiential Learning Approach for Software Engineering Courses at Higher Education Level
Background: Software project management activities help to introducing software process models in Software Engineering courses. However, these activities should be adequately aligned with the learning outcomes and support student's progression.
Objective: Present and evaluate an approach to help students acquire theoretical and practical knowledge and experience real-world software projects' challenges. The approach combines a serious game and a design-implement task in which students develop a controlled-scale software system.
Methods: To evaluate our approach, we analyzed the students' perceptions collected through an online survey, their project plans, and their final reports using thematic analysis.
Results: Results suggest that the approach promotes knowledge acquisition, enables students' progression, reinforces theoretical concepts, and is properly aligned with the course's learning outcomes.
Conclusion: The approach seems to help introducing software process models in Software Engineering courses. Our experience can also be inspiring for educators willing to apply our approach in similar courses.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Journal of Computing in Higher Education |
Publisher | Springer |
Risks and risk mitigation in global software development: an update
Journal of Software: Evolution and Process (2021).Status: Submitted
Risks and risk mitigation in global software development: an update
Context: Interest in global software development (GSD) has led to the publication of numerous studies. Over time, these studies should be updated to verify if their findings and conclusions remain valid.
Objective: To update a tertiary study, published in 2014, focused on investigating risks and risk mitigation advice in the context of GSD.
Method: We conducted a systematic literature study based on forward snowballing, out of which we identified and selected 25 unique studies.
Results: We extracted: a) 122 risks (75 of which already identified in the original work), and b) 123 mitigation advice (44 of which were identified in the original work). The evidence supporting these risks and mitigation were extracted from 619 and 389 primary sources, respectively. Given the amount of evidence found, we have reported only those with higher empirical support. The raw data, including all the values obtained, are available online as complementary material.
Conclusions: Interest in GSD, its potential risks, and possible mitigation strategies, remains high. This update has allowed increasing the level of empirical support provided by the findings of the original work.
Afilliation | Software Engineering |
Project(s) | Department of IT Management, EDOS: Effective Digitalization of Public Sector |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Journal of Software: Evolution and Process |
Publisher | John Wiley & Sons Ltd |
Journal Article
An empirically evaluated checklist for surveys in software engineering
Information and Software Technology 119 (2020): 106240.Status: Published
An empirically evaluated checklist for surveys in software engineering
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Information and Software Technology |
Volume | 119 |
Pagination | 106240 |
Date Published | Jan-03-2020 |
Publisher | Elsevier |
ISSN | 09505849 |
URL | https://doi.org/10.1016/j.infsof.2019.106240 |
DOI | 10.1016/j.infsof.2019.106240 |
Journal Article
CERSE - Catalog for empirical research in software engineering: A Systematic mapping study
Information and Software Technology 105 (2019): 117-149.Status: Published
CERSE - Catalog for empirical research in software engineering: A Systematic mapping study
Context Empirical research in software engineering contributes towards developing scientific knowledge in this field, which in turn is relevant to inform decision-making in industry. A number of empirical studies have been carried out to date in software engineering, and the need for guidelines for conducting and evaluating such research has been stressed.
Objective: The main goal of this mapping study is to identify and summarize the body of knowledge on research guidelines, assessment instruments and knowledge organization systems on how to conduct and evaluate empirical research in software engineering.
Method: A systematic mapping study employing manual search and snowballing techniques was carried out to identify the suitable papers. To build up the catalog, we extracted and categorized information provided by the identified papers.
Results: The mapping study comprises a list of 341 methodological papers, classified according to research methods, research phases covered, and type of instrument provided. Later, we derived a brief explanatory review of the instruments provided for each of the research methods.
Conclusion: We provide: an aggregated body of knowledge on the state of the art relating to guidelines, assessment instruments and knowledge organization systems for carrying out empirical software engineering research; an exemplary usage scenario that can be used to guide those carrying out such studies is also provided. Finally, we discuss the catalog’s implications for research practice and the needs for further research.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Information and Software Technology |
Volume | 105 |
Pagination | 117 - 149 |
Date Published | Jan-01-2019 |
Publisher | Elsevier |
ISSN | 09505849 |
Keywords | Empirical methods, Empirical research, Mapping study |
URL | https://linkinghub.elsevier.com/retrieve/pii/S0950584917301118https://ap... |
DOI | 10.1016/j.infsof.2018.08.008 |
Proceedings, refereed
Experiences of studying Attention through EEG in the Context of Review Tasks
In EASE '19: Proceedings of the Evaluation and Assessment on Software Engineering. New York, NY, USA: ACM Press, 2019.Status: Published
Experiences of studying Attention through EEG in the Context of Review Tasks
Context: Electroencephalograms (EEG) have been used in a few cases in the context of software engineering (SE). EEGs allow capturing emotions and cognitive functioning. Such human factors have already shown to be important to understand software engineering tasks. Therefore, it is essential to gain experience in the community to utilize EEG as a research tool. Objective: To report experiences of using EEG in the context of a software engineering education (review of master theses proposals). We provide our reflections and lessons learned of (1) how to plan an EEG study, (2) how to conduct and execute (e.g., tools), (3) how to analyze. Method: We carried out an experiment using an EEG headset to measure the participants' attention rate. The experiment task includes reviewing three master thesis project plans. Results: We describe how we evolved our understanding of experimentation practices to collect and analyze psychological and cognitive data. We also provide a set of lessons learned regarding the application of EEG technology for research. Conclusions: We believe that that EEG could benefit software engineering research to collect cognitive information under certain conditions. The lessons learned reported here should be used as inputs for future experiments in software engineering, where human aspects are of interest.
Afilliation | Software Engineering |
Project(s) | Department of IT Management, Simula Metropolitan Center for Digital Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | EASE '19: Proceedings of the Evaluation and Assessment on Software Engineering |
Pagination | 313-318 |
Publisher | ACM Press |
Place Published | New York, NY, USA |
ISBN Number | 9781450371452 |
Keywords | attention, experiment, human subjects, lectroencephalogram |
URL | http://dl.acm.org/citation.cfm?doid=3319008http://dl.acm.org/citation.cf... |
DOI | 10.1145/331900810.1145/3319008.3319357 |
A Legacy Game for Project Management in Software Engineering Courses
In the 3rd European Conference of Software Engineering EducationProceedings of the 3rd European Conference of Software Engineering Education on ZZZ - ECSEE'18. Seeon/ Bavaria, GermanyNew York, New York, USA: ACM Press, 2018.Status: Published
A Legacy Game for Project Management in Software Engineering Courses
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type |
Teaching students critical appraisal of scientific literature using checklists
In the 3rd European Conference of Software Engineering EducationProceedings of the 3rd European Conference of Software Engineering Education on ZZZ - ECSEE'18. Seeon/ Bavaria, GermanyNew York, New York, USA: ACM Press, 2018.Status: Published
Teaching students critical appraisal of scientific literature using checklists
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type |
Journal Article
Towards understanding the relation between citations and research quality in software engineering studies
Scientometrics 117 (2018): 1453-1478.Status: Published
Towards understanding the relation between citations and research quality in software engineering studies
The importance of achieving high quality in research practice has been highlighted in different disciplines. At the same time, citations are utilized to measure the impact of academic researchers and institutions. One open question is whether the quality in the reporting of research is related to scientific impact, which would be desired. In this exploratory study we aim to: (1) Investigate how consistently a scoring rubric for rigor and relevance has been used to assess research quality of software engineering studies; (2) Explore the relationship between rigor, relevance and citation count. Through backward snowball sampling we identified 718 primary studies assessed through the scoring rubric. We utilized cluster analysis and conditional inference tree to explore the relationship between quality in the reporting of research (represented by rigor and relevance) and scientiometrics (represented by normalized citations). The results show that only rigor is related to studies’ normalized citations. Besides that, confounding factors are likely to influence the number of citations. The results also suggest that the scoring rubric is not applied the same way by all studies, and one of the likely reasons is because it was found to be too abstract and in need to be further refined. Our findings could be used as a basis to further understand the relation between the quality in the reporting of research and scientific impact, and foster new discussions on how to fairly acknowledge studies for performing well with respect to the emphasized research quality. Furthermore, we highlighted the need to further improve the scoring rubric.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | Scientometrics |
Volume | 117 |
Pagination | 1453 - 1478 |
Date Published | Jan-12-2018 |
Publisher | Springer |
ISSN | 0138-9130 |
Keywords | Conditional inference tree, empirical software engineering, Exploratory study, Reporting of research, Research practice, Scientific impact |
URL | http://link.springer.com/10.1007/s11192-018-2907-3http://link.springer.c... |
DOI | 10.1007/s11192-018-2907-3 |
Proceedings, refereed
Perception of Scientific Evidence: Do Industry and Academia Share an Understanding?
In Lärarlärdom 2016. Högskolan Kristianstad, 2016.Status: Published
Perception of Scientific Evidence: Do Industry and Academia Share an Understanding?
Context: Collaboration depends on communication and upon having a similar understanding of the notions that are being discussed, and a similar appraisal of their value. Existing work seems to show that the collaboration between industry and academia is hampered by a difference in values. In particular, academic work focuses more on generalizing on the basis of existing evidence, while industry prefers to particularize conclusions to individual cases. This has lead to the conclusion that industry values scientific evidence less than academia does.
Objective: This paper seeks to re-evaluate that conclusion, and investigate if industry and academia share a definition of scientific evidence. If evidence can be found of competing views, we propose a more finely grained model of empirical evidence and its role in building software engineering knowledge. Moreover, we seek to determine if a more nuanced look the notion of scientific evidence has an influence on how academics and industry practitioners perceive that notion.
Method: We have developed a model of key concepts related to under- standing empirical evidence in software engineering. An initial validation has been conducted, consisting of a survey of master students, to deter- mine if competing views of evidence exist at that level. The model will be validated by further literature study and semi- structured interviews with industry practitioners.
Results: We propose a model of empirical evidence in software engineering, and an initial validation of that model by means of a survey. The results of the survey indicate that conflicting opinions already exist in the student body regarding the notion of evidence, how trustworthy different sources of evidence and knowledge are, and which sources of evidence and types of evidence are more appropriate in various situations. Conclusion: Rather than a difference in how industry and academia value scientific evidence, we see evidence of misunderstanding, of different notions of what constitutes scientific evidence and what strength of evidence is required to achieve specific goals. We propose a model of empirical evidence, to provide a better understanding of what is required in various situations and a better platform for communication between industry and academia.
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Lärarlärdom 2016 |
Publisher | Högskolan Kristianstad |
Proceedings, refereed
SESRA: a web-based automated tool to support the systematic literature review process
In the 19th International ConferenceProceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering - EASE '15. Nanjing, ChinaNew York, New York, USA: ACM Press, 2015.Status: Published
SESRA: a web-based automated tool to support the systematic literature review process
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | the 19th International ConferenceProceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering - EASE '15 |
Publisher | ACM Press |
Place Published | Nanjing, ChinaNew York, New York, USA |
ISBN Number | 9781450333504 |
URL | http://dl.acm.org/citation.cfm?doid=2745802http://dl.acm.org/citation.cf... |
DOI | 10.1145/274580210.1145/2745802.2745825 |
Proceedings, refereed
Proposal of an Automated Approach to Support the Systematic Review of Literature Process {(S)
In The 25th International Conference on Software Engineering and Knowledge Engineering, Boston, MA, USA, June 27-29, 2013. Knowledge Systems Institute Graduate School, 2013.Status: Published
Proposal of an Automated Approach to Support the Systematic Review of Literature Process {(S)
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | The 25th International Conference on Software Engineering and Knowledge Engineering, Boston, MA, USA, June 27-29, 2013 |
Pagination | 488–493 |
Publisher | Knowledge Systems Institute Graduate School |
Proceedings, refereed
Automated Approaches to Support Secondary Study Processes: a Systematic Review
In Proceedings of the 24th International Conference on Software Engineering {&} Knowledge Engineering (SEKE'2012), Hotel Sofitel, Redwood City, San Francisco Bay, {USA} July 1-3, 2012. Knowledge Systems Institute, 2012.Status: Published
Automated Approaches to Support Secondary Study Processes: a Systematic Review
Afilliation | Software Engineering |
Project(s) | Department of IT Management |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | Proceedings of the 24th International Conference on Software Engineering {&} Knowledge Engineering (SEKE'2012), Hotel Sofitel, Redwood City, San Francisco Bay, {USA} July 1-3, 2012 |
Pagination | 143–147 |
Publisher | Knowledge Systems Institute |