Publications
Proceedings, refereed
Discovering Gateway Ports in Maritime Using Temporal Graph Neural Network Port Classification
In 35th Canadian Conference on Artificial Intelligence, 2022.Status: Accepted
Discovering Gateway Ports in Maritime Using Temporal Graph Neural Network Port Classification
Vessel navigation is influenced by various factors, such as dynamic environmental factors that change over time or static features such as vessel type or depth of the ocean. These dynamic and static navigational factors impose limitations on vessels, such as long waiting times in regions outside the actual ports, and we call these waiting regions gateway ports. Identifying gateway ports and their associated features such as congestion and available utilities can enhance vessel navigation by planning on fuel optimization or saving time in cargo operation. In this paper, we propose a novel temporal graph neural network (TGNN) based port classification method to enable vessels to discover gateway ports efficiently, thus optimizing their operations. The proposed method processes vessel trajectory data to build dynamic graphs capturing spatio-temporal dependencies between a set of static and dynamic navigational features in the data, and it is evaluated in terms of port classification accuracy on a real-world data set collected from ten vessels operating in Halifax, NS, Canada. The experimental results indicate that our TGNN-based port classification method provides an f-score of 95% in classifying ports.
Afilliation | Software Engineering |
Project(s) | TRANSACT |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | 35th Canadian Conference on Artificial Intelligence |
Other Numbers | arXiv:2204.11855 |
Keywords | actual ports, AIS data, gateway ports, maritime situational awareness, maritime traffic, port area, port classification, port congestion, spatio-temporal, temporal graph neural networks, vessel trajectories |
Journal Article
Industry-Academia Research Collaboration and Knowledge Co-creation: Patterns and Anti-patterns
ACM Transactions on Software Engineering and Methodology (2022).Status: Published
Industry-Academia Research Collaboration and Knowledge Co-creation: Patterns and Anti-patterns
Afilliation | Software Engineering |
Project(s) | T3AS |
Publication Type | Journal Article |
Year of Publication | 2022 |
Journal | ACM Transactions on Software Engineering and Methodology |
Publisher | ACM |
ISSN | 1049-331X |
URL | https://dl.acm.org/doi/10.1145/3494519https://dl.acm.org/doi/pdf/10.1145... |
DOI | 10.1145/3494519 |
Proceedings, refereed
DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing
In 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2021.Status: Published
DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing
Continuous integration testing is an important step in the modern software engineering life cycle. Test prioritization is a method that can improve the efficiency of continuous integration testing by selecting test cases that can detect faults in the early stage of each cycle. As continuous integration testing produces voluminous test execution data, test history is a commonly used artifact in test prioritization. However, existing test prioritization techniques for continuous integration either cannot handle large test history or are optimized for using a limited number of historical test cycles. We show that such a limitation can decrease fault detection effectiveness of prioritized test suites.
This work introduces DeepOrder, a deep learning-based model that works on the basis of regression machine learning. DeepOrder ranks test cases based on the historical record of test executions from any number of previous test cycles. DeepOrder learns failed test cases based on multiple factors including the duration and execution status of test cases. We experimentally show that deep neural networks, as a simple regression model, can be efficiently used for test case prioritization in continuous integration testing. DeepOrder is evaluated with respect to time-effectiveness and fault detection effectiveness in comparison with an industry practice and the state of the art approaches. The results show that DeepOrder outperforms the industry practice and state-of-the-art test prioritization approaches in terms of these two metrics.
Afilliation | Software Engineering, Machine Learning |
Project(s) | T3AS |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME) |
Number of Volumes | 10 |
Pagination | 525-534 |
Publisher | IEEE |
DOI | 10.1109/ICSME52107.2021.00053 |
Improving the Reliability of Autonomous Software Systems through Metamorphic Testing
In Proceedings of the 31st European Safety and Reliability Conference (ESREL). ESREL, 2021.Status: Published
Improving the Reliability of Autonomous Software Systems through Metamorphic Testing
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo) |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | Proceedings of the 31st European Safety and Reliability Conference (ESREL) |
Pagination | 1-page abstract |
Publisher | ESREL |
ISBN Number | ISBN: 978-981-18-2016-8 |
Journal Article
Industry-Academia research collaboration in software engineering: The Certus model
Information and Software Technology 132 (2021).Status: Published
Industry-Academia research collaboration in software engineering: The Certus model
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI), T3AS, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Information and Software Technology |
Volume | 132 |
Number | 106473 |
Publisher | Elsevier |
Book Chapter
Testing Industrial Robotic Systems: A New Battlefield!
In Software Engineering for Robotics, 109-137. Cham: Springer Nature, 2021.Status: Published
Testing Industrial Robotic Systems: A New Battlefield!
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-LARGO) , Department of Validation Intelligence for Autonomous Software Systems, AI4EU, Testing of Learning Robots (T-Largo) |
Publication Type | Book Chapter |
Year of Publication | 2021 |
Book Title | Software Engineering for Robotics |
Chapter | 5 |
Pagination | 109-137 |
Date Published | 06/2021 |
Publisher | Springer Nature |
Place Published | Cham |
ISBN Number | ISBN 978-3-030-66493-0 |
URL | https://www.springer.com/gp/book/9783030664930#aboutBook |
Technical reports
Blockchain for Healthcare: Opportunities, Challenges, and Future Directions
Simula Research Laboratory, 2020.Status: Published
Blockchain for Healthcare: Opportunities, Challenges, and Future Directions
The continuous generation of large volume of medical data from different sources, such as patient monitoring, clinical trials management, and processing payments and reimbursement claims, makes healthcare a data-intensive domain. This data needs to be shared among different medical facilities for various purposes, such as in-depth data analysis and collaborative research, to achieve innovative advances in the medical treatment procedures and drug developments, and providing personalised healthcare services to the patients. However, the existing data exchange solutions in healthcare domain exhibits several challenges, like data security, patient privacy, data owner consent management, interoperability, and chain-of-custody. Recently, the industry and research community turned its focus on the possible use of blockchain technology to solve one or more of the above challenges in healthcare domain. The blockchain technology along with the support from smart contracts is considered as an adequate solution for secure and efficient medical data sharing. It is due to its unique features, such as decentralization, trustlessness, immutability, traceability, and security. In this paper, we provide a comprehensive survey on the state-of-the-art efforts that envision the usage of blockchain-based solutions to improve one or more aspects in healthcare domain.
To this end, first we categorize the existing works based on a set of key challenge(s) that they address in healthcare by using blockchain technology. The particular challenges that we consider for the categorization of the surveyed articles are as follows: (i) security and privacy, (ii) interoperability, (iii) compliance management, (iv) medical records management, and (v) chain-of-custody. Second, we discuss the benefits and limitations that the surveyed solutions exhibit when incorporating blockchain in healthcare. We also investigate the causal relationships among these challenges to demonstrate how various factors causing these challenges are connected with each other. Third, we identify the practical issues that blockchain-driven healthcare must overcome to become a success, and we suggest the additional future research required to address the identified issues. Our survey shows that there is an exponential increase in the number of efforts towards the use blockchain and smart contracts to improve various functionalities in several healthcare services such as clinical trials, medical data management, and drug supply chain management. However, most of these efforts are still in their initial phases, and a lot of work still remains before their integration in practical healthcare scenarios. This work is currently in-progress, and it is planned to be performed in an accepted research-based innovation project by Research Council of Norway called “SMARTMED – Secure and accountable sharing of medical records using smart contracts and blockchain”.
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, SmartMed |
Publication Type | Technical reports |
Year of Publication | 2020 |
Publisher | Simula Research Laboratory |
Keywords | blockchain, Cloud computing, Electronic Health Records, Internet of things, Medical data, security and privacy |
Journal Article
ITE: A Lightweight Implementation of Stratified Reasoning for Constructive Logical Operators
International Journal on Artificial Intelligence Tools 29, no. 3-4 (2020): 23.Status: Published
ITE: A Lightweight Implementation of Stratified Reasoning for Constructive Logical Operators
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo), Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | International Journal on Artificial Intelligence Tools |
Volume | 29 |
Issue | 3-4 |
Pagination | 23 |
Date Published | 06/2020 |
Publisher | World Scientific Publishing |
DOI | 10.1142/S0218213020600064. |
Proceedings, refereed
Lessons Learned on Research Co-Creation: Making Industry-Academia Collaboration Work
In The 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). Euromicro, 2020.Status: Published
Lessons Learned on Research Co-Creation: Making Industry-Academia Collaboration Work
Afilliation | Software Engineering |
Project(s) | T3AS, The Certus Centre (SFI), Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | The 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) |
Publisher | Euromicro |
Neural Network Classification for Improving Continuous Regression Testing
In The IEEE Second International Conference On Artificial Intelligence Testing (AITest 2020). IEEE, 2020.Status: Published
Neural Network Classification for Improving Continuous Regression Testing
Afilliation | Software Engineering |
Project(s) | T3AS, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | The IEEE Second International Conference On Artificial Intelligence Testing (AITest 2020) |
Publisher | IEEE |
Opening the Software Engineering Toolbox for the Assessment of Trustworthy AI
In 1st International Workshop on New Foundations for Human-Centered AI @ ECAI 2020. CEUR Workshop Proceedings, 2020.Status: Published
Opening the Software Engineering Toolbox for the Assessment of Trustworthy AI
Trustworthiness is a central requirement for the acceptance and success of human-centered artificial intelligence (AI). To deem an AI system as trustworthy, it is crucial to assess its behaviour and characteristics against a gold standard of Trustworthy AI, consisting of guidelines, requirements, or only expectations. While AI systems are highly complex, their implementations are still based on software. The software engineering community has a long-established toolbox for the assessment of software systems, especially in the context of software testing. In this paper, we argue for the application of software engineering and testing practices for the assessment of trustworthy AI. We make the connection between the seven key requirements as defined by the European Commission's AI high-level expert group and established procedures from software engineering and raise questions for future work.
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, AI4EU |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | 1st International Workshop on New Foundations for Human-Centered AI @ ECAI 2020 |
Pagination | 67-70 |
Date Published | 08/2020 |
Publisher | CEUR Workshop Proceedings |
Other Numbers | arXiv:2007.07768 |
URL | http://ceur-ws.org/Vol-2659/ahuja.pdf |
RobTest: A CP Approach to Generate Maximal Test Trajectories for Industrial Robots
In International Conference on Principles and Practice of Constraint Programming. LNCS ed. Vol. 12333. Springer, 2020.Status: Published
RobTest: A CP Approach to Generate Maximal Test Trajectories for Industrial Robots
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI), Testing of Learning Robots (T-LARGO) , Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-Largo) |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | International Conference on Principles and Practice of Constraint Programming |
Volume | 12333 |
Edition | LNCS |
Pagination | 1-17 |
Date Published | 08/2020 |
Publisher | Springer |
Software Testing for Machine Learning
In The Thirty-Fourth AAAI Conference on Artificial Intelligence. AAAI'20: AAAI, 2020.Status: Published
Software Testing for Machine Learning
Afilliation | Software Engineering |
Project(s) | T3AS, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | The Thirty-Fourth AAAI Conference on Artificial Intelligence |
Date Published | 2020 |
Publisher | AAAI |
Place Published | AAAI'20 |
Talks, invited
Requirements collection for the AI4EU platform development
In Big Data Value Forum, 3-5 November, Berlin, Germany, 2020.Status: Published
Requirements collection for the AI4EU platform development
Afilliation | Software Engineering |
Project(s) | AI4EU, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Talks, invited |
Year of Publication | 2020 |
Location of Talk | Big Data Value Forum, 3-5 November, Berlin, Germany |
Edited books
Seventh International Workshop on Software Engineering Research and Industrial Practice
In International Conference on Software Engineering (ICSE). South Korea: ACM, 2020.Status: Published
Seventh International Workshop on Software Engineering Research and Industrial Practice
Afilliation | Software Engineering |
Project(s) | T3AS, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Edited books |
Year of Publication | 2020 |
Secondary Title | International Conference on Software Engineering (ICSE) |
Publisher | ACM |
Place Published | South Korea |
Journal Article
A learning algorithm for optimizing continuous integration development and testing practice
Software: Practice and Experience 49, no. 2 (2019): 192-213.Status: Published
A learning algorithm for optimizing continuous integration development and testing practice
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Software: Practice and Experience |
Volume | 49 |
Issue | 2 |
Pagination | 192-213 |
Publisher | Wiley Online Library |
Keywords | Continuous Integration, Decision Trees, Machine Leanring, Software Testing |
Good Practices in Aligning Software Engineering Research and Industry Practice
ACM SIGSOFT Software Engineering Notes 44, no. 3 (2019).Status: Published
Good Practices in Aligning Software Engineering Research and Industry Practice
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | ACM SIGSOFT Software Engineering Notes |
Volume | 44 |
Issue | 3 |
Publisher | ACM |
Implications of Resurgence in Artificial Intelligence for Research Collaborations in Software Engineering
ACM SIGSOFT Software Engineering Notes 44, no. 3 (2019).Status: Published
Implications of Resurgence in Artificial Intelligence for Research Collaborations in Software Engineering
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | ACM SIGSOFT Software Engineering Notes |
Volume | 44 |
Issue | 3 |
Publisher | ACM |
Proceedings, refereed
Challenges of Testing Machine Learning Based Systems
In Proceedings of the 1st IEEE Artificial Intelligence Testing Conference (AI Test 2019). San Francisco, CA, USA: IEEE, 2019.Status: Published
Challenges of Testing Machine Learning Based Systems
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI), Testing of Learning Robots (T-Largo), T3AS |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | Proceedings of the 1st IEEE Artificial Intelligence Testing Conference (AI Test 2019) |
Publisher | IEEE |
Place Published | San Francisco, CA, USA |
Edited books
Proceedings of the Joint 7th International Workshop on Conducting Empirical Studies in Industry and 6th International Workshop on Software Engineering Research and Industrial Practice
In International Conference on Software Engineering (ICSE). ACM/IEEE, 2019.Status: Published
Proceedings of the Joint 7th International Workshop on Conducting Empirical Studies in Industry and 6th International Workshop on Software Engineering Research and Industrial Practice
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Edited books |
Year of Publication | 2019 |
Secondary Title | International Conference on Software Engineering (ICSE) |
Publisher | ACM/IEEE |
Technical reports
SARCoS: Machine Learning Models for the Selection of Security Test Scenarios
Simula Research Laboratory, 2019.Status: Accepted
SARCoS: Machine Learning Models for the Selection of Security Test Scenarios
Afilliation | Software Engineering |
Project(s) | T-Sar |
Publication Type | Technical reports |
Year of Publication | 2019 |
Publisher | Simula Research Laboratory |
Survey on Testing of Deep Learning Systems
Simula Research Laboratory, 2019.Status: Published
Survey on Testing of Deep Learning Systems
Recent studies have shown that deep learning algorithms used for image classification or object recognition are not sufficiently reliable. These algorithms can be easily fooled by applying perturbations to images or generating artificial images that result in misclassification. In this paper, we provide an overview of software testing methods present in literature to test deep learning systems. We have explored different methods of testing deep neural networks, namely metamorphic testing, mutation testing, differential testing, and adversarial perturbation testing. We present the main findings available from the literature and compare these methods systematically and comprehensively. The results show that systematic testing of deep learning systems can further help to increase the performance of state-of-the-art systems.
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-Largo) |
Publication Type | Technical reports |
Year of Publication | 2019 |
Publisher | Simula Research Laboratory |
Talks, contributed
Statistics AIS Dataset from Statsat
In Simula Research Laboratory, 2019.Status: Published
Statistics AIS Dataset from Statsat
Afilliation | Software Engineering, Machine Learning |
Project(s) | T-Sar, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Talks, contributed |
Year of Publication | 2019 |
Location of Talk | Simula Research Laboratory |
The TSAR Project
In 14th Certus User Partner Workshop, Sep. 2019, Larvik, Norway, 2019.Status: Published
The TSAR Project
Afilliation | Software Engineering |
Project(s) | T-Sar |
Publication Type | Talks, contributed |
Year of Publication | 2019 |
Location of Talk | 14th Certus User Partner Workshop, Sep. 2019, Larvik, Norway |
Journal Article
Certus: an organizational effort towards research-based innovation in software verification and validation
International Journal of Systems Assurance Engineering and Management 9, no. 2 (2018): 313-322.Status: Published
Certus: an organizational effort towards research-based innovation in software verification and validation
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | International Journal of Systems Assurance Engineering and Management |
Volume | 9 |
Issue | 2 |
Pagination | 313-322 |
Publisher | Springer |
Proceedings, refereed
DevOps Enhancement with Continuous Test Optimization
In The 30th International Conference on Software Engineering and Knowledge Engineering (SEKE). KSI Research Inc. and Knowledge Systems Institute Graduate School, 2018.Status: Published
DevOps Enhancement with Continuous Test Optimization
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | The 30th International Conference on Software Engineering and Knowledge Engineering (SEKE) |
Pagination | 535-536 |
Publisher | KSI Research Inc. and Knowledge Systems Institute Graduate School |
DOI | 10.18293/SEKE2018-168 |
DevOps Improvements for Reduced Cycle Times with Integrated Test Optimizations for Continuous Integration
In 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC). IEEE, 2018.Status: Published
DevOps Improvements for Reduced Cycle Times with Integrated Test Optimizations for Continuous Integration
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC) |
Pagination | 22-27 |
Publisher | IEEE |
DOI | 10.1109/COMPSAC.2018.00012 |
Practical Selective Regression Testing with Effective Redundancy in Interleaved Tests
In Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP'18). ACM, 2018.Status: Published
Practical Selective Regression Testing with Effective Redundancy in Interleaved Tests
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP'18) |
Pagination | 153-162 |
Publisher | ACM |
ISBN Number | 978-1-4503-5659-6 |
DOI | 10.1145/3183519.3183532 |
Stratified Constructive Disjunction and Negation in Constraint Programming
In Proc. of IEEE Int. Conf. on Tools with Artificial Intelligence (ICTAI-18). Volos, Greece. Nov. 2018. IEEE, 2018.Status: Published
Stratified Constructive Disjunction and Negation in Constraint Programming
Constraint Programming (CP) is a powerful declarative programming paradigm combining inference and search in order to find solutions to various type of constraint systems. Dealing with highly disjunctive constraint systems is notoriously difficult in CP. Apart from trying to solve each disjunct independently from each other, there is little hope and effort to succeed in constructing intermediate results combining the knowledge originating from several disjuncts. In this paper, we propose If Then Else (ITE), a lightweight approach for implementing stratified constructive disjunction and negation on top of an existing CP solver, namely SICStus Prolog clp(FD). Although constructive disjunction is known for more than three decades, it does not have straightforward implementations in most CP solvers. ITE is a freely available library proposing stratified and constructive reasoning for various operators, including disjunction and negation, implication and conditional. Our preliminary experimental results show that ITE is competitive with existing approaches that handle disjunctive constraint systems.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proc. of IEEE Int. Conf. on Tools with Artificial Intelligence (ICTAI-18). Volos, Greece. Nov. 2018 |
Pagination | 106-113 |
Publisher | IEEE |
Other Numbers | arXiv:1811.03906 |
DOI | 10.1109/ICTAI.2018.00026 |
Edited books
Fifth International Workshop on Software Engineering Research and Industrial Practice
In International Conference on Software Engineering (ICSE). Gothenburg, Sweden: ACM, 2018.Status: Published
Fifth International Workshop on Software Engineering Research and Industrial Practice
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Edited books |
Year of Publication | 2018 |
Secondary Title | International Conference on Software Engineering (ICSE) |
Date Published | May |
Publisher | ACM |
Place Published | Gothenburg, Sweden |
Talks, contributed
Practical selective regression testing with effective redundancy in interleaved tests
In International Conference on Software Engineering (ICSE), Gothenburg, Sweden. Software Engineering in Practice, 2018.Status: Published
Practical selective regression testing with effective redundancy in interleaved tests
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, contributed |
Year of Publication | 2018 |
Location of Talk | International Conference on Software Engineering (ICSE), Gothenburg, Sweden |
Place Published | Software Engineering in Practice |
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
In Gesellschaft für Informatik Software Engineering Conference 2018 (SE18), Ulm, Germany, 2018.Status: Published
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, contributed |
Year of Publication | 2018 |
Location of Talk | Gesellschaft für Informatik Software Engineering Conference 2018 (SE18), Ulm, Germany |
Proceedings, refereed
Detecting and Reducing Redundancy in Software Testing for Highly Configurable Systems
In IEEE International Symposium on High Assurance Systems Engineering. ACM/IEEE, 2017.Status: Published
Detecting and Reducing Redundancy in Software Testing for Highly Configurable Systems
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | IEEE International Symposium on High Assurance Systems Engineering |
Publisher | ACM/IEEE |
ISBN Number | 978-1-5090-4636-2 |
DOI | 10.1109/HASE.2017.31 |
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. New York, NY, USA: ACM, 2017.Status: Published
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis |
Pagination | 12-22 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 978-1-4503-5076-1 |
Other Numbers | arXiv:1811.04122 |
DOI | 10.1145/3092703.3092709 |
Test Prioritization with Optimally Balanced Configuration Coverage
In IEEE International Symposium on High Assurance Systems Engineering. ACM/IEEE, 2017.Status: Published
Test Prioritization with Optimally Balanced Configuration Coverage
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | IEEE International Symposium on High Assurance Systems Engineering |
Publisher | ACM/IEEE |
ISBN Number | 978-1-5090-4636-2 |
DOI | 10.1109/HASE.2017.26 |
TITAN: Test Suite Optimization for Highly Configurable Software
In International Conference on Software Testing, Verification and Validation (ICST 2017) . IEEE, 2017.Status: Published
TITAN: Test Suite Optimization for Highly Configurable Software
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | International Conference on Software Testing, Verification and Validation (ICST 2017) |
Publisher | IEEE |
ISBN Number | 978-1-5090-6031-3 |
DOI | 10.1109/ICST.2017.60 |
Journal Article
Modeling and Verifying Combinatorial Interactions to Test Data Intensive Systems: Experience at the Norwegian Customs Directorate
IEEE Transactions on Reliability 66 (2017): 3-16.Status: Published
Modeling and Verifying Combinatorial Interactions to Test Data Intensive Systems: Experience at the Norwegian Customs Directorate
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | IEEE Transactions on Reliability |
Volume | 66 |
Number | 1 |
Pagination | 3–16 |
Publisher | IEEE Transactions on Reliability |
URL | https://doi.org/10.1109/TR.2016.2618121 |
DOI | 10.1109/TR.2016.2618121 |
Using Global Constraints to Automate Regression Testing
AI Magazine 38, no. Spring (2017).Status: Published
Using Global Constraints to Automate Regression Testing
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | AI Magazine |
Volume | 38 |
Issue | Spring |
Number | 1 |
Publisher | AAAI |
Book Chapter
Software Product Line Test Suite Reduction with Constraint Optimization
In Software Technologies, 68-87. Vol. 743. Springer International Publishing, 2017.Status: Published
Software Product Line Test Suite Reduction with Constraint Optimization
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Book Chapter |
Year of Publication | 2017 |
Book Title | Software Technologies |
Volume | 743 |
Pagination | 68-87 |
Publisher | Springer International Publishing |
Proceedings, refereed
A New Approach to Feature-based Test Suite Reduction in Software Product Line Testing
In ICSOFT-EA 2016, 11th Int. Conf. on Software Engineering and Applications, Lisbon, July 2016, Awarded Best Paper. INSTICC Press, 2016.Status: Published
A New Approach to Feature-based Test Suite Reduction in Software Product Line Testing
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | ICSOFT-EA 2016, 11th Int. Conf. on Software Engineering and Applications, Lisbon, July 2016, Awarded Best Paper |
Date Published | 07/2016 |
Publisher | INSTICC Press |
Automated Regression Testing Using Constraint Programming
In Twenty-Eighth Conference on Innovative Applications of Artificial Intelligence (IAAI-16), Phoenix, AZ, USA, Feb. 2016, 2016.Status: Published
Automated Regression Testing Using Constraint Programming
In software validation, regression testing aims to check the absence of regression faults in new releases of a software system. Typically, test cases used in regression testing are executed during a limited amount of time and are selected to check a given set of user requirements. When testing large systems the number of regression tests grows quickly over the years, and yet the available time slot stays limited. In order to overcome this problem, an approach known as test suite reduction (TSR), has been developed in software engineering to select a smallest subset of test cases, so that each requirement remains covered at least once. However solving the TSR problem is difficult as the underlying optimization problem is NP-hard, but it is also crucial for vendors interested in reducing the time to market of new software releases.
In this paper, we address regression testing and TSR with Constraint Programming (CP). More specifically, we propose new CP models to solve TSR that exploit global constraints, namely NValue and GCC. We reuse a set of preprocessing rules to reduce a priori each instance, and we introduce a structure-aware search heuristic. We evaluated our CP models and proposed improvements against existing approaches, including a simple greedy approach and MINTS, the state-of-the-art tool of the software engineering community. Our experiments show that CP outperforms both the greedy approach and MINTS when it is interfaced with MiniSAT, in terms of percentage of reduction and execution time. When MINTS is interfaced with CPLEX, we show that our CP model performs better only on percentage of reduction. Finally, by working closely with validation engineers from Cisco Systems, Norway, we integrated our CP model into an industrial regression testing process.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Twenty-Eighth Conference on Innovative Applications of Artificial Intelligence (IAAI-16), Phoenix, AZ, USA, Feb. 2016 |
Date Published | 02/2016 |
Coverage-based Test Prioritization for Regression Testing of Configurable Software
In IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), 2016.Status: Published
Coverage-based Test Prioritization for Regression Testing of Configurable Software
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | IEEE 27th International Symposium on Software Reliability Engineering (ISSRE) |
Effect of Time Window on the Performance of Continuous Regression Testing
In 32nd IEEE International Conference on Software Maintenance and Evolution (ICSME), 2016.Status: Published
Effect of Time Window on the Performance of Continuous Regression Testing
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | 32nd IEEE International Conference on Software Maintenance and Evolution (ICSME) |
Improving Configurable Software Testing with Statistical Test Selection
In 31st IEEE/ACM International Conference on Automated Software Engineering (ASE) Workshops, 2016.Status: Published
Improving Configurable Software Testing with Statistical Test Selection
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | 31st IEEE/ACM International Conference on Automated Software Engineering (ASE) Workshops |
Journal Article
Modelling and Verifying Combinatorial Interactions to Test Data Intensive Systems: Experience with Optimal Archiving at the Norwegian Customs and Excise Directorate
IEEE Transaction on Reliability, no. 99 (2016): 1-14.Status: Published
Modelling and Verifying Combinatorial Interactions to Test Data Intensive Systems: Experience with Optimal Archiving at the Norwegian Customs and Excise Directorate
Testing data-intensive systems is paramount to increase
our reliance on information processed in e-governance,
scientific/medical research, and social networks. Data accrued in
these systems often go through several manual and computational
steps involving human inputs in interactive media and complex
batch appications that aim to ensure high quality of data in
terms of validity, correctness, and adherence to business rules. A
common industrial practice in testing data-intensive systems is
to extract test databases from live production streams and verify
the data in them through a checklist of requirements either
by tedious manual observation or by executing complex SQL
queries composed and understood by very few domain experts.
We elevate the specification of such requirements on data by
modelling data interactions between fields cross-cutting the test
database’s schema. These interactions are modelled as test cases
in a classification tree model. The model documents intuitive
expert knowledge about what to expect in the test database
and is given executable semantics using our human-in-the-loop
tool DEPICT. DEPICT verifies if interactions occurred or not
in systematically extracted test databases. Non-occurrence of
expected interactions or occurrence of unexpected interactions
indicate faults in the data. We present experiences on how our
model-driven approach has been successfully applied to verify
test databases in the Norwegian Public Sector. In particular, we
present case studies at (1) the Norwegian Customs and Excise
Directorate for verifying the adherence to customs regulations
and (2) the Cancer Registry of Norway to verify its data quality
management process involving both human coders and complex
legacy batches.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | IEEE Transaction on Reliability |
Issue | 99 |
Pagination | 1-14 |
Publisher | IEEE |
Practical Minimization of Pairwise-Covering Test Configurations Using Constraint Programming
Information and Software Technology 71 (2016): 129-146.Status: Published
Practical Minimization of Pairwise-Covering Test Configurations Using Constraint Programming
Context: Testing highly-configurable software systems is challenging due to a large number of test configurations that have to be carefully selected in order to reduce the testing effort as much as possible, while maintaining high software quality. Finding the smallest set of valid test configurations that ensure sufficient coverage of the system's feature interactions is thus the objective of validation engineers, especially when the execution of test configurations is costly or time-consuming. However, this problem is NP-hard in general and approximation algorithms have often been used to address it in practice.
Objective: In this paper, we explore an alternative exact approach based on constraint programming that will allow engineers to increase the effectiveness of configuration testing while keeping the number of configurations as low as possible.
Method: Our approach consists in using a (time-aware) minimization algorithm based on constraint programming. Given the amount of time, our solution generates a minimized set of valid test configurations that ensure coverage of all pairs of feature values (a.k.a. pairwise coverage). The approach has been implemented in a tool called PACOGEN.
Results: PACOGEN was evaluated on 224 feature models from the standard benchmark repository SPLOT, and compared
in comparison with the two existing tools that are based on a greedy algorithm. For 79% of 224 feature models, PACOGEN generated up to 60% fewer test configurations than the competitor tools. We further evaluated PACOGEN in the case study of an industrial video conferencing product line with a feature model of 169 features, and found 60% fewer configurations compared with the manual approach followed by test engineers. The set of test configurations generated by PACOGEN decreased the time required by test engineers in manual test configuration by 85%, increasing the feature-pairs coverage at the same time
Conclusion: Our experimental evaluation concluded that optimal time-aware minimization of pairwise-covering test configurations is efficiently addressed using constraint programming techniques.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | Information and Software Technology |
Volume | 71 |
Pagination | 129-146 |
Date Published | 03/2016 |
Publisher | Elsevier |
Journal Article
Certus: Glimpses of a Centre for Research-Based Innovation in Software Verification and Validation
International Journal of System Assurance Engineering and Management (2015): 1-25.Status: Published
Certus: Glimpses of a Centre for Research-Based Innovation in Software Verification and Validation
What is gratifying to a software engineering researcher? Three of many possible answers to this question are (a) the intellectual exercise in developing/disseminating approaches that address emerging and existing challenges, (b) recognition from impact in a community of researchers and (c) widespread use of novel ideas, including software, in the society at large leading to enhancement of human ability and job creation. A culmination of these sources requires an organizational effort. This article presents a detailed account of a research-based innovation centre, Certus, to facilitate such a culmination for software engineering researchers. Certus has established a body of knowledge, methods and tools for the validation and verification of software systems in the Norwegian private and public sector. Certus works in close cooperation with five founding user partners and is hosted by the Simula Research Laboratory. We present the organizational structure of Certus and describe how Certus’s life and health is planned and evaluated on a regular basis as a research-based innovation centre. We expound two successful collaborations, with (a) the private sector entity Cisco systems, Norway and (b) the public sector entity the Norwegian Customs and Excise. We hope that this document will serve as a basis to encourage national/international funding schemes to create call for proposals for long-term research-based innovation centres. This, we believe, is one way to justify use of tax payers resources in creating a win–win situation for the triple helix: government, researchers and industry.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | International Journal of System Assurance Engineering and Management |
Pagination | 1-25 |
Date Published | 03/2015 |
Publisher | Springer |
Proceedings, refereed
Evaluating Reconfiguration Impact in Self-Adaptive Systems – An Approach Based on Combinatorial Interaction Testing
In The 41st Euromicro Conference on Software Engineering and Advanced Applications (SEAA). Madeira, Portugal, August 26-28. Funchal: IEEE, 2015.Status: Published
Evaluating Reconfiguration Impact in Self-Adaptive Systems – An Approach Based on Combinatorial Interaction Testing
Self-adaptive software adapts its behavior to the operational context via automatic run-time reconfiguration of software components. Particular reconfigurations may negatively affect the system Quality of Service (QoS), and therefore their impact over the system performance needs to be thoroughly evaluated. In this paper, we present an approach, based on Combinatorial Interaction Testing (CIT), that generates a sequence of configurations aimed at evaluating the extent to which reconfigurations affect the system QoS. Specifically, we transform a Classification Tree Models (CTM) of the configurations domain to a Constraint Satisfaction Problem (CSP) in Alloy, whose solution is a sequence of reconfigurations achieving T-wise coverage between system features, and R-wise coverage between configurations in the sequence. The resolution of the CSP is performed by an incremental growth algorithm that divides the generation of the sequence into sub-problems, and merges the results into a final set of test configurations. Preliminary validation in a self-adaptive vision system shows that our methodology effectively identifies critical reconfigurations exhibiting a high variation in QoS. This result encourages the use of CIT as a strategy to evaluate the performance of self-adaptive systems.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | The 41st Euromicro Conference on Software Engineering and Advanced Applications (SEAA). Madeira, Portugal, August 26-28 |
Pagination | 250 - 254 |
Date Published | 08/2015 |
Publisher | IEEE |
Place Published | Funchal |
DOI | 10.1109/SEAA.2015.72 |
Multi-perspective Regression Test Prioritization for Time-Constrained Environments
In IEEE International Conference on Software Quality, Reliability and Security. Vancouver, Canada, August 3-5, 2015.Status: Published
Multi-perspective Regression Test Prioritization for Time-Constrained Environments
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | IEEE International Conference on Software Quality, Reliability and Security |
Place Published | Vancouver, Canada, August 3-5 |
Towards More Relational Feature Models
In ICSOFT-EA 2015 - Proceedings of the 10th International Conference on Software Engineering and Applications, Colmar, Alsace, France, 20-22 July. SciTePress, 2015.Status: Published
Towards More Relational Feature Models
Feature modeling is of paramount importance to capture variabilities and commonalities within a software product line. Nevertheless, current feature modeling notations are limited, representing only propositional formulae over attributed variables. This position paper advocates the extension of feature modeling formalisms with richer computational domains and relational operations. In particular, it proposes to extend feature modeling with finite and continuous domain variables, with first-order logic quantifiers, and with N-ary relations between features attributes, and with so-called global constraints. In order to extend the expressiveness while preserving automated analysis facilities, feature models could be semantically interpreted as first-order logic formulae (instead of propositional logic formulae), including global and continuous dependency between features. In simpler words, this paper emphasizes the importance of having more relational feature models and presents next generation applications.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | ICSOFT-EA 2015 - Proceedings of the 10th International Conference on Software Engineering and Applications, Colmar, Alsace, France, 20-22 July. |
Pagination | 381-386 |
Date Published | 07/2015 |
Publisher | SciTePress |
ISBN Number | 978-989-758-114-4 |
Proceedings, refereed
FLOWER: Optimal Test Suite Reduction As a Network Maximum Flow
In Proceedings of Int. Symp. on Soft. Testing and Analysis (ISSTA'14), San José, CA, USA, Jul. 2014. New York, USA: ACM, 2014.Status: Published
FLOWER: Optimal Test Suite Reduction As a Network Maximum Flow
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of Int. Symp. on Soft. Testing and Analysis (ISSTA'14), San José, CA, USA, Jul. 2014 |
Pagination | 171-180 |
Date Published | 07/2014 |
Publisher | ACM |
Place Published | New York, USA |
ISBN Number | 978-1-4503-2645-2 |
Keywords | Conference |
DOI | 10.1145/2610384.2610416 |
Talks, invited
Variability Testing of Highly-Configurable Software
In The Norwegian Computer Society. .: , 2014.Status: Published
Variability Testing of Highly-Configurable Software
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, invited |
Year of Publication | 2014 |
Location of Talk | The Norwegian Computer Society |
Publisher | . |
Place Published | . |
Talks, invited
Managing Test Configurations in High-Variability Testing Environments With TITAN and Pure::variants
In pure::variants Solutions Forum, SPLC. .: , 2013.Status: Published
Managing Test Configurations in High-Variability Testing Environments With TITAN and Pure::variants
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, invited |
Year of Publication | 2013 |
Location of Talk | pure::variants Solutions Forum, SPLC |
Publisher | . |
Place Published | . |
Proceedings, refereed
Practical Pairwise Testing for Software Product Lines
In Proceedings of the International Software Product Line Conference (SPLC). New York, NY, USA: ACM, 2013.Status: Published
Practical Pairwise Testing for Software Product Lines
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | Proceedings of the International Software Product Line Conference (SPLC) |
Pagination | 227-235 |
Publisher | ACM |
Place Published | New York, NY, USA |
Test Case Prioritization for Continuous Regression Testing: an Industrial Case Study
In Proceedings of the International Conference on Software Maintenance (ICSM). Eindhoven: IEEE, 2013.Status: Published
Test Case Prioritization for Continuous Regression Testing: an Industrial Case Study
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | Proceedings of the International Conference on Software Maintenance (ICSM) |
Publisher | IEEE |
Place Published | Eindhoven |
Technical reports
Test Selection Based on Data Interactions in Data-Intensive Systems
Simula Research Laboratory, 2013.Status: Published
Test Selection Based on Data Interactions in Data-Intensive Systems
Testing data-intensive systems is paramount to increase our reliance on information in e-governance, scientific/ medical research, and social networks. Common practice to test these systems is by using a live production database. This testing approach is space and time inefficient and lacks clarity about what test cases or scenarios are covered. In this paper, we leverage classification tree modelling to specify desired test cases as data interactions between a set of fields across multiple tables of an existing database. Our methodology and tool, DEPICT, uses test case specifications in classification tree models to (a) automatically derive a spanning tree representing a relationship between any set of fields for any given database schema (b) generates queries to create an efficient inner join between related tables in the spanning tree (c) extract records from various tables that satisfy data interactions in the classification tree model (d) discovers holes or unsatisfied test cases in the test databases. We perform experiments to show that our approach is fast and scalable to extract test databases. Our experiments are based on selecting test databases from 8000 declarations for 60,000 items from the Norwegian Customs and Excise information system TVINN.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Technical reports |
Year of Publication | 2013 |
Number | 2013-03 |
Date Published | 10/2013 |
Publisher | Simula Research Laboratory |
Keywords | Conference |
Proceedings, refereed
A Review of Two Experiences From Applying Model Based Testing in Practice
In International Symposium on Software Reliability Engineering Workshops (ISSREW). Dallas, TX: IEEE, 2012.Status: Published
A Review of Two Experiences From Applying Model Based Testing in Practice
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | International Symposium on Software Reliability Engineering Workshops (ISSREW) |
Publisher | IEEE |
Place Published | Dallas, TX |
Technical reports
Research report on test configuration generation
Simula Research Laboratory, 2012.Status: Published
Research report on test configuration generation
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Technical reports |
Year of Publication | 2012 |
Publisher | Simula Research Laboratory |