Publications
Proceedings, refereed
Approche générique pour l’acquisition de contraintes qualitatives
In JFPC, 2023.Status: Accepted
Approche générique pour l’acquisition de contraintes qualitatives
De nombreux problèmes de planification et d’ordonnance-
ment impliquent la conception de subtiles combinaisons lo-
giques de contraintes temporelles ou spatiales. D’une part,
la modélisation précise de ces contraintes, qui sont formu-
lées dans diverses algèbres de relations, implique un cer-
tain nombre de combinaisons logiques possibles et néces-
site une expertise en modélisation basée sur les contraintes.
D’autre part, l’acquisition active de contraintes (AC) a été
utilisée avec succès pour aider les utilisateurs non expé-
rimentés à apprendre les réseaux de contraintes conjonc-
tives par la génération d’une séquence de requêtes. Dans
cet article, nous proposons GEQCA, pour Generic Quali-
tative Constraint Acquisition, une méthode d’AC active qui
apprend les contraintes qualitatives via le concept de re-
quêtes qualitatives. GEQCA combine les requêtes quali-
tatives avec la cohérence de chemin limitée dans le temps
(PC pour Path Consistency) et la propagation des connais-
sances de base pour acquérir les contraintes qualitatives.
Nous prouvons la correction, la complétude et la terminai-
son de GEQCA et nous donnons une évaluation expéri-
mentale qui montre (i) l’efficacité de notre approche dans
l’apprentissage des contraintes temporelles et, (ii) l’utilisa-
tion de GEQCA sur des instances réelles d’ordonnance-
ment.
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2023 |
Conference Name | JFPC |
Automated Program Analysis: Revisiting Precondition Inference through Constraint Acquisition
In Knowledge Representation (KR), Research Track, Rhodes, Greece, September 2-8, 2023.Status: Accepted
Automated Program Analysis: Revisiting Precondition Inference through Constraint Acquisition
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2023 |
Conference Name | Knowledge Representation (KR), Research Track, Rhodes, Greece, September 2-8 |
Constraint-guided Test Execution Scheduling: An Experience Report at ABB Robotics
In SAFECOMP2023 42nd International Conference on Computer Safety, Reliability and Security 19-22 September 2023, Toulouse, France, 2023.Status: Accepted
Constraint-guided Test Execution Scheduling: An Experience Report at ABB Robotics
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) |
Publication Type | Proceedings, refereed |
Year of Publication | 2023 |
Conference Name | SAFECOMP2023 42nd International Conference on Computer Safety, Reliability and Security 19-22 September 2023, Toulouse, France |
Other Numbers | arXiv:2306.01529 |
Towards Trustworthy-AI-by-Design Methodology for Intelligent Radiology Systems
In 'Santé et IA séminaire' at PFIA 23, Strasbrourg, France. July 6, 2023.Status: Accepted
Towards Trustworthy-AI-by-Design Methodology for Intelligent Radiology Systems
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2023 |
Conference Name | 'Santé et IA séminaire' at PFIA 23, Strasbrourg, France. July 6 |
Proceedings, refereed
Automated Program Analysis: Revisiting Precondition Inference through Constraint Acquisition
In 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence (IJCAI-ECAI 22). IJCAI, 2022.Status: Published
Automated Program Analysis: Revisiting Precondition Inference through Constraint Acquisition
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo) |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence (IJCAI-ECAI 22) |
Publisher | IJCAI |
GEQCA: Generic Qualitative Constraint Acquisition
In Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. AAAI, 2022.Status: Published
GEQCA: Generic Qualitative Constraint Acquisition
Many planning, scheduling or multi-dimensional packing problems involve the design of subtle logical combinations of temporal or spatial constraints. On the one hand, the pre- cise modelling of these constraints, which are formulated in various relation algebras, entails a number of possible logical combinations and requires expertise in constraint-based mod- elling. On the other hand, active constraint acquisition (CA) has been used successfully to support non-experienced users in learning conjunctive constraint networks through the gen- eration of a sequence of queries. In this paper, we propose GEQCA, which stands for Generic Qualitative Constraint Acquisition, an active CA method that learns qualitative con- straints via the concept of qualitative queries. GEQCA com- bines qualitative queries with time-bounded path consistency (PC) and background knowledge propagation to acquire the qualitative constraints of any scheduling or packing prob- lem. We prove soundness, completeness and termination of GEQCA by exploiting the jointly exhaustive and pairwise disjoint property of qualitative calculus and we give an ex- perimental evaluation that shows (i) the efficiency of our ap- proach in learning temporal constraints and, (ii) the use of GEQCA on real scheduling instances.
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo), Department of Validation Intelligence for Autonomous Software Systems, AutoCSP: Self-Supervised Neuro-Symbolic Solvers for Constraint Satisfaction |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | Proceedings of the AAAI Conference on Artificial Intelligence |
Volume | 36 |
Number of Volumes | 4 |
Pagination | 3690-3697 |
Date Published | 06/2022 |
Publisher | AAAI |
URL | https://ojs.aaai.org/index.php/AAAI/article/view/20282 |
DOI | 10.1609/aaai.v36i4.20282 |
Méthodologie d’anonymisation dès la conception d’un jeu de données en imagerie médicale
In Applications Pratiques de l'Intelligence Artificielle (APIA). AFIA, 2022.Status: Published
Méthodologie d’anonymisation dès la conception d’un jeu de données en imagerie médicale
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | Applications Pratiques de l'Intelligence Artificielle (APIA) |
Publisher | AFIA |
Testing Deep Learning Models: A First Comparative Study of Multiple Testing Techniques
In Artificial Intelligence in Software Testing @ 2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 2022.Status: Published
Testing Deep Learning Models: A First Comparative Study of Multiple Testing Techniques
Deep Learning (DL) has revolutionized the capabilities of vision-based systems (VBS) in critical applications such as autonomous driving, robotic surgery, critical infrastructure surveillance, air and maritime traffic control, etc. By analyzing images, voice, videos, or any type of complex signals, DL has considerably increased the situation awareness of these systems. At the same time, while relying more and more on trained DL models, the reliability and robustness of VBS have been challenged and it has become crucial to test thoroughly these models to assess their capabilities and potential errors. To discover faults in DL models, existing software testing methods have been adapted and refined accordingly. In this article, we provide an overview of these software testing methods, namely differential, metamorphic, mutation, and combinatorial testing, as well as adversarial perturbation testing and review some challenges in their deployment for boosting perception systems used in VBS. We also provide a first experimental comparative study on a classical benchmark used in VBS and discuss its results.
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo) |
Publication Type | Proceedings, refereed |
Year of Publication | 2022 |
Conference Name | Artificial Intelligence in Software Testing @ 2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) |
Publisher | IEEE |
ISBN Number | 978-1-6654-9628-5 |
Other Numbers | arXiv:2202.12139 |
URL | https://ieeexplore.ieee.org/abstract/document/9787976 |
DOI | 10.1109/ICSTW55395.2022.00035 |
Talks, contributed
GEQCA: Generic Qualitative Constraint Acquisition
In Lernen. Wissen. Daten. Analysen. (LWDA) - KDML Track, 2022.Status: Published
GEQCA: Generic Qualitative Constraint Acquisition
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Talks, contributed |
Year of Publication | 2022 |
Location of Talk | Lernen. Wissen. Daten. Analysen. (LWDA) - KDML Track |
URL | https://lwda2022.de |
Poster
T-Largo: Testing of Learning Robots
Simula Research Laboratory - KA23, Oslo, Norway, 2022.Status: Published
T-Largo: Testing of Learning Robots
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo) |
Publication Type | Poster |
Year of Publication | 2022 |
Place Published | Simula Research Laboratory - KA23, Oslo, Norway |
Type of Work | Poster for celebrating the 20th anniversary of Simula |
Talks, invited
Advanced Testing Methods for Robotic Software-Systems
In Bristol Robotics Lab., Bristol, UK. Bristol Robotics Lab., Bristol, UK. 12th May, 2021.Status: Published
Advanced Testing Methods for Robotic Software-Systems
Industrial collaborative robots are ever-growing complex systems which embed more and more self-decision and autonomous planning capabilities. For that reason, it is crucial to thoroughly validate their software systems with appropriate software testing techniques. As it has become very difficult to predict exactly and accurately their expected behaviors in all situations, several Artificial Intelligence methods are explored to facilitate the selection and scheduling of test cases and to predict expected test results. My talk will review some of these methods and how they are deployed to testing robotic software systems.
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-Largo) |
Publication Type | Talks, invited |
Year of Publication | 2021 |
Location of Talk | Bristol Robotics Lab., Bristol, UK |
Place Published | Bristol Robotics Lab., Bristol, UK. 12th May |
Type of Talk | Seminar |
AI-based Testing of Autonomous Software Systems
In IRT SystemX, France. IRT SystemX, France - 27 October, 2021, 2021.Status: Published
AI-based Testing of Autonomous Software Systems
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo) |
Publication Type | Talks, invited |
Year of Publication | 2021 |
Location of Talk | IRT SystemX, France |
Place Published | IRT SystemX, France - 27 October, 2021 |
Breaking silos in data innovation in Europe: Experiences of AI4EU, EUH4D, and DIH4AI
In Data Week 2021, 2021.Status: Published
Breaking silos in data innovation in Europe: Experiences of AI4EU, EUH4D, and DIH4AI
Afilliation | Software Engineering |
Project(s) | AI4EU |
Publication Type | Talks, invited |
Year of Publication | 2021 |
Location of Talk | Data Week 2021 |
Type of Talk | Contributions to panel discussion - https://www.big-data-value.eu/dw21-agenda/ |
Code-Based Testing with Constraints
In HUAWEI Paris, France. HUAWEI Paris - 31 March 2021, 2021.Status: Published
Code-Based Testing with Constraints
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo) |
Publication Type | Talks, invited |
Year of Publication | 2021 |
Location of Talk | HUAWEI Paris, France |
Place Published | HUAWEI Paris - 31 March 2021 |
Type of Talk | Seminar invitation |
Proceedings, refereed
Apprentissage auto-supervisé pour la détection d’actions illégales lors de la surveillance du trafic maritime
In Applications Pratiques de l’Intelligence Artificielle. AFIA, 2021.Status: Published
Apprentissage auto-supervisé pour la détection d’actions illégales lors de la surveillance du trafic maritime
Afilliation | Software Engineering, Machine Learning |
Project(s) | T-Sar, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | Applications Pratiques de l’Intelligence Artificielle |
Publisher | AFIA |
Encoding Temporal and Spatial Vessel Context using Self-Supervised Learning Model (Student Abstract)
In Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. AAAI, 2021.Status: Published
Encoding Temporal and Spatial Vessel Context using Self-Supervised Learning Model (Student Abstract)
Afilliation | Software Engineering |
Project(s) | T-Sar, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | Proceedings of the AAAI Conference on Artificial Intelligence |
Volume | 35 |
Pagination | 15757-15758 |
Publisher | AAAI |
Improving the Reliability of Autonomous Software Systems through Metamorphic Testing
In Proceedings of the 31st European Safety and Reliability Conference (ESREL). ESREL, 2021.Status: Published
Improving the Reliability of Autonomous Software Systems through Metamorphic Testing
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo) |
Publication Type | Proceedings, refereed |
Year of Publication | 2021 |
Conference Name | Proceedings of the 31st European Safety and Reliability Conference (ESREL) |
Pagination | 1-page abstract |
Publisher | ESREL |
ISBN Number | ISBN: 978-981-18-2016-8 |
Poster
Encoding Temporal and Spatial Vessel Context using Self-Supervised Learning Model (Student Abstract)
AAAI Conference on Artificial Intelligence (AAAI-21), Student Abstract and Poster Program, 2021.Status: Published
Encoding Temporal and Spatial Vessel Context using Self-Supervised Learning Model (Student Abstract)
Afilliation | Software Engineering, Machine Learning |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, T-Sar |
Publication Type | Poster |
Year of Publication | 2021 |
Place Published | AAAI Conference on Artificial Intelligence (AAAI-21), Student Abstract and Poster Program |
Journal Article
Industry-Academia research collaboration in software engineering: The Certus model
Information and Software Technology 132 (2021).Status: Published
Industry-Academia research collaboration in software engineering: The Certus model
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI), T3AS, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | Information and Software Technology |
Volume | 132 |
Number | 106473 |
Publisher | Elsevier |
Predictive Machine Learning of Objective Boundaries for Solving COPs
AI 2, no. 4 (2021): 527-551.Status: Published
Predictive Machine Learning of Objective Boundaries for Solving COPs
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, AI4EU, Testing of Learning Robots (T-Largo) |
Publication Type | Journal Article |
Year of Publication | 2021 |
Journal | AI |
Volume | 2 |
Issue | 4 |
Pagination | 527 - 551 |
Date Published | 12/2021 |
Publisher | MDPI |
Place Published | Basel/Switzerland |
Other Numbers | arXiv:2111.03160 |
URL | https://www.mdpi.com/2673-2688/2/4/33 |
DOI | 10.3390/ai2040033 |
Talks, contributed
Learning to Generate Fault-revealing Test Cases in Metamorphic Testing
In Software Engineering 2021. Bonn: Gesellschaft für Informatik e.V, 2021.Status: Published
Learning to Generate Fault-revealing Test Cases in Metamorphic Testing
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo) |
Publication Type | Talks, contributed |
Year of Publication | 2021 |
Location of Talk | Software Engineering 2021 |
Publisher | Gesellschaft für Informatik e.V. |
Place Published | Bonn |
URL | https://dl.gi.de/handle/20.500.12116/34533 |
DOI | 10.18420/SE2021_37 |
Solve Optimization Problems with Unknown Constraint Networks
In PTHG workshop in CP (online), 2021.Status: Published
Solve Optimization Problems with Unknown Constraint Networks
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo), Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Talks, contributed |
Year of Publication | 2021 |
Location of Talk | PTHG workshop in CP (online) |
Type of Talk | Workshop presentation |
URL | https://www.youtube.com/watch?v=JvX7yKm_WS4&t=8s |
Summary of: Adaptive Metamorphic Testing with Contextual Bandits
In IEEE International Conference on Software Testing (ICST), 2021.Status: Published
Summary of: Adaptive Metamorphic Testing with Contextual Bandits
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Talks, contributed |
Year of Publication | 2021 |
Location of Talk | IEEE International Conference on Software Testing (ICST) |
Type of Talk | Journal-First Track |
Other Numbers | arXiv:1910.00262 |
Talk, keynote
Leveraging AI Methods for Testing Non-testable Autonomous Systems
In 17th European Dependable Computing Conference 13-16 September 2021 Munich, Germany, 2021.Status: Published
Leveraging AI Methods for Testing Non-testable Autonomous Systems
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo) |
Publication Type | Talk, keynote |
Year of Publication | 2021 |
Location of Talk | 17th European Dependable Computing Conference 13-16 September 2021 Munich, Germany |
Proceedings, non-refereed
Solve Optimization Problems with Unknown Constraint Networks
In PTHG workshop in CP, 2021.Status: Accepted
Solve Optimization Problems with Unknown Constraint Networks
In most optimization problems, users have a clear understanding of the function to optimize (e.g., minimize the makespan for schedul- ing problems). However, the constraints may be difficult to state and their modelling often requires expertise in Constraint Program- ming. Active constraint acquisition has been successfully used to sup- port non-experienced users in learning constraint networks through the generation of a sequence of queries. In this paper, we propose Learn&Optimize, a method to solve optimization problems with known objective function and unknown constraint network. It uses an active constraint acquisition algorithm which learns the unknown constraints and computes boundaries for the optimal solution during the learning process. As a result, our method allows users to solve op- timization problems without learning the overall constraint network.
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-LARGO) , Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-Largo) |
Publication Type | Proceedings, non-refereed |
Year of Publication | 2021 |
Conference Name | PTHG workshop in CP |
Book Chapter
Testing Industrial Robotic Systems: A New Battlefield!
In Software Engineering for Robotics, 109-137. Cham: Springer Nature, 2021.Status: Published
Testing Industrial Robotic Systems: A New Battlefield!
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-LARGO) , Department of Validation Intelligence for Autonomous Software Systems, AI4EU, Testing of Learning Robots (T-Largo) |
Publication Type | Book Chapter |
Year of Publication | 2021 |
Book Title | Software Engineering for Robotics |
Chapter | 5 |
Pagination | 109-137 |
Date Published | 06/2021 |
Publisher | Springer Nature |
Place Published | Cham |
ISBN Number | ISBN 978-3-030-66493-0 |
URL | https://www.springer.com/gp/book/9783030664930#aboutBook |
Journal Article
Adaptive Metamorphic Testing with Contextual Bandits
Journal of Systems and Software 165 (2020).Status: Published
Adaptive Metamorphic Testing with Contextual Bandits
Metamorphic Testing is a software testing paradigm which aims at using necessary properties of a system-under-test, called metamorphic relations, to either check its expected outputs, or to generate new test cases. Metamorphic Testing has been successful to test programs for which a full oracle is not available or to test programs for which there are uncertainties on expected outputs such as learning systems. In this article, we propose Adaptive Metamorphic Testing as a generalization of a simple yet powerful reinforcement learning technique, namely contextual bandits, to select one of the multiple metamorphic relations available for a program. By using contextual bandits, Adaptive Metamorphic Testing learns which metamorphic relations are likely to transform a source test case, such that it has higher chance to discover faults. We present experimental results over two major case studies in machine learning, namely image classification and object detection, and identify weaknesses and robustness boundaries. Adaptive Metamorphic Testing efficiently identifies weaknesses of the tested systems in context of the source test case.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI), Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo) |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | Journal of Systems and Software |
Volume | 165 |
Date Published | 07/2020 |
Publisher | Elsevier |
Other Numbers | arXiv:1910.00262 |
DOI | 10.1016/j.jss.2020.110574 |
ITE: A Lightweight Implementation of Stratified Reasoning for Constructive Logical Operators
International Journal on Artificial Intelligence Tools 29, no. 3-4 (2020): 23.Status: Published
ITE: A Lightweight Implementation of Stratified Reasoning for Constructive Logical Operators
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-LARGO) , Testing of Learning Robots (T-Largo), Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Journal Article |
Year of Publication | 2020 |
Journal | International Journal on Artificial Intelligence Tools |
Volume | 29 |
Issue | 3-4 |
Pagination | 23 |
Date Published | 06/2020 |
Publisher | World Scientific Publishing |
DOI | 10.1142/S0218213020600064. |
Talks, invited
AI4EU: Pilot Experiments with the platform
In Big Data Value Forum, Nov. 3-5, Berlin, Germany, 2020.Status: Published
AI4EU: Pilot Experiments with the platform
Afilliation | Software Engineering |
Project(s) | AI4EU, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Talks, invited |
Year of Publication | 2020 |
Location of Talk | Big Data Value Forum, Nov. 3-5, Berlin, Germany |
Type of Talk | https://www.european-big-data-value-forum.eu/ |
Metamorphic Testing: A Validation Technique for Trustworthy AI?
In Workshop on Lessons learnt on Trustworthy AI made in Europe: Challenges and Answers, 2020.Status: Published
Metamorphic Testing: A Validation Technique for Trustworthy AI?
Afilliation | Software Engineering |
Project(s) | AI4EU, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Talks, invited |
Year of Publication | 2020 |
Location of Talk | Workshop on Lessons learnt on Trustworthy AI made in Europe: Challenges and Answers |
Type of Talk | Online. Nov. 13th, 2020. Video available online: https://www.youtube.com/watch?v=B9rAbqcayok |
Poster
Detection of False Data Injection in AIS Vessels Communication Using Machine Learning
FEMTO-ST Institute, Besancon, France, 2020.Status: Published
Detection of False Data Injection in AIS Vessels Communication Using Machine Learning
Afilliation | Software Engineering |
Project(s) | T-Sar, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Poster |
Year of Publication | 2020 |
Place Published | FEMTO-ST Institute, Besancon, France |
Proceedings, refereed
Learning Objective Boundaries for Constraint Optimization Problems
In International Conference on Machine Learning, Optimization, and Data Science. Springer, 2020.Status: Published
Learning Objective Boundaries for Constraint Optimization Problems
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | International Conference on Machine Learning, Optimization, and Data Science |
Pagination | 394-408 |
Date Published | 01/2021 |
Publisher | Springer |
Other Numbers | arXiv:2006.11560 |
DOI | 10.1007/978-3-030-64580-9_33 |
Lessons Learned on Research Co-Creation: Making Industry-Academia Collaboration Work
In The 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). Euromicro, 2020.Status: Published
Lessons Learned on Research Co-Creation: Making Industry-Academia Collaboration Work
Afilliation | Software Engineering |
Project(s) | T3AS, The Certus Centre (SFI), Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | The 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) |
Publisher | Euromicro |
Neural Network Classification for Improving Continuous Regression Testing
In The IEEE Second International Conference On Artificial Intelligence Testing (AITest 2020). IEEE, 2020.Status: Published
Neural Network Classification for Improving Continuous Regression Testing
Afilliation | Software Engineering |
Project(s) | T3AS, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | The IEEE Second International Conference On Artificial Intelligence Testing (AITest 2020) |
Publisher | IEEE |
Opening the Software Engineering Toolbox for the Assessment of Trustworthy AI
In 1st International Workshop on New Foundations for Human-Centered AI @ ECAI 2020. CEUR Workshop Proceedings, 2020.Status: Published
Opening the Software Engineering Toolbox for the Assessment of Trustworthy AI
Trustworthiness is a central requirement for the acceptance and success of human-centered artificial intelligence (AI). To deem an AI system as trustworthy, it is crucial to assess its behaviour and characteristics against a gold standard of Trustworthy AI, consisting of guidelines, requirements, or only expectations. While AI systems are highly complex, their implementations are still based on software. The software engineering community has a long-established toolbox for the assessment of software systems, especially in the context of software testing. In this paper, we argue for the application of software engineering and testing practices for the assessment of trustworthy AI. We make the connection between the seven key requirements as defined by the European Commission's AI high-level expert group and established procedures from software engineering and raise questions for future work.
Afilliation | Software Engineering |
Project(s) | Department of Validation Intelligence for Autonomous Software Systems, AI4EU |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | 1st International Workshop on New Foundations for Human-Centered AI @ ECAI 2020 |
Pagination | 67-70 |
Date Published | 08/2020 |
Publisher | CEUR Workshop Proceedings |
Other Numbers | arXiv:2007.07768 |
URL | http://ceur-ws.org/Vol-2659/ahuja.pdf |
RobTest: A CP Approach to Generate Maximal Test Trajectories for Industrial Robots
In International Conference on Principles and Practice of Constraint Programming. LNCS ed. Vol. 12333. Springer, 2020.Status: Published
RobTest: A CP Approach to Generate Maximal Test Trajectories for Industrial Robots
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI), Testing of Learning Robots (T-LARGO) , Department of Validation Intelligence for Autonomous Software Systems, Testing of Learning Robots (T-Largo) |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | International Conference on Principles and Practice of Constraint Programming |
Volume | 12333 |
Edition | LNCS |
Pagination | 1-17 |
Date Published | 08/2020 |
Publisher | Springer |
Software Testing for Machine Learning
In The Thirty-Fourth AAAI Conference on Artificial Intelligence. AAAI'20: AAAI, 2020.Status: Published
Software Testing for Machine Learning
Afilliation | Software Engineering |
Project(s) | T3AS, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Proceedings, refereed |
Year of Publication | 2020 |
Conference Name | The Thirty-Fourth AAAI Conference on Artificial Intelligence |
Date Published | 2020 |
Publisher | AAAI |
Place Published | AAAI'20 |
Journal Article
A learning algorithm for optimizing continuous integration development and testing practice
Software: Practice and Experience 49, no. 2 (2019): 192-213.Status: Published
A learning algorithm for optimizing continuous integration development and testing practice
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2019 |
Journal | Software: Practice and Experience |
Volume | 49 |
Issue | 2 |
Pagination | 192-213 |
Publisher | Wiley Online Library |
Keywords | Continuous Integration, Decision Trees, Machine Leanring, Software Testing |
Talks, invited
AI4EU: Building The European AI-on-demand Platform
In Paris Open Source Summit (POSS'19), Dec. 10-11, 2019, Dock Pullmann, Paris, France, 2019.Status: Published
AI4EU: Building The European AI-on-demand Platform
Afilliation | Software Engineering |
Project(s) | AI4EU |
Publication Type | Talks, invited |
Year of Publication | 2019 |
Location of Talk | Paris Open Source Summit (POSS'19), Dec. 10-11, 2019, Dock Pullmann, Paris, France |
AI4EU: The European AI-On-Demand Platform
In Seminar AI in Retail, Paris, France, 2019.Status: Published
AI4EU: The European AI-On-Demand Platform
Afilliation | Software Engineering |
Project(s) | AI4EU |
Publication Type | Talks, invited |
Year of Publication | 2019 |
Location of Talk | Seminar AI in Retail, Paris, France |
URL | https://nlegrand3.wixsite.com/ia-by-picom |
Current Panorama of AI in Europe
In AI4EU Event AI Ecosystem Development Forum, Athens, Greece, 2019.Status: Published
Current Panorama of AI in Europe
Afilliation | Software Engineering |
Project(s) | AI4EU |
Publication Type | Talks, invited |
Year of Publication | 2019 |
Location of Talk | AI4EU Event AI Ecosystem Development Forum, Athens, Greece |
Type of Talk | Panel Discussion, Presentation of findings, https://www.youtube.com/watch?v=hd5itSaDh-o |
URL | https://www.youtube.com/watch?v=hd5itSaDh-o |
Testing Robotic Systems: A New Battlefield!
In RoboSoft: Software Engineering for Robotics - Royal Academy of Engineering, London, UK, 2019.Status: Published
Testing Robotic Systems: A New Battlefield!
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-Largo) |
Publication Type | Talks, invited |
Year of Publication | 2019 |
Location of Talk | RoboSoft: Software Engineering for Robotics - Royal Academy of Engineering, London, UK |
Type of Talk | https://www.cs.york.ac.uk/robostar/robosoft/ |
Proceedings, refereed
Challenges of Testing Machine Learning Based Systems
In Proceedings of the 1st IEEE Artificial Intelligence Testing Conference (AI Test 2019). San Francisco, CA, USA: IEEE, 2019.Status: Published
Challenges of Testing Machine Learning Based Systems
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI), Testing of Learning Robots (T-Largo), T3AS |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | Proceedings of the 1st IEEE Artificial Intelligence Testing Conference (AI Test 2019) |
Publisher | IEEE |
Place Published | San Francisco, CA, USA |
Deploying Smart Program Understanding on a Large Code Base
In 2019 IEEE International Conference On Artificial Intelligence Testing (AITest). IEEE, 2019.Status: Published
Deploying Smart Program Understanding on a Large Code Base
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 2019 IEEE International Conference On Artificial Intelligence Testing (AITest) |
Publisher | IEEE |
DOI | 10.1109/AITest.2019.000-4 |
Rotational Diversity in Multi-Cycle Assignment Problems
In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-19). Vol. 33. AAAI, 2019.Status: Published
Rotational Diversity in Multi-Cycle Assignment Problems
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-19) |
Volume | 33 |
Pagination | 7724-7731 |
Publisher | AAAI |
Other Numbers | arXiv:1811.03496 |
Stress Testing of Single-Arm Robots Through Constraint-Based Generation of Continuous Trajectories
In 2019 IEEE International Conference On Artificial Intelligence Testing (AITest). San Francisco, CA, USA: IEEE, 2019.Status: Published
Stress Testing of Single-Arm Robots Through Constraint-Based Generation of Continuous Trajectories
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2019 |
Conference Name | 2019 IEEE International Conference On Artificial Intelligence Testing (AITest) |
Pagination | 121-128 |
Date Published | 04/2019 |
Publisher | IEEE |
Place Published | San Francisco, CA, USA |
Talk, keynote
Intelligence Testing of Autonomous Software Systems
In 31st IFIP WG6.1 International Conference, ICTSS 2019, Paris, France. Springer LNCS 11812, 2019.Status: Published
Intelligence Testing of Autonomous Software Systems
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-Largo) |
Publication Type | Talk, keynote |
Year of Publication | 2019 |
Location of Talk | 31st IFIP WG6.1 International Conference, ICTSS 2019, Paris, France |
Publisher | Springer LNCS 11812 |
Type of Talk | Invited Keynote |
ISSN Number | 0302-9743 |
DOI |
Technical reports
SARCoS: Machine Learning Models for the Selection of Security Test Scenarios
Simula Research Laboratory, 2019.Status: Accepted
SARCoS: Machine Learning Models for the Selection of Security Test Scenarios
Afilliation | Software Engineering |
Project(s) | T-Sar |
Publication Type | Technical reports |
Year of Publication | 2019 |
Publisher | Simula Research Laboratory |
Survey on Testing of Deep Learning Systems
Simula Research Laboratory, 2019.Status: Published
Survey on Testing of Deep Learning Systems
Recent studies have shown that deep learning algorithms used for image classification or object recognition are not sufficiently reliable. These algorithms can be easily fooled by applying perturbations to images or generating artificial images that result in misclassification. In this paper, we provide an overview of software testing methods present in literature to test deep learning systems. We have explored different methods of testing deep neural networks, namely metamorphic testing, mutation testing, differential testing, and adversarial perturbation testing. We present the main findings available from the literature and compare these methods systematically and comprehensively. The results show that systematic testing of deep learning systems can further help to increase the performance of state-of-the-art systems.
Afilliation | Software Engineering |
Project(s) | Testing of Learning Robots (T-Largo) |
Publication Type | Technical reports |
Year of Publication | 2019 |
Publisher | Simula Research Laboratory |
Talks, contributed
Simula Research Laboratory: Constraint Programming for Software Engineering
In NordConsNet Workshop 2019 - The 18th workshop of NordConsNet, the Nordic Network for researchers and practitioners of Constraint Programming, Norway, 2019.Status: Published
Simula Research Laboratory: Constraint Programming for Software Engineering
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI), Testing of Learning Robots (T-Largo) |
Publication Type | Talks, contributed |
Year of Publication | 2019 |
Location of Talk | NordConsNet Workshop 2019 - The 18th workshop of NordConsNet, the Nordic Network for researchers and practitioners of Constraint Programming, Norway |
Statistics AIS Dataset from Statsat
In Simula Research Laboratory, 2019.Status: Published
Statistics AIS Dataset from Statsat
Afilliation | Software Engineering, Machine Learning |
Project(s) | T-Sar, Department of Validation Intelligence for Autonomous Software Systems |
Publication Type | Talks, contributed |
Year of Publication | 2019 |
Location of Talk | Simula Research Laboratory |
The TSAR Project
In 14th Certus User Partner Workshop, Sep. 2019, Larvik, Norway, 2019.Status: Published
The TSAR Project
Afilliation | Software Engineering |
Project(s) | T-Sar |
Publication Type | Talks, contributed |
Year of Publication | 2019 |
Location of Talk | 14th Certus User Partner Workshop, Sep. 2019, Larvik, Norway |
Time-aware Test Execution Scheduling for Cyber-Physical Systems
In Gesellschaft für Informatik Software Engineering Conference 2019 (SE19), Germany, 2019.Status: Published
Time-aware Test Execution Scheduling for Cyber-Physical Systems
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, contributed |
Year of Publication | 2019 |
Location of Talk | Gesellschaft für Informatik Software Engineering Conference 2019 (SE19), Germany |
Talk, keynote
AI-Powered Testing of Industrial Robots
In ALTEN TalentCamp, Paris, France, 2018.Status: Published
AI-Powered Testing of Industrial Robots
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talk, keynote |
Year of Publication | 2018 |
Location of Talk | ALTEN TalentCamp, Paris, France |
Type of Talk | Invited Talk |
Artificial Intelligence in Software Testing: An Overview. Application to Industrial Robotics
In French Days on Software Testing (JFTL'18), Paris, France, 2018.Status: Published
Artificial Intelligence in Software Testing: An Overview. Application to Industrial Robotics
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talk, keynote |
Year of Publication | 2018 |
Location of Talk | French Days on Software Testing (JFTL'18), Paris, France |
Type of Talk | Invited Keynote Speech |
Talks, invited
Boundary Estimation: Learning Boundaries for Constraint Optimization Problems
In International Symposium on Mathematical Optimization (ISMP'18), Bordeaux, France, 2018.Status: Published
Boundary Estimation: Learning Boundaries for Constraint Optimization Problems
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, invited |
Year of Publication | 2018 |
Location of Talk | International Symposium on Mathematical Optimization (ISMP'18), Bordeaux, France |
Type of Talk | Invited Talk |
Keywords | learning theory, Optimization |
Journal Article
Certus: an organizational effort towards research-based innovation in software verification and validation
International Journal of Systems Assurance Engineering and Management 9, no. 2 (2018): 313-322.Status: Published
Certus: an organizational effort towards research-based innovation in software verification and validation
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | International Journal of Systems Assurance Engineering and Management |
Volume | 9 |
Issue | 2 |
Pagination | 313-322 |
Publisher | Springer |
Discovering Program Topoi via Hierarchical Agglomerative Clustering
IEEE Transactions on Reliability 67, no. 3 (2018): 758-770.Status: Published
Discovering Program Topoi via Hierarchical Agglomerative Clustering
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2018 |
Journal | IEEE Transactions on Reliability |
Volume | 67 |
Issue | 3 |
Pagination | 758 - 770 |
Date Published | Jan-01-2018 |
Publisher | IEEE Reliability Society |
ISSN | 0018-9529 |
URL | https://ieeexplore.ieee.org/document/8359317/http://xplorestaging.ieee.o... |
DOI | 10.1109/TR.2018.2828135 |
Proceedings, refereed
Constraint-based Generation of Trajectories for Single-Arm Robots
In Doctoral Program of the 24th Constraint Programming (DP-CP 18). Lille, France, 2018.Status: Published
Constraint-based Generation of Trajectories for Single-Arm Robots
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Doctoral Program of the 24th Constraint Programming (DP-CP 18) |
Place Published | Lille, France |
Discovering Program Topoi Through Clustering
In Proceedings of the Thirty-Second IAAI/AAAI Conference on Innovative Applications of Artificial Intelligence. AAAI Press, 2018.Status: Published
Discovering Program Topoi Through Clustering
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proceedings of the Thirty-Second IAAI/AAAI Conference on Innovative Applications of Artificial Intelligence |
Publisher | AAAI Press |
URL | https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16045 |
Stratified Constructive Disjunction and Negation in Constraint Programming
In Proc. of IEEE Int. Conf. on Tools with Artificial Intelligence (ICTAI-18). Volos, Greece. Nov. 2018. IEEE, 2018.Status: Published
Stratified Constructive Disjunction and Negation in Constraint Programming
Constraint Programming (CP) is a powerful declarative programming paradigm combining inference and search in order to find solutions to various type of constraint systems. Dealing with highly disjunctive constraint systems is notoriously difficult in CP. Apart from trying to solve each disjunct independently from each other, there is little hope and effort to succeed in constructing intermediate results combining the knowledge originating from several disjuncts. In this paper, we propose If Then Else (ITE), a lightweight approach for implementing stratified constructive disjunction and negation on top of an existing CP solver, namely SICStus Prolog clp(FD). Although constructive disjunction is known for more than three decades, it does not have straightforward implementations in most CP solvers. ITE is a freely available library proposing stratified and constructive reasoning for various operators, including disjunction and negation, implication and conditional. Our preliminary experimental results show that ITE is competitive with existing approaches that handle disjunctive constraint systems.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Proc. of IEEE Int. Conf. on Tools with Artificial Intelligence (ICTAI-18). Volos, Greece. Nov. 2018 |
Pagination | 106-113 |
Publisher | IEEE |
Other Numbers | arXiv:1811.03906 |
DOI | 10.1109/ICTAI.2018.00026 |
Towards Hybrid Constraint Solving with Reinforcement Learning and Constraint-Based Local Search
In Data Science meets Optimization Workshop at Federated Artificial Intelligence Meeting, 2018.Status: Published
Towards Hybrid Constraint Solving with Reinforcement Learning and Constraint-Based Local Search
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2018 |
Conference Name | Data Science meets Optimization Workshop at Federated Artificial Intelligence Meeting |
Poster
Constraint-Based Generation of Trajectories for single-Arm Robots
DP-CP, Lille, France, 2018.Status: Published
Constraint-Based Generation of Trajectories for single-Arm Robots
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Poster |
Year of Publication | 2018 |
Date Published | 08/2019 |
Place Published | DP-CP, Lille, France |
Different Cycle, Different Assignment: Diversity in Assignment Problems with Multiple Cycles
AAAI-18, New Orleans, Louisiana, USA, 2018.Status: Published
Different Cycle, Different Assignment: Diversity in Assignment Problems with Multiple Cycles
We present approaches to handle diverse assignments in multi-cycle assignment problems. The goal is to assign a task to different agents in each cycle, such that all possible combinations are made over time. Our method combines the original profit value, that is to be optimized by the assignment problem with an additional assignment preference. By merging both, we steer the optimization towards diverse assignments without large trade-offs in the original profits.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Poster |
Year of Publication | 2018 |
Place Published | AAAI-18, New Orleans, Louisiana, USA |
Type of Work | Student Abstract |
Talks, contributed
Estimating Objective Boundaries for Constraint Optimization Problems
In NordConsNet Workshop, Gothenburg, Sweden, 2018.Status: Published
Estimating Objective Boundaries for Constraint Optimization Problems
Solving Constraint Optimization Problems (COP) requires exploring a large search space. By providing objective boundaries, this space can be pruned. Finding close boundaries, that correctly under- or overestimate the optimum, is difficult without having a heuristic function for the COP. We present a method for learning to estimate boundaries from problem instances using machine learning. The trained model can then estimate boundaries for unseen instances and thereby support the constraint solver through an additional boundary constraint.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, contributed |
Year of Publication | 2018 |
Location of Talk | NordConsNet Workshop, Gothenburg, Sweden |
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
In Gesellschaft für Informatik Software Engineering Conference 2018 (SE18), Ulm, Germany, 2018.Status: Published
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, contributed |
Year of Publication | 2018 |
Location of Talk | Gesellschaft für Informatik Software Engineering Conference 2018 (SE18), Ulm, Germany |
Proceedings, refereed
Constraint-Based Verification of a Mobile App Game Designed for Nudging People to Attend Cancer Screening
In Proceedings of Innovative Applications of Artificial Intelligence (IAAI-17), Feb. 2017, San Francisco, USA. AAAI, 2017.Status: Published
Constraint-Based Verification of a Mobile App Game Designed for Nudging People to Attend Cancer Screening
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | Proceedings of Innovative Applications of Artificial Intelligence (IAAI-17), Feb. 2017, San Francisco, USA |
Date Published | 02/2017 |
Publisher | AAAI |
Efficient and Complete FD-Solving for Extended Array Constraints
In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. IJCAI, 2017.Status: Published
Efficient and Complete FD-Solving for Extended Array Constraints
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence |
Pagination | 1231-1238 |
Date Published | 08/2017 |
Publisher | IJCAI |
DOI | 10.24963/ijcai.2017/171 |
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. New York, NY, USA: ACM, 2017.Status: Published
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis |
Pagination | 12-22 |
Publisher | ACM |
Place Published | New York, NY, USA |
ISBN Number | 978-1-4503-5076-1 |
Other Numbers | arXiv:1811.04122 |
DOI | 10.1145/3092703.3092709 |
Time-aware Test Case Execution Scheduling for Cyber-Physical Systems
In Proceedings of Principles of Constraint Programming (CP'17). Springer, 2017.Status: Published
Time-aware Test Case Execution Scheduling for Cyber-Physical Systems
Testing cyber-physical systems involves the execution of test cases on target-machines equipped with the latest release of a software control system. When testing industrial robots, it is common that the target machines need to share some common resources, e.g., costly hardware devices, and so there is a need to schedule test case execution on the target machines, accounting for these shared resources. With a large number of such tests executed on a regular basis, this scheduling becomes difficult to manage manually. In fact, with manual test execution planning and scheduling, some robots may remain unoccupied for long periods of time and some test cases may not be executed. This paper introduces TC-Sched, a time-aware method for automated test case execution scheduling. TC-Sched uses Constraint Programming to schedule tests to run on multiple machines constrained by the tests’ access to shared resources, such as measurement or networking devices. The CP model is written in SICStus Prolog and uses the Cumulatives global constraint. Given a set of test cases, a set of machines, and a set of shared resources, TC-Sched produces an execution schedule where each test is executed once with minimal time between when a source code change is committed and the test results are reported to the developer. Experiments reveal that TC-Sched can schedule 500 test cases over 100 machines in less than 4 minutes for 99.5% of the instances. In addition, TC-Sched largely outperforms simpler methods based on a greedy algorithm and is suitable for deployment on industrial robot testing.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | Proceedings of Principles of Constraint Programming (CP'17) |
Publisher | Springer |
Other Numbers | arXiv:1902.04627 |
DOI | 10.1007/978-3-319-66158-2_25 |
TITAN: Test Suite Optimization for Highly Configurable Software
In International Conference on Software Testing, Verification and Validation (ICST 2017) . IEEE, 2017.Status: Published
TITAN: Test Suite Optimization for Highly Configurable Software
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2017 |
Conference Name | International Conference on Software Testing, Verification and Validation (ICST 2017) |
Publisher | IEEE |
ISBN Number | 978-1-5090-6031-3 |
DOI | 10.1109/ICST.2017.60 |
Journal Article
Deploying Constraint Programming for Testing ABB’s Painting Robots
AI Magazine 39, no. Summer (2017).Status: Published
Deploying Constraint Programming for Testing ABB’s Painting Robots
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI), ABB Robotics: Automated Software Testing of Complex Industrial Robotics Systems |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | AI Magazine |
Volume | 39 |
Issue | Summer |
Number | 2 |
Publisher | AAAI |
Notes | Column on AI in Industry |
DOI | 10.1609/aimag.v38i2.2723 |
Using Global Constraints to Automate Regression Testing
AI Magazine 38, no. Spring (2017).Status: Published
Using Global Constraints to Automate Regression Testing
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2017 |
Journal | AI Magazine |
Volume | 38 |
Issue | Spring |
Number | 1 |
Publisher | AAAI |
Talks, invited
Intelligent Test Optimization
In TAROT Summer School on Software Testing, Jun. 26th, Naples, Italy, 2017.Status: Published
Intelligent Test Optimization
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, invited |
Year of Publication | 2017 |
Location of Talk | TAROT Summer School on Software Testing, Jun. 26th, Naples, Italy |
Talks, contributed
Learning How to Test Robotic Systems
In Simula-HiOA Seminar April 5th, Fornebu, Lysaker, Norway, 2017.Status: Published
Learning How to Test Robotic Systems
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, contributed |
Year of Publication | 2017 |
Location of Talk | Simula-HiOA Seminar April 5th, Fornebu, Lysaker, Norway |
Book Chapter
Software Product Line Test Suite Reduction with Constraint Optimization
In Software Technologies, 68-87. Vol. 743. Springer International Publishing, 2017.Status: Published
Software Product Line Test Suite Reduction with Constraint Optimization
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Book Chapter |
Year of Publication | 2017 |
Book Title | Software Technologies |
Volume | 743 |
Pagination | 68-87 |
Publisher | Springer International Publishing |
Talk, keynote
Testing Robotic Systems: A New Battlefield!
In French National Days of GDR-GPL 2017, Montpellier, France, 2017.Status: Published
Testing Robotic Systems: A New Battlefield!
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talk, keynote |
Year of Publication | 2017 |
Location of Talk | French National Days of GDR-GPL 2017, Montpellier, France |
Date Published | 06/2017 |
Type of Talk | Keynote Speech |
Proceedings, refereed
A New Approach to Feature-based Test Suite Reduction in Software Product Line Testing
In ICSOFT-EA 2016, 11th Int. Conf. on Software Engineering and Applications, Lisbon, July 2016, Awarded Best Paper. INSTICC Press, 2016.Status: Published
A New Approach to Feature-based Test Suite Reduction in Software Product Line Testing
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | ICSOFT-EA 2016, 11th Int. Conf. on Software Engineering and Applications, Lisbon, July 2016, Awarded Best Paper |
Date Published | 07/2016 |
Publisher | INSTICC Press |
Automated Regression Testing Using Constraint Programming
In Twenty-Eighth Conference on Innovative Applications of Artificial Intelligence (IAAI-16), Phoenix, AZ, USA, Feb. 2016, 2016.Status: Published
Automated Regression Testing Using Constraint Programming
In software validation, regression testing aims to check the absence of regression faults in new releases of a software system. Typically, test cases used in regression testing are executed during a limited amount of time and are selected to check a given set of user requirements. When testing large systems the number of regression tests grows quickly over the years, and yet the available time slot stays limited. In order to overcome this problem, an approach known as test suite reduction (TSR), has been developed in software engineering to select a smallest subset of test cases, so that each requirement remains covered at least once. However solving the TSR problem is difficult as the underlying optimization problem is NP-hard, but it is also crucial for vendors interested in reducing the time to market of new software releases.
In this paper, we address regression testing and TSR with Constraint Programming (CP). More specifically, we propose new CP models to solve TSR that exploit global constraints, namely NValue and GCC. We reuse a set of preprocessing rules to reduce a priori each instance, and we introduce a structure-aware search heuristic. We evaluated our CP models and proposed improvements against existing approaches, including a simple greedy approach and MINTS, the state-of-the-art tool of the software engineering community. Our experiments show that CP outperforms both the greedy approach and MINTS when it is interfaced with MiniSAT, in terms of percentage of reduction and execution time. When MINTS is interfaced with CPLEX, we show that our CP model performs better only on percentage of reduction. Finally, by working closely with validation engineers from Cisco Systems, Norway, we integrated our CP model into an industrial regression testing process.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Twenty-Eighth Conference on Innovative Applications of Artificial Intelligence (IAAI-16), Phoenix, AZ, USA, Feb. 2016 |
Date Published | 02/2016 |
Generating Tests for Robotized Painting Using Constraint Programming
In Int. Joint Conf. on Artificial Intelligence (IJCAI-16) - Sister Conference Best Paper Track. New York City, 2016.Status: Published
Generating Tests for Robotized Painting Using Constraint Programming
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Int. Joint Conf. on Artificial Intelligence (IJCAI-16) - Sister Conference Best Paper Track |
Date Published | 07/2016 |
Place Published | New York City |
Optimal Test Execution Scheduling on Multiple Machines with Resource Constraints
In Submitted, 2016.Status: Submitted
Optimal Test Execution Scheduling on Multiple Machines with Resource Constraints
When testing large-scale systems with hundreds of test cases in continuous integration, it is crucial to minimize the round-trip time, that is, the time from when a source code change is committed until the test results are reported back to the developer. To this end, scheduling as many test case executions as possible in the minimum amount of time is essential to increasing the effectiveness of continuous integration. This paper introduces TSNIC, a time-aware method for test execution scheduling on multiple machines with constraints on accessible resources, such as measurement devices or network equipment. The method uses as input a test suite, a set of machines, and a set of shared resources and produces an execution schedule. The schedule guarantees that each test will be executed once and minimizes the round-trip time. TSNIC has undergone extensive experimental evaluation using generated test suites derived from existing industrial test suites augmented with randomly-selected values. Our results provide evidence that TSNIC conducts effective test execution scheduling and is suitable for deployment in continuous integration. In particular, we show that automatic optimal scheduling of 500 test cases over 100 machines is reacheable in less than 4 minutes for 96.6% of the instances.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2016 |
Conference Name | Submitted |
Journal Article
A Systematic Test Case Selection Methodology for Product Lines: Results and Insights From an Industrial Case Study
Empirical Software Engineering 21, no. 4 (2016): 1586-1622.Status: Published
A Systematic Test Case Selection Methodology for Product Lines: Results and Insights From an Industrial Case Study
In the context of product lines, test case selection aims at obtaining a set of relevant test cases for a product from the entire set of test cases available for a product line. While working on a research-based innovation project on automated testing of product lines of Video Conferencing Systems (VCSs) developed by Cisco, we felt the need to devise a cost-effective way of selecting relevant test cases for a product. To fulfill such need, we propose a systematic and automated test selection methodology using: 1) Feature Model for Testing (FM\_T) to capture commonalities and variabilities of a product line; 2) Component Family Model for Testing (CFM\_T) to model the structure of test case repository; 3) A tool to automatically build restrictions from CFM\_T to FM\_T and traces from CFM\_T to the actual test cases. Using our methodology, a test engineer is only required to select relevant features through FM\_T at a higher level of abstraction for a product and the corresponding test cases will be obtained automatically. We evaluate our methodology by applying it to a VCS product line called Saturn with seven commercial products and the results show that our methodology can significantly reduce cost measured as test selection time and at the same time achieves higher effectiveness (feature coverage, feature pairwise coverage and fault detection) as compared with the current manual process. Moreover, we conduct a questionnaire-based study to solicit the views of test engineers who are involved in developing FM\_T and CFM\_T. The results show that test engineers are positive about adapting our methodology in their current practice. Finally, we present a set of lessons learnt while applying product line engineering at Cisco for test case selection.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | Empirical Software Engineering |
Volume | 21 |
Issue | 4 |
Pagination | 1586-1622 |
Date Published | 08/2016 |
Publisher | Springer |
Exploiting Binary Floating-Point Representations for Constraint Propagation
INFORMS Journal of Computing (JoC) 28, no. 1 (2016): 31-46.Status: Published
Exploiting Binary Floating-Point Representations for Constraint Propagation
Floating-point computations are quickly finding their way in the design of safety- and mission-critical systems, despite the fact that designing floating-point algorithms is significantly more difficult than designing integer algorithms. For this reason, verification and validation of floating-point computations is a hot research topic. An important verification technique, especially in some industrial sectors, is testing. However, generating test data for floating-point intensive programs proved to be a challenging problem. Existing approaches usually resort to random or search-based test data generation, but without symbolic reasoning it is almost impossible to generate test inputs that execute complex paths controlled by floating-point computations. Moreover, as constraint solvers over the reals or the rationals do not natively support the handling of rounding errors, the need arises for efficient constraint solvers over floating-point domains. In this paper, we present and fully justify improved algorithms for the filtering of arithmetic IEEE 754 binary floating-point constraints. The key point of these algorithms is a generalization of an idea by B. Marre and C. Michel that exploits a property of the representation of floating-point numbers.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | INFORMS Journal of Computing (JoC) |
Volume | 28 |
Issue | 1 |
Pagination | 31-46 |
Date Published | 01/2016 |
Publisher | INFORMS Journal of Computing (JoC) |
Practical Minimization of Pairwise-Covering Test Configurations Using Constraint Programming
Information and Software Technology 71 (2016): 129-146.Status: Published
Practical Minimization of Pairwise-Covering Test Configurations Using Constraint Programming
Context: Testing highly-configurable software systems is challenging due to a large number of test configurations that have to be carefully selected in order to reduce the testing effort as much as possible, while maintaining high software quality. Finding the smallest set of valid test configurations that ensure sufficient coverage of the system's feature interactions is thus the objective of validation engineers, especially when the execution of test configurations is costly or time-consuming. However, this problem is NP-hard in general and approximation algorithms have often been used to address it in practice.
Objective: In this paper, we explore an alternative exact approach based on constraint programming that will allow engineers to increase the effectiveness of configuration testing while keeping the number of configurations as low as possible.
Method: Our approach consists in using a (time-aware) minimization algorithm based on constraint programming. Given the amount of time, our solution generates a minimized set of valid test configurations that ensure coverage of all pairs of feature values (a.k.a. pairwise coverage). The approach has been implemented in a tool called PACOGEN.
Results: PACOGEN was evaluated on 224 feature models from the standard benchmark repository SPLOT, and compared
in comparison with the two existing tools that are based on a greedy algorithm. For 79% of 224 feature models, PACOGEN generated up to 60% fewer test configurations than the competitor tools. We further evaluated PACOGEN in the case study of an industrial video conferencing product line with a feature model of 169 features, and found 60% fewer configurations compared with the manual approach followed by test engineers. The set of test configurations generated by PACOGEN decreased the time required by test engineers in manual test configuration by 85%, increasing the feature-pairs coverage at the same time
Conclusion: Our experimental evaluation concluded that optimal time-aware minimization of pairwise-covering test configurations is efficiently addressed using constraint programming techniques.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2016 |
Journal | Information and Software Technology |
Volume | 71 |
Pagination | 129-146 |
Date Published | 03/2016 |
Publisher | Elsevier |
Talk, keynote
Constraint-Based Test Suite Optimization
In 28th International Conference on Testing Software and Systems (ICTSS'16), October 17-19, 2016, Graz, Austria., 2016.Status: Published
Constraint-Based Test Suite Optimization
Test suite optimization is a crucial topic in software testing which was recently boosted by the contributions of constraint programming and search-based algorithms. The increased complexity of testing procedures and the combinatorial nature of the underlying testing problems, namely (multi-criteria) test suite reduction, prioritization and scheduling requires the usage of advanced techniques which have been developed in other contexts. In this talk, I will review some of these advances and their application to real-world testing problems that we address in Certus, the Norwegian research-based innovation centre dedicated to Software Validation and Verification.
Afilliation | Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talk, keynote |
Year of Publication | 2016 |
Location of Talk | 28th International Conference on Testing Software and Systems (ICTSS'16), October 17-19, 2016, Graz, Austria. |
Date Published | 10/2016 |
Type of Talk | Invited Keynote Speaker |
Talks, invited
Constraint-Based Testing: An Emerging Trend in Software Testing
In XVI Jornadas sobre Programación y Lenguajes (PROLE 2016), Salamanca, Spain, 14-16 Sep. 2016, 2016.Status: Published
Constraint-Based Testing: An Emerging Trend in Software Testing
Software Testing involves the development of smart techniques to automatically find test inputs which uncover faults in programs. An emerging trend in this area, called ``Constraint-Based Testing'', aims at exploiting constraint solving techniques for this goal. My talk will review different techniques including dynamic symbolic execution, path-based exploration and constraint-based exploration and will emphasize the usage of advanced Constraint Programming tools.
Afilliation | Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, invited |
Year of Publication | 2016 |
Location of Talk | XVI Jornadas sobre Programación y Lenguajes (PROLE 2016), Salamanca, Spain, 14-16 Sep. 2016 |
Type of Talk | Invited Speaker |
URL | http://prole2016.webs.upv.es/ |
Software Testing Applications with Constraint Optimization
In SICS, Smart Programming Day, 2016, Nov. 29th, Stockholm, Sweden, 2016.Status: Published
Software Testing Applications with Constraint Optimization
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, invited |
Year of Publication | 2016 |
Location of Talk | SICS, Smart Programming Day, 2016, Nov. 29th, Stockholm, Sweden |
Talks, invited
A CP approach of the variability testing of software product lines
In Université Paris 1 Panthéon - La Sorbonne, 2015.Status: Published
A CP approach of the variability testing of software product lines
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Université Paris 1 Panthéon - La Sorbonne |
Type of Talk | Invited talk |
An Overview of Constraint-Based Testing
In Centre de Recherche en Informatique de Montreal (CRIM), 2015.Status: Published
An Overview of Constraint-Based Testing
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Centre de Recherche en Informatique de Montreal (CRIM) |
Type of Talk | Invited |
Global Constraints in Software Testing Applications
In Université de Montpellier, France - LIRMM, 2015.Status: Published
Global Constraints in Software Testing Applications
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, invited |
Year of Publication | 2015 |
Location of Talk | Université de Montpellier, France - LIRMM |
Type of Talk | Invited talk |
Journal Article
Automated Product Line Test Case Selection: Industrial Case Study and Controlled Experiment
Journal of Software and Systems Modeling (2015): 1-25.Status: Published
Automated Product Line Test Case Selection: Industrial Case Study and Controlled Experiment
Automated test case selection for a new product in a product line is challenging due to several reasons. First, the variability within the product line needs to be captured in a systematic way; second, the reusable test cases from the repository are required to be identified for testing a new product. The objective of such automated process is to reduce the overall effort for selection (e.g., selection time), while achieving an acceptable level of the coverage of testing functionalities. In this paper, we propose a systematic and automated methodology using a Feature Model for Testing (FM_T) to capture commonalities and variabilities of a product line and a Component Family Model for Testing (CFM_T) to capture the overall structure of test cases in the repository. With our methodology, a test engineer does not need to manually go through the repository to select a relevant set of test cases for a new product. Instead, a test engineer only needs to select a set of relevant features using FM_T at a higher level of abstraction for a product and a set of relevant test cases will be selected automatically. We evaluated our methodology via three different ways: 1) We applied our methodology to a product line of video conferencing systems called Saturn developed by Cisco and the results show that our methodology can reduce the selection effort significantly; 2) We conducted a questionnaire-based study to solicit the views of test engineers who were involved in developing FM_T and CFM_T. The results show that test engineers are positive about adapting our methodology and models (FM_T and CFM_T) in their current practice; 3) We conducted a controlled experiment with 20 graduate students to assess the performance (i.e., cost, effectiveness and efficiency) of our automated methodology as compared with the manual approach. The results showed that our methodology is cost-effective as compared with the manual approach and at the same time its efficiency is not affected by the increased complexity of products.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | Journal of Software and Systems Modeling |
Pagination | 1-25 |
Date Published | 04/2015 |
Publisher | Springer |
Certus: Glimpses of a Centre for Research-Based Innovation in Software Verification and Validation
International Journal of System Assurance Engineering and Management (2015): 1-25.Status: Published
Certus: Glimpses of a Centre for Research-Based Innovation in Software Verification and Validation
What is gratifying to a software engineering researcher? Three of many possible answers to this question are (a) the intellectual exercise in developing/disseminating approaches that address emerging and existing challenges, (b) recognition from impact in a community of researchers and (c) widespread use of novel ideas, including software, in the society at large leading to enhancement of human ability and job creation. A culmination of these sources requires an organizational effort. This article presents a detailed account of a research-based innovation centre, Certus, to facilitate such a culmination for software engineering researchers. Certus has established a body of knowledge, methods and tools for the validation and verification of software systems in the Norwegian private and public sector. Certus works in close cooperation with five founding user partners and is hosted by the Simula Research Laboratory. We present the organizational structure of Certus and describe how Certus’s life and health is planned and evaluated on a regular basis as a research-based innovation centre. We expound two successful collaborations, with (a) the private sector entity Cisco systems, Norway and (b) the public sector entity the Norwegian Customs and Excise. We hope that this document will serve as a basis to encourage national/international funding schemes to create call for proposals for long-term research-based innovation centres. This, we believe, is one way to justify use of tax payers resources in creating a win–win situation for the triple helix: government, researchers and industry.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | International Journal of System Assurance Engineering and Management |
Pagination | 1-25 |
Date Published | 03/2015 |
Publisher | Springer |
Combining Genetic Algorithms and Constraint Programming to Support Stress Testing of Task Deadlines
ACM Transactions on Software Engineering and Methodology (TOSEM) 25, no. 1 (2015): 1-37.Status: Published
Combining Genetic Algorithms and Constraint Programming to Support Stress Testing of Task Deadlines
Tasks in Real Time Embedded Systems (RTES) are often subject to hard deadlines, that constrain how quickly the system must react to external inputs. These inputs and their timing vary in a large domain depending on the environment state, and can never be fully predicted prior to system execution. Therefore, approaches for stress testing must be developed to uncover possible deadline misses of tasks for different input arrival times. In this paper, we describe stress test case generation as a search problem over the space of task arrival times. Specifically, we search for worst case scenarios maximizing deadline misses where each scenario characterizes a test case. In order to scale our search to large industrial-size problems, we combine two state-of-the-art search strategies, namely Genetic Algorithms (GA) and Constraint Programming (CP). Our experimental results show that, in comparison with GA and CP in isolation, GA+CP achieves nearly the same effectiveness as CP and the same efficiency and solution diversity as GA, thus combining the advantages of the two strategies. In light of these results, we conclude that a combined GA+CP approach to stress testing is more likely to scale to large and complex systems.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | ACM Transactions on Software Engineering and Methodology (TOSEM) |
Volume | 25 |
Issue | 1 |
Pagination | 1-37 |
Date Published | 12/2015 |
Publisher | ACM |
DOI | 10.1145/2818640 |
Cost-Effective Test Suite Minimization in Product Lines Using Search Techniques
Journal of Systems and Software 103 (2015): 370-391.Status: Published
Cost-Effective Test Suite Minimization in Product Lines Using Search Techniques
Cost-effective testing of a product in a product line requires obtaining a set of relevant test cases from the entire test suite via test selection and minimization techniques. In this paper, we particularly focus on test minimization for product lines, which identifies and eliminates redundant test cases from test suites in order to reduce the total number of test cases to execute, thereby improving the efficiency of testing. However, such minimization may result in the minimized test suite with low test coverage, low fault revealing capability, low priority test cases, and require more time than the allowed testing budget (e.g., time) as compared to the original test suite. To deal with the above issues, we formulated the minimization problem as a search problem and defined a fitness function considering various optimization objectives based on the above issues. To assess the performance of our fitness function, we conducted an extensive empirical evaluation by investigating the fitness function with three weight-based Genetic Algorithms (GAs) and seven multi-objective search algorithms using an industrial case study and 500 artificial problems inspired from the industrial case study. The results show that Random-Weighted Genetic Algorithm (RWGA) significantly outperforms the other algorithms since RWGA can balance all the objectives together by dynamically updating weights during each generation. Based on the results of our empirical evaluation, we also implemented a tool called TEst Minimization using Search Algorithms (TEMSA) to support test minimization using various search algorithms in the context of product lines.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | Journal of Systems and Software |
Volume | 103 |
Pagination | 370–391 |
Date Published | 05/2015 |
Publisher | Elsevier |
DOI | 10.1016/j.jss.2014.08.024 |
Focus section on quality software
Software, Practice and Experience 45, no. 7 (2015): 873-874.Status: Published
Focus section on quality software
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | Software, Practice and Experience |
Volume | 45 |
Issue | 7 |
Pagination | 873-874 |
Date Published | 07/2015 |
Publisher | John Wiley & Sons |
Infeasible Path Generalization in Dynamic Symbolic Execution
Information and Software Technology 58 (2015): 403-418.Status: Published
Infeasible Path Generalization in Dynamic Symbolic Execution
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | Information and Software Technology |
Volume | 58 |
Pagination | 403–418 |
Date Published | 02/2015 |
Publisher | Elsevier |
Testing Robot Controllers using Constraint Programming and Continuous Integration
Information and Software Technology 57 (2015): 169-185.Status: Published
Testing Robot Controllers using Constraint Programming and Continuous Integration
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2015 |
Journal | Information and Software Technology |
Volume | 57 |
Pagination | 169-185 |
Date Published | 01/2015 |
Publisher | Elsevier |
Book Chapter
Constraint-Based Testing: An Emerging Trend in Software Testing
In Advances in Computers, 67-101. ADCOM, UK: Academic Press ed. Vol. 99. Elsevier, 2015.Status: Published
Constraint-Based Testing: An Emerging Trend in Software Testing
Constraint-based testing is an emerging trend in software testing research and innovation. The general idea beneath this testing paradigm is to exploit a constraint solver to calculate the test inputs to be used for testing a software-system, so that the fault-revealing capabilities of the testing process can be improved. The incredible progresses achieved these last years on the development of powerful constraint solvers have fostered the emergence of effective and efficient constraint-based testing techniques. This chapter reviews in detail the main achievements reached in the field by illustrating them on simple examples and mentioning the main bibliographic references.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Book Chapter |
Year of Publication | 2015 |
Book Title | Advances in Computers |
Volume | 99 |
Edition | ADCOM, UK: Academic Press |
Chapter | 2 |
Pagination | 67-101 |
Date Published | 10/2015 |
Publisher | Elsevier |
ISBN Number | 978-0-12-802131-6 |
URL | http://elsev.spi-bpo.com/books/EComp |
Proceedings, refereed
Synthesis of attributed feature models from product descriptions
In Proceedings of the 19th International Conference on Software Product Line, SPLC'15, Nashville, USA, July 20-24., 2015.Status: Published
Synthesis of attributed feature models from product descriptions
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | Proceedings of the 19th International Conference on Software Product Line, SPLC'15, Nashville, USA, July 20-24. |
Pagination | 1-10 |
Date Published | 07/2015 |
ISBN Number | 978-1-4503-3613-0 |
Towards More Relational Feature Models
In ICSOFT-EA 2015 - Proceedings of the 10th International Conference on Software Engineering and Applications, Colmar, Alsace, France, 20-22 July. SciTePress, 2015.Status: Published
Towards More Relational Feature Models
Feature modeling is of paramount importance to capture variabilities and commonalities within a software product line. Nevertheless, current feature modeling notations are limited, representing only propositional formulae over attributed variables. This position paper advocates the extension of feature modeling formalisms with richer computational domains and relational operations. In particular, it proposes to extend feature modeling with finite and continuous domain variables, with first-order logic quantifiers, and with N-ary relations between features attributes, and with so-called global constraints. In order to extend the expressiveness while preserving automated analysis facilities, feature models could be semantically interpreted as first-order logic formulae (instead of propositional logic formulae), including global and continuous dependency between features. In simpler words, this paper emphasizes the importance of having more relational feature models and presents next generation applications.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2015 |
Conference Name | ICSOFT-EA 2015 - Proceedings of the 10th International Conference on Software Engineering and Applications, Colmar, Alsace, France, 20-22 July. |
Pagination | 381-386 |
Date Published | 07/2015 |
Publisher | SciTePress |
ISBN Number | 978-989-758-114-4 |
Technical reports
Synthesis of Attributed Feature Models From Product Descriptions: Foundations
INRIA, 2015.Status: Published
Synthesis of Attributed Feature Models From Product Descriptions: Foundations
Feature modeling is a widely used formalism to characterize a set of products (also called configurations). As a manual elaboration is a long and arduous task, numerous techniques have been proposed to reverse engineer feature models from various kinds of artefacts. But none of them synthesize feature attributes (or constraints over attributes) despite the practical relevance of attributes for documenting the different values across a range of products. In this report, we develop an algorithm for synthesizing attributed feature models given a set of product descriptions. We present sound, complete, and parametrizable techniques for computing all possible hierarchies, feature groups, placements of feature attributes, domain values, and constraints. We perform a complexity analysis w.r.t. number of features, attributes, configurations, and domain size. We also evaluate the scalability of our synthesis procedure using randomized configuration matrices. This report is a first step that aims to describe the foundations for synthesizing attributed feature models.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Technical reports |
Year of Publication | 2015 |
Number | RR-8680 |
Publisher | INRIA |
URL | https://hal.inria.fr/hal-01116663 |
Journal Article
A Combined Approach for Constraints Over Finite Domains and Arrays
journal (2014).Status: Submitted
A Combined Approach for Constraints Over Finite Domains and Arrays
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2014 |
Journal | journal |
Publisher | Springer Berlin Heidelberg |
Talks, contributed
Challenges in Constraint-Based Testing
In Selected talk at 'Verification meets CP' workshop 2014, Lyon, France. .: , 2014.Status: Published
Challenges in Constraint-Based Testing
Constraint-Based Testing emerged ten years ago as an important software testing topic. Many developments in symbolic execution, a technique used to symbolically explore the execution paths of a program under test, revealed the pressing needs for having at hand powerful constraint solving methods.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, contributed |
Year of Publication | 2014 |
Location of Talk | Selected talk at 'Verification meets CP' workshop 2014, Lyon, France |
Publisher | . |
Place Published | . |
Keywords | Workshop |
Symbolic Path-Oriented Test Data Generation for Floating-Point Programs
In Dagsthul Seminar (Symbolic Methods and Constraint Solving) , 2014.Status: Published
Symbolic Path-Oriented Test Data Generation for Floating-Point Programs
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, contributed |
Year of Publication | 2014 |
Location of Talk | Dagsthul Seminar (Symbolic Methods and Constraint Solving) |
Type of Talk | Contrinuted talk |
Towards an Effective Formally Certified Constraint Solver
In Selected talk at the 'Verification meets CP' 2014 workshop, Lyon, France. .: , 2014.Status: Published
Towards an Effective Formally Certified Constraint Solver
When used to verify safety-critical properties of a software system, the answers provided by a CP-based constraint solver must be formally certied through trusted and faithful methodologies. In such cases, one has often to formally verify that a constraint system is unsatisable, as it is required to prove that no reachable states can violate the property to be ensured. Unfortunately, proving that a constraint system is unsatisable is much more complex for a CP-solver than checking a possible solution.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, contributed |
Year of Publication | 2014 |
Location of Talk | Selected talk at the 'Verification meets CP' 2014 workshop, Lyon, France |
Publisher | . |
Place Published | . |
Keywords | Workshop |
Proceedings, refereed
FLOWER: Optimal Test Suite Reduction As a Network Maximum Flow
In Proceedings of Int. Symp. on Soft. Testing and Analysis (ISSTA'14), San José, CA, USA, Jul. 2014. New York, USA: ACM, 2014.Status: Published
FLOWER: Optimal Test Suite Reduction As a Network Maximum Flow
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of Int. Symp. on Soft. Testing and Analysis (ISSTA'14), San José, CA, USA, Jul. 2014 |
Pagination | 171-180 |
Date Published | 07/2014 |
Publisher | ACM |
Place Published | New York, USA |
ISBN Number | 978-1-4503-2645-2 |
Keywords | Conference |
DOI | 10.1145/2610384.2610416 |
Multi-Objective Test Prioritization in Software Product Line Testing: an Industrial Case Study
In Software Product Line Conference (SPLC14), Florence, Italy. New York, USA: ACM, 2014.Status: Published
Multi-Objective Test Prioritization in Software Product Line Testing: an Industrial Case Study
Test prioritization is crucial for testing products in a product line considering limited budget in terms of available time and resources. In general, it is not practically feasible to execute all the possible test cases and so, ordering test case execution permits test engineers to discover faults earlier in the testing process. An efficient prioritization of test cases for one or more products requires a clear consideration of the tradeoff among various costs (e.g., time, required resources) and effectiveness (e.g., feature coverage) objectives. As an integral part of the future Cisco's test scheduling system for validating video conferencing products, we introduce a search-based multi-objective test prioritization technique, considering multiple cost and effectiveness measures. In particular, our multi-objective optimization setup includes the minimization of execution cost (e.g., time), and the maximization of number of prioritized test cases, feature pairwise coverage and fault detection capability. Based on cost-effectiveness measures, a novel fitness function is defined for such test prioritization problem. The fitness function is empirically evaluated together with three commonly used search algorithms (e.g., (1+1) Evolutionary algorithm (EA)) and Random Search as a comparison baseline based on the Cisco's industrial case study and 500 artificial designed problems. The results show that (1+1) EA achieves the best performance for solving the test prioritization problem and it scales up to solve the problems of varying complexity.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Software Product Line Conference (SPLC14), Florence, Italy |
Publisher | ACM |
Place Published | New York, USA |
Keywords | Conference |
Random-Weighted Search-Based Multi-Objective Optimization Revisited
In International Symposium on Search-Based Software Engineering (SSBSE'14), Fortaleza, Brazil. Switzerland : Springer International Publishing, 2014.Status: Published
Random-Weighted Search-Based Multi-Objective Optimization Revisited
Weight-based multi-objective optimization requires assigning appro- priate weights using a weight strategy to each of the objectives such that an overall optimal solution can be obtained with a search algorithm. Choosing weights using an appropriate weight strategy has a huge impact on the obtained solutions and thus warrants the need to seek the best weight strategy. In this pa- per, we propose a new weight strategy called Uniformly Distributed Weights (UDW), which generates weights from uniform distribution, while satisfying a set of user-defined constraints among various cost and effectiveness measures. We compare UDW with two commonly used weight strategies, i.e., Fixed Weights (FW) and Randomly-Assigned Weights (RAW), based on five cost/effectiveness measures for an industrial problem of test minimization de- fined in the context of Video Conferencing System Product Line developed by Cisco Systems. We empirically evaluate the performance of UDW, FW, and RAW in conjunction with four search algorithms ((1+1) Evolutionary Algorithm (EA), Genetic Algorithm, Alternating Variable Method, and Random Search) using the industrial case study and 500 artificial problems of varying complexi- ty. Results show that UDW along with (1+1) EA achieves the best performance among the other combinations of weight strategies and algorithms.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | International Symposium on Search-Based Software Engineering (SSBSE'14), Fortaleza, Brazil |
Date Published | August |
Publisher | Springer International Publishing |
Place Published | Switzerland |
Keywords | Conference |
Testing Robotized Paint System Using Constraint Programming: an Industrial Case Study
In Proceedings of IFIP Int. Conf. on Testing Software and Systems (ICTSS'14), Madrid, Spain, Sep. 2014 - Application Track. Berlin, Heidelberg: LNCS, 2014.Status: Published
Testing Robotized Paint System Using Constraint Programming: an Industrial Case Study
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of IFIP Int. Conf. on Testing Software and Systems (ICTSS'14), Madrid, Spain, Sep. 2014 - Application Track |
Publisher | LNCS |
Place Published | Berlin, Heidelberg |
Keywords | Conference |
Using CP in Automatic Test Generation for ABB Robotics' Paint Control System
In Proceedings of Principles of Constraint Programming (CP'14), Lyon, France, Sep. 2014 - Best Application Track Paper. Zwitzerland: LNCS, 2014.Status: Published
Using CP in Automatic Test Generation for ABB Robotics' Paint Control System
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | Proceedings of Principles of Constraint Programming (CP'14), Lyon, France, Sep. 2014 - Best Application Track Paper |
Publisher | LNCS |
Place Published | Zwitzerland |
Keywords | Conference |
Worst-case Scheduling of Software Tasks – A Constraint Optimization Model to Support Performance Testing
In The 20th International Conference on Principles and Practice of Constraint Programming (CP 2014), 2014.Status: Published
Worst-case Scheduling of Software Tasks – A Constraint Optimization Model to Support Performance Testing
Real-Time Embedded Systems (RTES) in safety-critical domains, such as maritime and energy, must satisfy strict performance requirements to be deemed safe. Therefore, such systems have to be thoroughly tested to ensure their correct behavior even under the worst operating conditions. In this paper, we address the need of deriving worst case scenarios with respect to three common performance requirements, namely task deadlines, response time, and CPU usage. Specifically, we investigate whether this worst-case analysis can be effectively re-expressed as a Constrained Optimization Problem (COP) over the space of possible inputs to the system. Solving this problem means finding the sets of inputs that maximize the chance to violate performance requirements at runtime. Such inputs can in turn be used to test if the target RTES meets the expected performance even in the worst case. We develop an OPL model for IBM ILOG CP Optimizer that implements a task priority-based preemptive scheduling, and apply it to a case study from the maritime and energy domain. Our validation shows that (1) the input to our model can be provided with reasonable effort in an industrial setting, and (2) the COP effectively identifies test cases that maximize deadline misses, response time, and CPU usage.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2014 |
Conference Name | The 20th International Conference on Principles and Practice of Constraint Programming (CP 2014) |
DOI | 10.1007/978-3-319-10428-7_58 |
Edited books
Proceedings of the Software Engineering in Practice (SEIP) Track of ICSE'14, June 1-7, Hyderabad, India
IEEE, 2014.Status: Published
Proceedings of the Software Engineering in Practice (SEIP) Track of ICSE'14, June 1-7, Hyderabad, India
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Edited books |
Year of Publication | 2014 |
Publisher | IEEE |
Proceedings, refereed
Automated Product Line Methodologies to Support Model-Based Testing
In The ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS), online publication at CEUR. .: IEEE, 2013.Status: Published
Automated Product Line Methodologies to Support Model-Based Testing
Testing products in a cost-efficient way remains an attractive topic for Model-Based Testing (MBT) of product lines in both academia and industry, which can be addressed by employing systematic and automated approaches based on models (such as feature models and UML models). Cost-effective testing products can be divided into three main problems, i.e., test selection, test generation, and test minimization. Driven by the needs of our industrial problems for testing Video Conferencing Systems (VCSs) product line developed by Cisco, Norway, this paper presents Product Line Model-based Testing Methodologies (PL-MTM) to tackle the above-mentioned three problems for cost- effective testing a product in product line, which includes: 1) an systematic and automated test selection methodology; 2) an automated test minimization approach; and 3) an automated and systematic test generation methodology
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | The ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS), online publication at CEUR |
Publisher | IEEE |
Place Published | . |
Automated Test Case Selection Using Feature Model: an Industrial Case Study
In ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS), Best Paper Award. .: ACM/IEEE, 2013.Status: Published
Automated Test Case Selection Using Feature Model: an Industrial Case Study
Automated test case selection for a new product in a product line is challenging due to several reasons. First, the variability within the product line needs to be captured in a systematic way; second, the reusable test cases from the repository are required to be identified for testing a new product. The objective of such automated process is to reduce the overall effort for selection (e.g., selection time), while achieving an acceptable level of the coverage of testing functionalities. In this paper, we propose a systematic and automated methodology using a Feature Model for Testing (FM\_T) to capture commonalities and variabilities of a product line and a Component Family Model for Testing (CFM\_T) to capture the overall structure of test cases in the repository. With our methodology, a test engineer does not need to manually go through the repository to select a relevant set of test cases for a new product. Instead, a test engineer only needs to select a set of relevant features using FM\_T at a higher level of abstraction for a product and a set of relevant test cases will be selected automatically. We applied our methodology to a product line of video conferencing systems called Saturn developed by Cisco and the results show that our methodology can reduce the selection effort significantly. Moreover, we conducted a questionnaire-based study to solicit the views of test engineers who were involved in developing FM\_T and CFM\_T. The results show that test engineers are positive about adapting our method- ology and models (FM\_T and CFM\_T) in their current practice.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS), Best Paper Award |
Publisher | ACM/IEEE |
Place Published | . |
Dealing With Software Model Quality in Practice: Experience in a Research Project
In 13th International Conference on Quality Software (QSIC 2013). Najing: IEEE, 2013.Status: Published
Dealing With Software Model Quality in Practice: Experience in a Research Project
Although past research has resulted in different means to deal with software model quality, creation of adequate software models remains challenging. Any modelling effort must be carefully analysed and planned before it starts, and definition or adoption of modelling guidelines is usually necessary. In addition, the amount of publications addressing model quality in practice is low, and the knowledge about others' experience regarding model quality is limited. This paper reports on our experience in dealing with software model quality in the context of a project between industry and academia. Such a project corresponds to a large-scale research project in which modelling has been used both as part of the necessary work for executing the project and for creating project results. We present how we have dealt with model quality in requirements modelling and in conceptual model specification, as well as a set of lessons learned. The insights provided can help both researchers and practitioners when having to deal with software model quality.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | 13th International Conference on Quality Software (QSIC 2013) |
Publisher | IEEE |
Place Published | Najing |
Minimizing Test Suites in Software Product Lines Using Weight-Based Genetic Algorithms
In ACM Genetic and Evolutionary Computation Conference (GECCO). New York, NY, USA: ACM, 2013.Status: Published
Minimizing Test Suites in Software Product Lines Using Weight-Based Genetic Algorithms
Test minimization techniques aim at identifying and eliminating redundant test cases from test suites in order to reduce the total number of test cases to execute, thereby improving the efficiency of testing. In the context of software product line, we can save efforts and cost in the selection and minimization of test cases for testing a specific product by modeling the product line. However, minimizing the test suite for a product requires addressing two potential issues: 1) the reduced test suite may not cover all test requirements compared with the original suite; 2) the reduced test suite may have less fault revealing capability than the original suite. In this paper, we apply weight-based Genetic Algorithms (GAs) to minimize the test suite for testing a product, while preserving fault detection capability and testing coverage of the original test suite. The challenge behind concerns the definition of an appropriate fitness function, which is able to preserve the coverage of complex testing criteria (e.g., Combinatorial Interaction Testing criterion). Based on the defined fitness function, we have empirically evaluated three different weight- based GAs on an industrial case study provided by Cisco Systems, Inc. Norway. We also presented our results of applying the three weight-based GAs on five existing case studies from the literature. Based on these case studies, we conclude that among the three weight-based GAs, Random-Weighted GA (RWGA) achieved significantly better performance than the other ones.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | ACM Genetic and Evolutionary Computation Conference (GECCO) |
Publisher | ACM |
Place Published | New York, NY, USA |
Modelling Data Interaction Requirements: a Position Paper
In 2013 International Workshop on Model-Driven Requirements Engineering (MoDRE 2013). Rio de Janeiro : IEEE, 2013.Status: Published
Modelling Data Interaction Requirements: a Position Paper
Data-intensive information systems constitute the backbone of e-commerce and e-governance services running worldwide. Structured data is a central artefact in these information systems. Requirements for structure in data are typically modelled in a database schema. However, information system behaviour is often a function of interactions that cross-cut database features such as field values in different tables. For instance, consultants at the Norwegian Customs and Excise reveal that taxation rules are triggered due to data interactions between 10,000 items, 88 country groups, and 934 tax codes. There are about 12.9 trillion possible three-wise interactions of which only about 220,000 interactions are used in reality as customs rules. Therefore, we ask, how can we model data interaction requirements to further bound the input domain of an information system? In this position paper, we address this question by modelling data interaction requirements using classification tree models. We also present different applications of data interaction requirements in the development of information systems.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | 2013 International Workshop on Model-Driven Requirements Engineering (MoDRE 2013) |
Publisher | IEEE |
Place Published | Rio de Janeiro |
Practical Pairwise Testing for Software Product Lines
In Proceedings of the International Software Product Line Conference (SPLC). New York, NY, USA: ACM, 2013.Status: Published
Practical Pairwise Testing for Software Product Lines
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | Proceedings of the International Software Product Line Conference (SPLC) |
Pagination | 227-235 |
Publisher | ACM |
Place Published | New York, NY, USA |
Scenario Realizability With Constraint Optimization
In Proc. of. Fundamental Aspects of Software Engineering (FASE'2013). Berlin, Heidelberg: Springer, 2013.Status: Published
Scenario Realizability With Constraint Optimization
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | Proc. of. Fundamental Aspects of Software Engineering (FASE'2013) |
Date Published | March |
Publisher | Springer |
Place Published | Berlin, Heidelberg |
Keywords | Conference |
Scenario realizability with constraint optimization, In: Proc. Of Fundamental Aspects of Software Engineering
In Proc. Of Fundamental Aspects of Software Engineering (FASE 2013), . Springer, 2013.Status: Published
Scenario realizability with constraint optimization, In: Proc. Of Fundamental Aspects of Software Engineering
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | Proc. Of Fundamental Aspects of Software Engineering (FASE 2013), |
Publisher | Springer |
Stress Testing of Task Deadlines: A Constraint Programming Approach
In The 24th IEEE International Symposium on Software Reliability Engineering (ISSRE 2013), 2013.Status: Published
Stress Testing of Task Deadlines: A Constraint Programming Approach
Safety-critical Real Time Embedded Systems (RTESs) are usually subject to strict timing and performance requirements that must be satisfied for the system to be deemed safe. In this paper, we use effective search strategies whose goal is finding worst case scenarios with respect to deadline misses. Such scenarios can in turn be used to test the target RTES and ensure that it satisfies its timing requirements even under worst case conditions. Specifically, we develop an approach based on Constraint Programming (CP) to automate the generation of test cases that reveal, or are likely to, task deadline misses. We evaluate it through a comparison with a state-of-the-art approach based on Genetic Algorithms (GA). In particular, we compare CP and GA in five case studies for efficiency, effectiveness, and scalability. Our experimental results show that, on the largest and more complex case studies, CP performs significantly better than GA. Furthermore, CP offers some advantages over GA, such as it guarantees a complete search when there is sufficient time, and, being deterministic, it doesn't rely on parameters that potentially have a significant effect on the search and therefore need to be tuned. Hence, we conclude that our results are encouraging and suggest this is an advantageous approach for stress testing of RTESs with respect to timing constraints.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | The 24th IEEE International Symposium on Software Reliability Engineering (ISSRE 2013) |
DOI | 10.1109/ISSRE.2013.6698915 |
Symbolic Path-Oriented Test Data Generation for Floating-Point Programs
In Proc. of Int. Conf. on Soft. Testing, Validation and Verification (ICST'2013) - 10 pages. Luxembourg: IEEE, 2013.Status: Published
Symbolic Path-Oriented Test Data Generation for Floating-Point Programs
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | Proc. of Int. Conf. on Soft. Testing, Validation and Verification (ICST'2013) - 10 pages |
Date Published | March |
Publisher | IEEE |
Place Published | Luxembourg |
Test Case Prioritization for Continuous Regression Testing: an Industrial Case Study
In Proceedings of the International Conference on Software Maintenance (ICSM). Eindhoven: IEEE, 2013.Status: Published
Test Case Prioritization for Continuous Regression Testing: an Industrial Case Study
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | Proceedings of the International Conference on Software Maintenance (ICSM) |
Publisher | IEEE |
Place Published | Eindhoven |
Test Generation for Robotized Paint Systems Using Constraint Programming in a Continuous Integration Environment
In IEEE ICST 2013 (2 pages). Luxembourg: IEEE, 2013.Status: Published
Test Generation for Robotized Paint Systems Using Constraint Programming in a Continuous Integration Environment
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | IEEE ICST 2013 (2 pages) |
Pagination | 489-290 |
Date Published | April |
Publisher | IEEE |
Place Published | Luxembourg |
Testing a Data-Intensive System With Generated Data Interactions: the Norwegian Customs and Excise Case Study
In Proceedings of 25th International Conference on Advanced Information Systems Engineering (CAiSE'13),June 17-21 2013, Valencia, Spain. Springer, 2013.Status: Published
Testing a Data-Intensive System With Generated Data Interactions: the Norwegian Customs and Excise Case Study
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2013 |
Conference Name | Proceedings of 25th International Conference on Advanced Information Systems Engineering (CAiSE'13),June 17-21 2013, Valencia, Spain |
Date Published | June |
Publisher | Springer |
Technical reports
Automated Search-Based Test Suite Minimization in Product Lines: An Empirical Study
Simula Research Laboratory, 2013.Status: Published
Automated Search-Based Test Suite Minimization in Product Lines: An Empirical Study
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Technical reports |
Year of Publication | 2013 |
Publisher | Simula Research Laboratory |
Test Selection Based on Data Interactions in Data-Intensive Systems
Simula Research Laboratory, 2013.Status: Published
Test Selection Based on Data Interactions in Data-Intensive Systems
Testing data-intensive systems is paramount to increase our reliance on information in e-governance, scientific/ medical research, and social networks. Common practice to test these systems is by using a live production database. This testing approach is space and time inefficient and lacks clarity about what test cases or scenarios are covered. In this paper, we leverage classification tree modelling to specify desired test cases as data interactions between a set of fields across multiple tables of an existing database. Our methodology and tool, DEPICT, uses test case specifications in classification tree models to (a) automatically derive a spanning tree representing a relationship between any set of fields for any given database schema (b) generates queries to create an efficient inner join between related tables in the spanning tree (c) extract records from various tables that satisfy data interactions in the classification tree model (d) discovers holes or unsatisfied test cases in the test databases. We perform experiments to show that our approach is fast and scalable to extract test databases. Our experiments are based on selecting test databases from 8000 declarations for 60,000 items from the Norwegian Customs and Excise information system TVINN.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Technical reports |
Year of Publication | 2013 |
Number | 2013-03 |
Date Published | 10/2013 |
Publisher | Simula Research Laboratory |
Keywords | Conference |
Journal Article
FocalTest: a Constraint Programming Approach for Property-Based Testing
Communications in Computer and Information Science (2013): 140-155.Status: Published
FocalTest: a Constraint Programming Approach for Property-Based Testing
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2013 |
Journal | Communications in Computer and Information Science |
Number | 170 |
Pagination | 140-155 |
Publisher | Springer Berlin Heidelberg |
Notes | Revised selected papers from ICSOFT 2010 - Edited by Cordeiro Virvou Shishkov (Eds), SPRINGER |
Edited books
Proceedings of 13th International Conference on Quality Software 2013 (QSIC 2013), Nanjing, China, Jul. 2013
Nanjing, China: Conference Publishing Services, 2013.Status: Published
Proceedings of 13th International Conference on Quality Software 2013 (QSIC 2013), Nanjing, China, Jul. 2013
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Edited books |
Year of Publication | 2013 |
Date Published | July |
Publisher | Conference Publishing Services |
Place Published | Nanjing, China |
Keywords | Conference |
Proceedings, refereed
A Certified Constraint Solver Over Finite Domains
In Proceedings of Formal Methods (FM'12), Paris, Aug. 2012. Vol. 7436. Berlin Heidelberg: Springer Berlin Heidelberg, 2012.Status: Published
A Certified Constraint Solver Over Finite Domains
Constraint programs such as those written in modern Constraint Programming languages and platforms aim at solving problems coming from optimization, scheduling, planning, etc. Recently CP programs have been used in business-critical or safety-critical areas as well, e.g., e-Commerce, air-traffic control applications, or software verification. This implies a more skeptical regard on the implementation of constraint solvers, especially when the result is that a constraint problem has no solution, i.e., unsatisfiability. For example, in software model checking, using an unsafe constraint solver may result in a dramatic wrong answer saying that a safety property is satisfied while there exist counterexamples. In this paper, we present a Coq formalization of a constraint filtering algorithm - AC3 and one of its variant AC2001 - and a simple labeling procedure. The proof of their soundness and completeness has been completed using Coq. As a result, a formally verified constraint solver written in OCaml has been automatically extracted from the Coq specification of the filtering and labeling algorithms. The solver, yet not as efficient as specialized existing (unsafe) implementations, can be used to formally certify that a constraint system is unsatisfiable.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | Proceedings of Formal Methods (FM'12), Paris, Aug. 2012 |
Volume | 7436 |
Pagination | 116-131 |
Date Published | August |
Publisher | Springer Berlin Heidelberg |
Place Published | Berlin Heidelberg |
Keywords | Conference |
DOI | 10.1007/978-3-642-32759-9\_12 |
A First Step in the Design of a Formally Verified Constraint-Based Testing Tool: FocalTest
In Proceedings of the Tests And Proofs Conference (TAP 2012), Prague, Czech Republic. Vol. 7305. Berlin Heidelberg: Springer Berlin Heidelberg, 2012.Status: Published
A First Step in the Design of a Formally Verified Constraint-Based Testing Tool: FocalTest
Constraint-based test data generators rely on SMT or constraint solvers to automatically generate test data (e.g., Pex, Sage, Gatel, Path\-Crawler, Euclide, etc.). However, for some test data generation requests corresponding to particular testing objectives, these tools may fail to deliver the expected test data because they rather focus on efficiency rather than correction and completeness. In our work, we adopt an opposite view in the development of FocalTest, a test data generation tool for Focalize programs. The goal of the tool is to generate an MC/DC-compliant set of test data over the precondition of user-defined program properties. The development of such a correct-by-construction test data generator requires 1) to provide a formally verified translation of Focalize programs and properties into constraint systems; 2) to introduce a formally verified constraint solver able to solve those constraint systems. This paper is concerned with the first step only where we formally demonstrate with Coq the correctness of the translation of an intermediate functional language into a constraint system. This objective requires to formally define the operational semantics of the source language that features the manipulation of concrete data types via pattern-matching and function calls, constructions that are mirrored in the constraint language. Although such a semantics-oriented formalization is only a first step of a larger goal which is to provide a formally verified constraint-based testing tool, we argue that it is an important contribution to the building of more robust software testing tools.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | Proceedings of the Tests And Proofs Conference (TAP 2012), Prague, Czech Republic |
Volume | 7305 |
Pagination | 35-50 |
Publisher | Springer Berlin Heidelberg |
Place Published | Berlin Heidelberg |
Keywords | Conference |
DOI | 10.1007/978-3-642-30473-6\_5 |
Automatic Selection of Test Execution Plans from a Video Conferencing System Product Line
In Proceedings of the VARiability for You Workshop: Variability Modeling Made Useful for Everyone. ACM, 2012.Status: Published
Automatic Selection of Test Execution Plans from a Video Conferencing System Product Line
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | Proceedings of the VARiability for You Workshop: Variability Modeling Made Useful for Everyone |
Pagination | 32 - 37 |
Publisher | ACM |
ISBN Number | 978-1-4503-1809-9 |
DOI | 10.1145/2425415.2425422 |
fdcc: a Combined Approach for Solving Constraints Over Finite Domains and Arrays
In Proceedings of CPAIOR 2012 (Constraint Programming-Artificial Intelligence-Operations Research), Nantes, France. Berlin Heidelberg: Springer Berlin Heidelberg, 2012.Status: Published
fdcc: a Combined Approach for Solving Constraints Over Finite Domains and Arrays
Arrays are ubiquitous in the context of software verification. However, effective reasoning over arrays is still rare in CP, as local reasoning is dramatically ill-conditioned for constraints over arrays. In this paper, we propose an approach combining both global symbolic reasoning and local filtering in order to solve constraint systems involving arrays (with accesses, updates and size constraints) and finite-domain constraints over their elements and indexes. Our approach, named FDCC, is based on a combination of a congruence closure algorithm for the standard theory of arrays and a CP solver over finite domains. The tricky part of the work lies in the bi-directional communication mechanism between both solvers. We identify the significant information to share, and design ways to master the communication overhead. Experiments on random instances show that FDCC solves more formulas than any portfolio combination of the two solvers taken in isolation, while overhead is kept reasonable.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | Proceedings of CPAIOR 2012 (Constraint Programming-Artificial Intelligence-Operations Research), Nantes, France |
Pagination | 17-33 |
Publisher | Springer Berlin Heidelberg |
Place Published | Berlin Heidelberg |
Keywords | Conference |
DOI | 10.1007/978-3-642-29828-8\_2 |
Managing Execution Environment Variability During Software Testing: an Industrial Experience
In Proceedings of the 24th IFIP Int. Conference on Testing Software and Systems (ICTSS'12). .: LNCS, 2012.Status: Published
Managing Execution Environment Variability During Software Testing: an Industrial Experience
Nowadays, telecom software applications are expected to run on a tremendous variety of execution environments. For example, network connection software must deliver the same functionalities on distinct physical platforms, which themselves run several distinct operating systems, with various applications and physical devices. Testing those applications is challenging as it is simply impossible to consider every possible environment configuration. This paper reports on an industrial case study called BIEW (Business and Internet EveryWhere) where the combinatorial explosion of environment configurations has been tackled with a dedicated and original methodology devised by KEREVAL, a french SME focusing on software testing services. The proposed solution samples a subset of configurations to be tested, based on environment modelling, requirement analysis and systematic traceability. From the experience on this case study, we outline the challenges to develop means to select relevant environment configurations from variability modelling and requirement analysis in the testing processes of telecom software.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | Proceedings of the 24th IFIP Int. Conference on Testing Software and Systems (ICTSS'12) |
Date Published | November |
Publisher | LNCS |
Place Published | . |
Minimum Pairwise Coverage Using Constraint Programming Techniques
In 2012 IEEE Fifth International Conference. IEEE, 2012.Status: Published
Minimum Pairwise Coverage Using Constraint Programming Techniques
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | 2012 IEEE Fifth International Conference |
Pagination | 773 - 774 |
Date Published | 04/2012 |
Publisher | IEEE |
ISBN Number | 978-1-4577-1906-6 |
DOI | 10.1109/ICST.2012.174 |
Model-Based Automated and Guided Configuration of Embedded Software Systems
In ECMFA 2012. Vol. 7349. Berlin Heidelberg: Springer Berlin Heidelberg, 2012.Status: Published
Model-Based Automated and Guided Configuration of Embedded Software Systems
Configuring Integrated Control Systems (ICSs) is largely manual, time-consuming and error-prone. In this paper, we propose a model-based configuration approach that interactively guides engineers to configure software embedded in ICSs. Our approach verifies engineers' decisions at each configuration iteration, and further, automates some of the decisions. We use a constraint solver, SICStus Prolog, to automatically infer configuration decisions and to ensure the consistency of configuration data. We evaluated our approach by applying it to a real subsea oil production system. Specifically, we rebuilt a number of existing verified product configurations of our industry partner. Our experience shows that our approach successfully enforces consistency of configurations, can automatically infer up to 50% of the configuration decisions, and reduces the complexity of making configuration decisions.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | ECMFA 2012 |
Volume | 7349 |
Pagination | 226-243 |
Publisher | Springer Berlin Heidelberg |
Place Published | Berlin Heidelberg |
Keywords | Conference |
DOI | 10.1007/978-3-642-31491-9\_18 |
MuTIL: Mutation-Based Statistical Test Inputs Generation for Automatic Fault Localization
In Proceedings of the Sixth International Conference on Software Security and Reliability (SERE-12), Washington, USA. .: , 2012.Status: Published
MuTIL: Mutation-Based Statistical Test Inputs Generation for Automatic Fault Localization
Automatic Fault Localization (AFL) is a process to locate faults automatically in software programs. Essentially, an AFL method takes as input a set of test cases including failed test cases, and ranks the statements of a program from the most likely to the least likely to contain a fault. As a result, the efficiency of an AFL method depends on the "quality" of the test cases used to rank statements. More specifically, in order to improve the accuracy of their ranking within test budget constraints, we have to ensure that program statements are executed by a reasonably large number of test cases which provide a coverage as uniform as possible of the input domain. This paper proposes μTIL, a new statistical test inputs generation method dedicated to AFL, based on constraint solving and mutation testing. Using mutants where the locations of injected faults are known, μTIL is able to significantly reduce the length of an AFL test suite while retaining its accuracy (i.e., the code size to examine before spotting the fault). In order to address the motivations stated above, the statistical generator objectives are two-fold: 1) each feasible path of the program is activated with the same probability; 2) the subdomain associated to each feasible path is uniformly covered. Using several widely used ranking techniques (i.e., Tarantula, Jaccard, Ochiai), we show on a small but realistic program that a proof-of-concept implementation of μTIL can generate test sets with significantly better fault localization accuracy than both random testing and adaptive random testing. We also show on the same program that using mutation testing enables a 75% length reduction of the AFL test suite without decrease in accuracy.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | Proceedings of the Sixth International Conference on Software Security and Reliability (SERE-12), Washington, USA |
Date Published | 06/2012 |
Publisher | . |
Place Published | . |
Keywords | Conference |
Testing Deadline Misses for Real-Time Systems Using Constraint Optimization Techniques
In The 4th Workshop on Constraints in Software Testing, Verification, and Analysis (CSTVA 2012), 2012.Status: Published
Testing Deadline Misses for Real-Time Systems Using Constraint Optimization Techniques
Safety-critical real-time applications are typically subject to stringent timing constraints which are dictated by the surrounding physical environments. Specifically, tasks in these applications need to finish their execution before given deadlines, otherwise the system is deemed unsafe. It is therefore important to test real-time systems for deadline misses. In this paper, we present a strategy for testing real-time applications that aims at finding test scenarios in which deadline misses become more likely. We identify such test scenarios by searching the possible ways that a set of real-time tasks can be executed according to the scheduling policy of the operating system on which they are running. We formulate this search problem using a constraint optimization model that includes (1) a set of constraints capturing how a given set of tasks with real-time constraints are executed according to a particular scheduling policy, and (2) a cost function that estimates how likely the given tasks are to miss their deadlines. We implement our constraint optimization model in ILOG, apply our model to several examples, and report on the performance results.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2012 |
Conference Name | The 4th Workshop on Constraints in Software Testing, Verification, and Analysis (CSTVA 2012) |
DOI | 10.1109/ICST.2012.171 |
Journal Article
A CP Framework for Testing CP
Constraints 17 (2012): 123-147.Status: Published
A CP Framework for Testing CP
The success of several constraint-based modeling languages such as OPL, ZINC or COMET, appeals for better software engineering practices, particularly in the testing phase. This paper introduces a testing framework enabling automated test case generation for constraint programming. We propose a general framework of constraint program development which supposes that a first declarative and simple constraint model is available from the problem specifications analysis. Then, this model is refined using classical techniques such as constraint reformulation, surrogate, redundant, implied, global constraint and symmetry-breaking to form an improved constraint model that must be thoroughly tested before being used to address real-sized problems. We think that most of the faults are introduced in this refinement step and propose a process which takes the first declarative model as an oracle for detecting non-conformities and derive practical test purposes from this process. We implemented this approach in a new tool called {\sf CPTEST} that was used to automatically detect non-conformities on classical benchmark programs, such as the Golomb rulers, n-queens, social golfer and the car-sequencing problems.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2012 |
Journal | Constraints |
Volume | 17 |
Number | 2 |
Pagination | 123-147 |
Date Published | February |
Publisher | Springer US |
Tcas Software Verification Using Constraint Programming
The Knowledge Engineering Review 27 (2012).Status: Published
Tcas Software Verification Using Constraint Programming
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Journal Article |
Year of Publication | 2012 |
Journal | The Knowledge Engineering Review |
Volume | 27 |
Number | 3 |
Date Published | September |
Publisher | Cambridge University Press New York |
Technical reports
Automated Selection of Test Cases using Feature Model for Product Lines: An Industrial Case Study
Simula Research Laboratory, 2012.Status: Published
Automated Selection of Test Cases using Feature Model for Product Lines: An Industrial Case Study
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Technical reports |
Year of Publication | 2012 |
Number | 2012-20 |
Publisher | Simula Research Laboratory |
Notes | This technical report is an early version of a published conference paper. The published version can be found here: |
Automated Test Case Selection using Feature Model: an Industrial Case Study
Simula Research Laboratory, 2012.Status: Published
Automated Test Case Selection using Feature Model: an Industrial Case Study
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Technical reports |
Year of Publication | 2012 |
Number | 2012-28 |
Publisher | Simula Research Laboratory |
Notes | This technical report is an early version of a published conference paper. The published version can be found here: |
Automatic Derivation of Test Execution Plans From a Video Conferencing System Product Line Modelling
Simula Research Laboratory, 2012.Status: Published
Automatic Derivation of Test Execution Plans From a Video Conferencing System Product Line Modelling
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Technical reports |
Year of Publication | 2012 |
Number | 2012-02 |
Publisher | Simula Research Laboratory |
Notes | This technical report is an early version of a published poster. |
Guided Interactive Configuration of Embedded Software Systems Using Constraint Satisfaction Over Finite Domains
Simula Research Laboratory, 2012.Status: Published
Guided Interactive Configuration of Embedded Software Systems Using Constraint Satisfaction Over Finite Domains
Modern society is increasingly dependent on highly-configurable software systems, in particular, architecturally configurable software such as that embedded in Integrated Control Systems (ICSs). Configuring ICSs is expensive, time-consuming and error-prone. This is due, in large part, to the fact that the hardware and software configuration processes are, typically, rather isolated from one another, resulting in many configuration errors to be detected only after the integration of software and hardware. In this work, we propose a model-based configuration approach that allows us to configure software in a stepwise manner, to automate some configuration decisions, and to iteratively validate software and hardware configuration decisions. Our approach has two major steps. In the first step (modeling), a generic model describing an ICS family is built. In the second step (configuration), we interactively guide a user to derive a particular product specification complying with the generic model of its ICS family. We use a constraint solver, SICStus Prolog, to evaluate user decisions at each round, to automatically infer configuration decisions and to ensure that software and hardware configurations are consistent. We evaluated our approach by applying it to a real subsea production system. Specifically, we rebuilt a number of existing verified product configurations of our industry partner. Our experience shows that our approach successfully enforces consistency of configurations, can automatically infer up to 50% of the configuration decisions, and reduces the complexity of making configuration decisions by guiding the user during the configuration process.
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Technical reports |
Year of Publication | 2012 |
Number | 2012-05 |
Publisher | Simula Research Laboratory |
Keywords | Workshop |
Minimizing Test Suites in Software Product Lines Using Weight-Based Genetic Algorithms
Simula Research Laboratory, 2012.Status: Published
Minimizing Test Suites in Software Product Lines Using Weight-Based Genetic Algorithms
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Technical reports |
Year of Publication | 2012 |
Number | 2012-25 |
Publisher | Simula Research Laboratory |
Notes | This technical report is an early version of a published conference paper. The published paper can be found here: |
Research report on test configuration generation
Simula Research Laboratory, 2012.Status: Published
Research report on test configuration generation
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Technical reports |
Year of Publication | 2012 |
Publisher | Simula Research Laboratory |
Poster
Automatic Derivation of Test Execution Plans From a Video Conferencing System Product Line Modelling
, 2012.Status: Published
Automatic Derivation of Test Execution Plans From a Video Conferencing System Product Line Modelling
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Poster |
Year of Publication | 2012 |
Place Published | . |
Talks, invited
Constraint-Based Reacheability
In INFINITY Workshop, co-located with FM 2012, Paris. .: , 2012.Status: Published
Constraint-Based Reacheability
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Talks, invited |
Year of Publication | 2012 |
Location of Talk | INFINITY Workshop, co-located with FM 2012, Paris |
Publisher | . |
Place Published | . |
Notes | INFINITY Workshop, co-located with FM 2012, Paris |
Proceedings, refereed
Filtering by ULP Maximum
In Proceedings of the 23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI'2011), Nov. 7-9, 2011, Boca Raton, Florida, USA. IEEE, 2011.Status: Published
Filtering by ULP Maximum
Constraint solving over floating-point numbers is an emerging topic that found interesting applications in software analysis and testing. Even for IEEE-754 compliant programs, correct reasoning over floating-point computations is challenging and requires dedicated constraint solving approaches to be developed. Recent advances indicate that numerical properties of floating-point numbers can be used to efficiently prune the search space. In this paper, we reformulate the Marre and Michel property over floating-point addition/subtraction constraint to ease its implementation in real-world floating-point constraint solvers. We also generalize the property to the case of multiplication/division in order to benefit from its improvements in more cases.
Afilliation | Software Engineering, Software Engineering |
Publication Type | Proceedings, refereed |
Year of Publication | 2011 |
Conference Name | Proceedings of the 23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI'2011), Nov. 7-9, 2011, Boca Raton, Florida, USA |
Date Published | 11/2011 |
Publisher | IEEE |
Filtering by ULP maximum, In: Proceedings of the 23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI'2011),
In 23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI'2011). IEEE, 2011.Status: Published
Filtering by ULP maximum, In: Proceedings of the 23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI'2011),
Afilliation | Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2011 |
Conference Name | 23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI'2011) |
Publisher | IEEE |
PACOGEN: Automatic Generation of Pairwise Test Configurations From Feature Models
In Proceedings of the IEEE 22nd International Symposium on Software Reliability Engineering (ISSRE 2011). Hiroshima: IEEE Computer Society, 2011.Status: Published
PACOGEN: Automatic Generation of Pairwise Test Configurations From Feature Models
Feature models are commonly used to specify variability in software product lines. Several tools support feature models for variability management at different steps in the development process. However, tool support for test configuration generation is currently limited. This test generation task consists in systematically selecting a set of configurations that represent a relevant sample of the variability space and that can be used to test the product line. In this paper we propose PACOGEN to analyze feature models and automatically generate a set of configurations that cover all pairwise interactions between features. PACOGEN relies on constraint programming to generate configurations that satisfy all constraints imposed by the feature model and to minimize the set of the tests configurations. This work also proposes an extensive experiment, based on the state-of-the art SPLOT feature models repository, showing that PACOGEN scales over variability spaces with millions of configurations and covers pairwise with less configurations than other available tools.
Afilliation | Software Engineering, Software Engineering, Software Engineering |
Project(s) | The Certus Centre (SFI) |
Publication Type | Proceedings, refereed |
Year of Publication | 2011 |
Conference Name | Proceedings of the IEEE 22nd International Symposium on Software Reliability Engineering (ISSRE 2011) |
Date Published | November |
Publisher | IEEE Computer Society |
Place Published | Hiroshima |
ISBN Number | 978-0-7695-4568-4 |