Projects
MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System
The innovation planned in this project is an add-on to the digitization project currently being undertaken by the Cancer Registry of Norway (CR). The project started in 2009 and aims to transform the current paper-based/manual system into an ICT-based Automated Cancer Registry System (ACRS). The planned innovation project aims to develop systematic, automated and cost-effective model-based approaches for ensuring the quality of the evolving ACRS system and therefore significantly improving the efficiency of the patient history registration process. This will positively affect all its end users, including researchers, patients, doctors, and government officials.
Funding source:
Regionale forskningsfond
All partners:
- Simula Research Laboratory
- Cancer Registry of Norway
Project leaders:
- Simula: Tao Yue (PI) and Shaukat Ali (Co-PI),
- Cancer Registry: Jan F. Nygård
Publications for MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System
Journal Article
Automated Refactoring of OCL Constraints with Search
IEEE Transactions on Software Engineering (TSE) (2017).Status: Published
Automated Refactoring of OCL Constraints with Search
Object Constraint Language (OCL) constraints are typically used to provide precise semantics to models developed with the Unified Modeling Language (UML). When OCL constraints evolve regularly, it is essential that they are easy to understand and maintain. For instance, in cancer registries, to ensure the quality of cancer data, more than one thousand medical rules are defined and evolve regularly. Such rules can be specified with OCL. It is, therefore, important to ensure the understandability and maintainability of medical rules specified with OCL. To tackle such a challenge, we propose an automated search-based OCL constraint refactoring approach (SBORA) by defining and applying four semantics-preserving refactoring operators (i.e., Context Change, Swap, Split and Merge) and three OCL quality metrics (Complexity, Coupling, and Cohesion) to measure the understandability and maintainability of OCL constraints. We evaluate SBORA along with six commonly used multi-objective search algorithms (e.g., Indicator-Based Evolutionary Algorithm (IBEA)) by employing four case studies from different domains: healthcare (i.e., cancer registry system from Cancer Registry of Norway (CRN)), Oil&Gas (i.e., subsea production systems), warehouse (i.e., handling systems), and an open source case study named SEPA. Results show: 1) IBEA achieves the best performance among all the search algorithms and 2) the refactoring approach along with IBEA can manage to reduce on average 29.25% Complexity and 39% Coupling and improve 47.75% Cohesion, as compared to the original OCL constraint set from CRN. Furthermore, we conducted a controlled experiment with 96 subjects and results show that the understandability and maintainability of the original constraint set can be improved significantly from the perspectives of the 96 participants of the controlled experiment.
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | IEEE Transactions on Software Engineering (TSE) |
| Publisher | IEEE |
| URL | http://ieeexplore.ieee.org/document/8114267/ |
| DOI | 10.1109/TSE.2017.2774829 |
IOCL: An Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
Science of Computer Programming (SCP) 149 (2017): 3-8.Status: Published
IOCL: An Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
The Object Constraint Language (OCL) is commonly used for specifying additional constraints on models, in addition, to the ones enforced by the semantics of the models. However, a lot of practitioners and even researchers are reluctant in using OCL to some extent due to the lack of sufficient familiarity with OCL. To facilitate practitioners and researchers in specifying OCL constraints, we designed and developed a web-based tool called interactive OCL (iOCL) for interactively specifying constraints on a given model. The core idea behind iOCL is to present and display only relevant details (e.g., operations) of OCL to users at a given step of constraint specification process, in addition to helping modelers with its syntax. We evaluated iOCL using a real-world case study from Cancer Registry of Norway and the results showed that iOCL can significantly reduce the time required to specify OCL constraints and decrease the possibility of making syntactic errors during the specification process. Thus, we conclude that iOCL can facilitate the process of OCL constraint specification.
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Science of Computer Programming (SCP) |
| Volume | 149 |
| Pagination | 3-8 |
| Date Published | 08/2017 |
| Publisher | Elsevier |
Search and similarity based selection of use case scenarios: An empirical study
Empirical Software Engineering (2017): 1-78.Status: Published
Search and similarity based selection of use case scenarios: An empirical study
Use case modeling is a well-known requirements specification method and has been widely applied in practice. Use case scenarios of use case models are input elements for requirements inspection and analysis, requirements-based testing, and other downstream activities. It is, however, a practical challenge to inspect all use case scenarios that can be obtained from any non-trivial use case model, as such an inspection activity is often performed manually by domain experts. Therefore, it is needed to propose an automated solution for selecting a subset of use case scenarios with the ultimate aim of enabling cost-effective requirements (use case) inspection, analysis, and other relevant activities. Our solution is built on a natural language based, restricted use case modeling methodology (named as RUCM), in the sense that requirements specifications are specified as RUCM use case models. Use case scenarios can be automatically derived from RUCM use case models with the already established Zen-RUCM framework. In this paper, we propose a search-based and similarity-based approach called S3RCUM, through an empirical study, to select most diverse use case scenarios to enable cost-effective use case inspections. The empirical study was designed to evaluate the performance of three search algorithms together with eight similarity functions, through one real-world case study and six case studies from literature. Results show that (1+1) Evolutionary Algorithm together with Needleman-Wunsch similarity function significantly outperformed the other 31 combinations of the search algorithms and similarity functions. The combination managed to select 50% of all the generated RUCM use case scenarios for all the case studies to detect all the seeded defects.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems , U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Empirical Software Engineering |
| Pagination | 1-78 |
| Date Published | 04/2017 |
| Publisher | Springer |
Master's thesis
A Large-Scale OCL Constraint Repository And Comprehensive Analysis For Supporting Automated Cancer Registry System
In The Department of Informatics, University of Oslo, 2017.Status: Published
A Large-Scale OCL Constraint Repository And Comprehensive Analysis For Supporting Automated Cancer Registry System
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Master's thesis |
| Year of Publication | 2017 |
| Degree awarding institution | The Department of Informatics, University of Oslo |
A Rule-Based Framework for Supporting Automated Change Impact Analysis in the Cancer Registry of Norway
In The Department of Informatics, University of Oslo, 2017.Status: Published
A Rule-Based Framework for Supporting Automated Change Impact Analysis in the Cancer Registry of Norway
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Master's thesis |
| Year of Publication | 2017 |
| Degree awarding institution | The Department of Informatics, University of Oslo |
Proceedings, refereed
RCIA: Automated Change Impact Analysis to Facilitate a Practical Cancer Registry System
In The International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2017.Status: Published
RCIA: Automated Change Impact Analysis to Facilitate a Practical Cancer Registry System
The Cancer Registry of Norway (CRN) employs a cancer registry system to collect cancer patient data (e.g., diagnosis and treatments) from various medical entities (e.g., clinic hospitals). The collected data are then checked for validity (i.e., validation) and assembled as cancer cases (i.e., aggregation) based on more than 1000 cancer coding rules in the system. However, it is frequent in practice that the collected cancer data changes due to various reasons (e.g., different treatments) and the cancer coding rules can also change/evolve due to new medical knowledge. Thus, such a cancer registry system requires an efficient means to automatically analyze these changes and provide consequent impacts to medical experts for further actions. This paper proposes an automated Rule-based Change Impact Analysis (CIA) approach named RCIA that includes: 1) a change classification to capture the potential changes that can occur at CRN; 2) in total 80 change impact analysis rules including 50 dependency rules and 30 impact rules; and 3) an efficient algorithm to analyze changes and produce consequent impacts. We evaluate RCIA via a case study with 12 real change sets from CRN and a conducted interview. The results showed that RCIA managed to produce 100% actual change impacts and the medical expert at CRN is quite positive to apply RCIA to facilitate their cancer registry system. We also shared a set of lessons learned based on the collaboration with CRN.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | The International Conference on Software Maintenance and Evolution (ICSME) |
| Pagination | 603-612 |
| Publisher | IEEE |
Talks, contributed
RCIA: Automated Change Impact Analysis to Facilitate a Practical Cancer Registry System
In The International Conference on Software Maintenance and Evolution (ICSME), Shanghai, China, 2017.Status: Published
RCIA: Automated Change Impact Analysis to Facilitate a Practical Cancer Registry System
The Cancer Registry of Norway (CRN) employs a cancer registry system to collect cancer patient data (e.g., diagnosis and treatments) from various medical entities (e.g., clinic hospitals). The collected data are then checked for validity (i.e., validation) and assembled as cancer cases (i.e., aggregation) based on more than 1000 cancer coding rules in the system. However, it is frequent in practice that the collected cancer data changes due to various reasons (e.g., different treatments) and the cancer coding rules can also change/evolve due to new medical knowledge. Thus, such a cancer registry system requires an efficient means to automatically analyze these changes and provide consequent impacts to medical experts for further actions. This paper proposes an automated Rule-based Change Impact Analysis (CIA) approach named RCIA that includes: 1) a change classification to capture the potential changes that can occur at CRN; 2) in total 80 change impact analysis rules including 50 dependency rules and 30 impact rules; and 3) an efficient algorithm to analyze changes and produce consequent impacts. We evaluate RCIA via a case study with 12 real change sets from CRN and a conducted interview. The results showed that RCIA managed to produce 100% actual change impacts and the medical expert at CRN is quite positive to apply RCIA to facilitate their cancer registry system. We also shared a set of lessons learned based on the collaboration with CRN.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Talks, contributed |
| Year of Publication | 2017 |
| Location of Talk | The International Conference on Software Maintenance and Evolution (ICSME), Shanghai, China |
Technical reports
A Pilot Experiment to Assess Interactive OCL Specification in a Real Setting
Simula Research Laboratory, 2017.Status: Published
A Pilot Experiment to Assess Interactive OCL Specification in a Real Setting
The Object Constraint Language (OCL) is a formal, declarative, and side-effect free language, standardized by the Object Management Group, for specifying constraints or queries on models specified in the Unified Modeling Language (UML). OCL was designed with the aim to bridge the gap between natural language and traditional formal languages requiring a strong mathematical background to understand and apply. OCL, along with UML, have been applied in practice for various purposes such as facilitating automated model-based testing. In most of such contexts of OCL, engineers with software engineering backgrounds specify OCL constraints. However, it is still a challenge for constraint authors (e.g., medical coders) who have no such background to apply OCL for other purposes (e.g., specifying medical rules). In this direction, in our previous work, we proposed a user-interactive specification framework, named iOCL, for facilitating OCL constraint specification and validation. The aim was to ease its adoption in practice in a wider application scope. In this paper, we present a pilot experiment that was conducted to assess the practical applicability of iOCL in Cancer Registry of Norway with real users of iOCL in terms of specifying medical cancer coding rules with iOCL. Results of the pilot experiment showed that, with iOCL, time to specify OCL constraints can be significantly reduced as compared to directly specifying OCL constraints without the tool support. In addition, participants of the experiment found that iOCL is easy to use.
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Technical reports |
| Year of Publication | 2017 |
| Publisher | Simula Research Laboratory |
Proceedings, refereed
A Model-Based Approach with Tool Support to Facilitate the Cancer Registration Process in Cancer Registry of Norway
In European Telemedicine Conference (ETC), 2016.Status: Published
A Model-Based Approach with Tool Support to Facilitate the Cancer Registration Process in Cancer Registry of Norway
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | European Telemedicine Conference (ETC) |
iOCL: A Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
In Tool Demonstrations Track, ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MODELS), 2016.Status: Published
iOCL: A Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
The Object Constraint Language (OCL) is frequently used to specify additional constraints on models, in addition, to the ones enforced by semantics of the models. It is a well- known fact that due to the lack of familiarity with OCL, practitioners and even researcher to some extent are reluctant in using OCL. To help practitioners and researchers in writing OCL constraints for their specific problem at hand, we developed a tool called interactive OCL (iOCL) for interactively specifying constraints on a given model. The basic philosophy behind the tool is to present only those details (e.g., operations) of OCL to modelers that are valid at a given step of constraint specification process, in addition to helping modelers with its syntax. Our ultimate aim is to reduce the effort required to specify constraints, subsequently lowering down training cost and increasing the correctness of the constraints. iOCL is a web-based ap- plication that integrates other tools including Eclipse OCL for validation and evaluation of OCL constraints, and EsOCL for automatically generating valid instances of models that satisfy the specified constraints.
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | Tool Demonstrations Track, ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MODELS) |
| Pagination | 1-7 |
| Date Published | 09/2016 |
U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies
Uncertainty is intrinsic in Cyber-Physical Systems (CPS) owning to novel interactions among software, embedded systems, networking equipment, cloud infrastructures, and agents (e.g., humans). Such systems have become predominantly visible in critical industrial domains (e.g., healthcare and transportation) and oblige the implementation of proper mechanisms to deal with uncertainty during their real operation. One way to ensure the correct implementation of such mechanisms is with automated testing. The U-Test project aims at ensuring that CPS are tested adequately under uncertainty using systematic and automated techniques such as model and search-based testing to facilitate their reliable operation.
U-Test keeps a full catalog of the project's publications, including publications from other academic institutions than Simula.
Final goal
To improve the dependability of Cyber-Physical Systems (CPS) via cost-effective model-based and search-based testing of CPS under uncertainty, by defining an Uncertainty Taxonomy and holistic modeling and testing frameworks with considerable reliance on standards.
Funding source
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. 645463. (ICT-01-2014 - Smart Cyber-Physical Systems)
All partners
- Oslo Medtech (Norway)
- Simula (Norway)
- Technical University of Vienna (Austria)
- Fraunhofer FOKUS (Germany)
- Future Position X (Sweden)
- ULMA Handling Systems (Spain)
- Nordic MedTest (Sweden)
- Easy Global Market (France)
- Ikerlan (Spain)
Technical project leader
Shaukat Ali (PI)
Standardization leader
Tao Yue (Co-PI)
Media presence
Publications for U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies
Journal Article
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
Journal of Systems and Software 153 (2019).Status: Published
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Journal of Systems and Software |
| Volume | 153 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
Journal Article
Specifying Uncertainty in Use Case Models
Journal of Systems and Software 144 (2018): 573-603.Status: Published
Specifying Uncertainty in Use Case Models
Context: Latent uncertainty in the context of software-intensive systems (e.g., Cyber-Physical Systems (CPSs)) demands explicit attention right from the start of development. Use case modeling—a commonly used method for specifying requirements in practice, should also be extended for explicitly specifying uncertainty.
Objective: Since uncertainty is a common phenomenon in requirements engineering, it is best to address it explicitly by identifying, qualifying, and, where possible, quantifying uncertainty at the beginning stage. The ultimate aim, though not within the scope of this paper, was to use these use cases as the starting point to create test-ready models to support automated testing of CPSs under uncertainty.
Method: We extend the Restricted Use Case Modeling (RUCM) methodology and its supporting tool to specify uncertainty as part of system requirements. Such uncertainties include those caused by insufficient domain expertise of stakeholders, disagreements among them, and known uncertainties about assumptions about the environment of the system. The extended RUCM, called U-RUCM, inherits the features of RUCM, such as automated analyses and generation of models, to mention but a few. Consequently, U-RUCM provides all the key benefits offered by RUCM (i.e., reducing ambiguities in requirements), but also, it allows specification of uncertainties with the possibilities of reasoning and refining existing ones and even uncovering unknown ones.
Results: We evaluated U-RUCM with two industrial CPS case studies. After refining RUCM models (specifying initial requirements), by applying the U-RUCM methodology, we successfully identified and specified additional 306% and 512% (previously unknown) uncertainty requirements, as compared to the initial requirements specified in RUCM. This showed that, with U-RUCM, we were able to get a significantly better and more precise characterization of uncertainties in requirement engineering.
Conclusion: Evaluation results show that U-RUCM is an effective methodology (with tool support) for dealing with uncertainty in requirements engineering. We present our experience, lessons learned, and future challenges, based on the two industrial case studies.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | Journal of Systems and Software |
| Volume | 144 |
| Pagination | 573-603 |
| Publisher | Elsevier |
| Keywords | Belief, Uncertainty, Use Case Modeling |
| DOI | 10.1016/j.jss.2018.06.075 |
PhD Thesis
Uncertainty-wise Cyber-Physical Systems Testing
In The University of Oslo. Vol. PhD. Norway: The University of Oslo, 2018.Status: Published
Uncertainty-wise Cyber-Physical Systems Testing
A Cyber-Physical Systems (CPS), as an integration of computing, communication, and control for making intelligent and autonomous systems, has been widely applied in various safety-critical domains, e.g., avionics and automotive. However, uncertainty is inherent in CPSs due to various reasons such as unpredictable environment under which the CPSs are operated. And, uncertainties may cause irreparable accidents once they cannot be handled properly by CPSs. Therefore, it is crucial to identify uncertainties in CPSs and test CPSs under the uncertainties, to ensure that CPSs are capable of handling the uncertainties during their actual operations, i.e., making CPSs less uncertain.
Towards this direction, five contributions were made in the thesis corresponding to five papers respectively: (C1) a conceptual model, named as U-Model, for helping develop a systematic and comprehensive understanding of uncertainty in CPSs; (C2) an use case modeling methodology, named as U-RUCM, for identifying, qualifying, and, where possible, quantifying uncertainty in requirements engineering; (C3) a test modeling methodology, named as UncerTum, for supporting the construction of test ready models with the explicit representation of uncertainties in CPSs; (C4) an evolution framework, named as UncerTolve, for interactively evolving test ready models specified with UncerTum based on real operational data; and (C5) a testing framework, named as UncerTest, for testing CPSs in the presence of uncertainties in their operating environments in a cost-effective manner using model-based and search-based testing techniques.
Based on our evaluations of the five contributions with the industrial CPS case studies, we observed that U-Model, as the foundation for this research, is sufficiently complete for characterizing and classifying uncertainties in CPSs. Then, the U-Model based modeling methodologies U-RUCM and UncerTum offer solutions to enable the identification and specification of uncertainties at two critical phases of a system development lifecycle: requirements engineering and testing. Furthermore, UncerTolve can successfully evolve model elements of the test ready models specified with UncerTum and calculate objective uncertainty measurements based on real operational data. Last, UncerTest managed to cost-effectively test CPSs in the presence of uncertainties and proactively identify unknown uncertainties by introducing the sources of the uncertainties into the test environments during test case execution.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBT4CPS: Model-Based Testing For Cyber-Physical Systems , The Certus Centre (SFI) |
| Publication Type | PhD Thesis |
| Year of Publication | 2018 |
| Degree awarding institution | The University of Oslo |
| Degree | PhD |
| Number of Pages | 292 |
| Date Published | 26/06/2018 |
| Publisher | The University of Oslo |
| Place Published | Norway |
| Keywords | Cyber-Physical System, Model-based Testing, Uncertainty |
Book Chapter
Uncertainty-wise Testing of Cyber-Physical Systems
In Advances in Computers, 23-94. Vol. 107. Elsevier, 2017.Status: Published
Uncertainty-wise Testing of Cyber-Physical Systems
As compared with classical software/system testing, uncertainty-wise testing explicitly addresses known uncertainty about the behavior of a System Under Test (SUT), its operating environment, and interactions between the SUT and its operational environment, across all testing phases, including test design, test generation, test optimization, and test execution, with the aim to mainly achieve the following two goals. First, uncertainty-wise testing aims to ensure that the SUT deals with known uncertainty adequately. Second, uncertainty-wise testing should be also capable of learning new (previously unknown) uncertainties such that the SUT’s implementation can be improved to guard against newly learned uncertainties during its operation. The necessity to integrate uncertainty in testing is becoming imperative because of the emergence of new types of intelligent and communicating software-based systems such as Cyber-Physical Systems (CPSs). Intrinsically, such systems are exposed to uncertainty because of their interactions with highly indeterminate physical environments. In this chapter, we provide our understanding and experience of uncertainty-wise testing from the aspects of uncertainty-wise model-based testing, uncertainty-wise modeling and evolution of test ready models, and uncertainty-wise multi-objective test optimization, in the context of testing CPSs under uncertainty. Furthermore, we present our vision about this new testing paradigm and its plausible future research directions.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Book Chapter |
| Year of Publication | 2017 |
| Book Title | Advances in Computers |
| Volume | 107 |
| Chapter | 2 |
| Pagination | 23-94 |
| Publisher | Elsevier |
Journal Article
Uncertainty-Wise Cyber-Physical System Test Modeling
Software & Systems Modeling (2017).Status: Published
Uncertainty-Wise Cyber-Physical System Test Modeling
It is important that a Cyber-Physical System (CPS) deals with uncertainty in its behavior caused by its unpredictable operating environment, to ensure its reliable operation. One method to ensure that the CPS will handle such uncertainty during its operation is by testing the CPS with Model-based Testing (MBT) techniques. However, existing MBT techniques do not explicitly capture uncertainty in test ready models i.e., capturing the uncertain expected behavior of a CPS in the presence of environment uncertainty. To fill this gap, we present an Uncertainty-Wise test-modeling framework, named as Uncertum, to create test ready models to support MBT of CPSs facing uncertainty. Uncertum relies on the definition of a UML profile (the UML Uncertainty Profile (UUP)) and a set of UML model libraries extending the UML profile for Modeling and Analysis of Real-Time and Embedded Systems (MARTE). Uncertum also benefits from the UML Testing Profile (UTP) V.2 to support standard-based MBT. Uncertum was evaluated with two industrial CPS case studies, one real-world case study, and one open source CPS case study from the following four perspectives: 1) Completeness and Coverage of the profiles and model libraries in terms of concepts defined in their underlying uncertainty conceptual model for CPSs (i.e., U-Model and MARTE, 2) Effort required to model uncertainty with Uncertum, and 3) Correctness of the developed test ready models, which was assessed via model execution. Based on the evaluation, we can conclude that we were successful in modeling all the uncertainties identified in the four case studies, which gives us an indication that Uncertum is sufficiently complete. In terms of modeling effort, we concluded that on average Uncertum requires18.5% more time to apply stereotypes from UUP on test ready models.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Software & Systems Modeling |
| Publisher | Springer |
| ISSN | 1619-1374 |
| Keywords | Cyber-Physical System; UML, Model-based Testing, Uncertainty |
| DOI | 10.1007/s10270-017-0609-6 |
Uncertainty-Wise Evolution of Test Ready Models
Information and Software Technology (IST) 87 (2017): 140-159.Status: Published
Uncertainty-Wise Evolution of Test Ready Models
Context: Cyber-Physical Systems (CPSs), when deployed for operation, are inherently prone to uncertainty. Considering their applications in critical domains (e.g., healthcare), it is important that such CPSs are tested sufficiently, with the explicit consideration of uncertainty. Model-based testing (MBT) involves creating test ready models capturing the expected behavior of a CPS and its operating environment. These test ready models are then used for generating executable test cases. It is, therefore, necessary to develop methods that can continuously evolve, based on real operational data collected during the operation of CPSs, test ready models and uncertainty captured in them, all together termed as Belief Test Ready Models (BMs)
Objective: Our objective is to propose a model evolution framework that can interactively improve the quality of BMs, based on operational data. Such BMs are developed by one or more test modelers (belief agents) with their assumptions about the expected behavior of a CPS, its expected physical environment, and potential future deployments. Thus, these models explicitly contain subjective uncertainty of the test modelers.
Method: We propose a framework (named as UncerTolve) for interactively evolving BMs (specified with extended UML notations) of CPSs with subjective uncertainty developed by test modelers. The key inputs of UncerTolve include initial BMs of CPSs with known subjective uncertainty and real data collected from the operation of CPSs. UncerTolve has three key features: 1) Validating the syntactic correctness and conformance of BMs against real operational data via model execution, 2) Evolving objective uncertainty measurements of BMs via model execution, and 3) Evolving state invariants (modeling test oracles) and guards of transitions (modeling constraints for test data generation) of BMs with a machine learning technique.
Results: As a proof-of-concept, we evaluated UncerTolve with one industrial CPS case study, i.e., GeoSports from the healthcare domain. Using UncerTolve, we managed to evolve 51% of belief elements, 18% of states, and 21% of transitions as compared to the initial BM developed in an industrial setting.
Conclusion: UncerTolve can successfully evolve model elements of the initial BM, in addition to objective uncertainty measurements using real operational data. The evolved model can be used to generate additional test cases covering evolved model elements and objective uncertainty. These additional test cases can be used to test the current and future deployments of a CPS to ensure that it will handle uncertainty gracefully during its operations.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Information and Software Technology (IST) |
| Volume | 87 |
| Pagination | 140-159 |
| Publisher | Elsevier |
| ISSN | 0950-5849 |
| Keywords | Belief Model, Belief Test Ready Model, Model Evolution, Model-based Testing, Uncertainty |
| DOI | 10.1016/j.infsof.2017.03.003 |
Search and similarity based selection of use case scenarios: An empirical study
Empirical Software Engineering (2017): 1-78.Status: Published
Search and similarity based selection of use case scenarios: An empirical study
Use case modeling is a well-known requirements specification method and has been widely applied in practice. Use case scenarios of use case models are input elements for requirements inspection and analysis, requirements-based testing, and other downstream activities. It is, however, a practical challenge to inspect all use case scenarios that can be obtained from any non-trivial use case model, as such an inspection activity is often performed manually by domain experts. Therefore, it is needed to propose an automated solution for selecting a subset of use case scenarios with the ultimate aim of enabling cost-effective requirements (use case) inspection, analysis, and other relevant activities. Our solution is built on a natural language based, restricted use case modeling methodology (named as RUCM), in the sense that requirements specifications are specified as RUCM use case models. Use case scenarios can be automatically derived from RUCM use case models with the already established Zen-RUCM framework. In this paper, we propose a search-based and similarity-based approach called S3RCUM, through an empirical study, to select most diverse use case scenarios to enable cost-effective use case inspections. The empirical study was designed to evaluate the performance of three search algorithms together with eight similarity functions, through one real-world case study and six case studies from literature. Results show that (1+1) Evolutionary Algorithm together with Needleman-Wunsch similarity function significantly outperformed the other 31 combinations of the search algorithms and similarity functions. The combination managed to select 50% of all the generated RUCM use case scenarios for all the case studies to detect all the seeded defects.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems , U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Empirical Software Engineering |
| Pagination | 1-78 |
| Date Published | 04/2017 |
| Publisher | Springer |
Miscellaneous
Uncertainty Testing Framework V.3
None, 2017.Status: Published
Uncertainty Testing Framework V.3
This is a public deliverable for U-Test project.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Miscellaneous |
| Year of Publication | 2017 |
| Publisher | None |
Uncertainty Testing Framework V.2
None, 2017.Status: Published
Uncertainty Testing Framework V.2
This is a public deliverable for the U-Test project.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Miscellaneous |
| Year of Publication | 2017 |
| Publisher | None |
Uncertainty Modeling Framework Version 2
None, 2017.Status: Published
Uncertainty Modeling Framework Version 2
This a U-Test public deliverable.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Miscellaneous |
| Year of Publication | 2017 |
| Publisher | None |
MPM4CPS: Multi-Paradigm Modelling for Cyber-Physical Systems
To date there is neither unifying theory, systematic design methods, nor techniques and tools for Cyber-Physical Systems (CPS). Individual engineering disciplines offer only partial solutions. Multi-paradigm Modelling (MPM) proposes to model every part and aspect of a system explicitly, at the most appropriate level of abstraction, using the most appropriate modelling formalism. The MPM4CPS action aims to promote the sharing of foundations, techniques, and tools to academia and industry. By bringing together and disseminating knowledge and experiments on CPS and MPM solutions across disciplines this goal can be achieved.
Funding source:
EU COST Action
Management Committee Members:
Tao Yue, Shaukat Ali
Publications for MPM4CPS: Multi-Paradigm Modelling for Cyber-Physical Systems
Talks, contributed
U-TCsGM: Generating and Minimizing Uncertainty-Based Test Cases for Cyber-Physical Systems (Tool Demo)
In MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016, 2016.Status: Published
U-TCsGM: Generating and Minimizing Uncertainty-Based Test Cases for Cyber-Physical Systems (Tool Demo)
In this tool demo, we will present the implementation of our recent research work on generating and minimizing executable test cases from the test models of a Cyber-Physical System tagged with subjective uncertainty. The algorithms are founded on uncertainty theory and NSGA-II—the most commonly used multi-objective search algorithm. We will demonstrate the complete process starting from creating a test model with uncertainty, generating test cases, minimizing test cases, and finally executing the minimized test cases using a real CPS case study.
| Afilliation | Software Engineering, Software Engineering |
| Project(s) | MPM4CPS: Multi-Paradigm Modelling for Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2016 |
| Location of Talk | MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016 |
| Keywords | Model-based Testing, Search-Based Testing, Uncertainty |
Model-Driven Testing of Cyber-Physical Systems with the Explicit Consideration of Uncertainty
In MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016, 2016.Status: Published
Model-Driven Testing of Cyber-Physical Systems with the Explicit Consideration of Uncertainty
It is a well-recognized fact that Cyber-Physical Systems (CPSs) face both known and unknown uncertainty during their operation. This demands the development of testing techniques that must take into account known uncertainty both in the CPS and its environment with the final goal of discovering unknown uncertainty against which the CPS can be tested. Eventually, the implementation of the CPS can be improved to shield it against the newly discovered uncertainty. In this presentation, we will present some of the results that we have achieved in an EU Horizon2020 project (U-Test) in this regard. We, first, demonstrate the modeling of test ready models of CPSs together with subjective uncertainty using the Uncertainty Modeling Framework. Second, we present two uncertainty-based test case generation and four test case minimization techniques relying on the test ready models founded on Uncertainty theory and multi-objective search. Third, we present the evaluation of the test case generation and minimization techniques that was conducted to select the best strategy to be used in the practice to test a real CPS. Fourth, we present the results of testing a real CPS with the selected best strategy.
| Afilliation | Software Engineering |
| Project(s) | MPM4CPS: Multi-Paradigm Modelling for Cyber-Physical Systems, The Certus Centre (SFI), U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Talks, contributed |
| Year of Publication | 2016 |
| Location of Talk | MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016 |
| Keywords | Cyber-Physical System, Model-based Testing, Search-Based Testing, Uncertainty |
Integrating Uncertainty Modelling with Use Case Modelling to Discover Unknowns
In MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016, 2016.Status: Published
Integrating Uncertainty Modelling with Use Case Modelling to Discover Unknowns
Use case modeling is a commonly used means for specifying requirements in practice. In the past, we have developed a use case modeling solution (with tool support), named as Restricted Use Case Modeling (RUCM), for the purpose of reducing inherent ambiguities of textual requirements and enabling automated analyses and generations. However, such use case models still contain uncertainties due to various reasons such as insufficient domain expertise and disagreement among stakeholders. We therefore, integrate RUCM with Uncertainty Modeling to provide requirements engineers an integrated platform (named as U-RUCM) for explicitly specifying uncertainties as part of use case models, such that both ambiguities and uncertainties in use case models can be reduced. U-RUCM was devised in the context of the EU Horizon 2020 U-Test project (http://www.u-test.eu/) and has been evaluated with two industrial case studies of two industrial partners of the U-Test consortium.
| Afilliation | Software Engineering, Software Engineering |
| Project(s) | MPM4CPS: Multi-Paradigm Modelling for Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2016 |
| Location of Talk | MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016 |
| Keywords | Modeling, RUCM, Uncertainty |
Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines
The goal of the Zen-Configurator project is to increase the efficiency and effectiveness, and thereby reduce the cost, of configuring large-scale Cyber Physical System (CPS) product lines. To achieve this goal, we maximally automate error-prone and costly manual configuration activities and optimally assist the interactive configuration process. On one hand, the project relies on advanced technologies of constraint solving/evaluation, optimization using search algorithms, and propose state-of-art algorithms to enable automated configuration activities. On the other hand, the project grounds itself to address real challenges faced by industry and propose a practical and applicable solution and apply it in at least one application domain.
Funding source:
Research Council of Norway
All partners:
Simula Research Laboratory
Project leaders:
Tao Yue (PI), Shaukat Ali (Co-PI)
Publications for Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines
Journal Article
A framework for automated multi‑stage and multi‑step product confguration of cyber‑physical systems
Software and Systems Modeling (SoSym) 19, no. 4 (2020): 1-55.Status: Published
A framework for automated multi‑stage and multi‑step product confguration of cyber‑physical systems
Product line engineering (PLE) has been employed to large-scale cyber-physical systems (CPSs) to provide customization based on users’ needs. A PLE methodology can be characterized by its support for capturing and managing the abstractions as commonalities and variabilities and the automation of the confguration process for efective selection and customization of reusable artifacts. The automation of a confguration process heavily relies on the captured abstractions and formally specifed constraints using a well-defned modeling methodology. Based on the results of our previous work and a thorough literature review, in this paper, we propose a conceptual framework to support multi-stage and multi-step automated product confguration of CPSs, including a comprehensive classifcation of constraints and a list of automated functionalities of a CPS confguration solution. Such a framework can serve as a guide for researchers and practitioners to evaluate an existing CPS PLE solution or devise a novel CPS PLE solution. To validate the framework, we conducted three real-world case studies. Results show that the framework fulflls all the requirements of the case studies in terms of capturing and managing variabilities and constraints. Results of the literature review indicate that the framework covers all the functionalities concerned by the literature, suggesting that the framework is complete for enabling the maximum automation of confguration in CPS PLE.
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | Software and Systems Modeling (SoSym) |
| Volume | 19 |
| Issue | 4 |
| Pagination | 1-55 |
| Date Published | 06/2020 |
| Publisher | springer |
| Keywords | Automated configuration, Constraint classification, Cyber-Physical Systems, Multi-stage and multi-step configuration process, Product Line Engineering, Real-world case studies, Variability Modeling |
| DOI | 10.1007/s10270-020-00803-8 |
Journal Article
Using multi-objective search and machine learning to infer rules constraining product configurations
Automated Software Engineering (2019): 1-62.Status: Published
Using multi-objective search and machine learning to infer rules constraining product configurations
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Automated Software Engineering |
| Pagination | 1-62 |
| Publisher | Springer |
| DOI | 10.1007/s10515-019-00266-2 |
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
Journal of Systems and Software 153 (2019).Status: Published
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Journal of Systems and Software |
| Volume | 153 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
Proceedings, refereed
Stability Analysis for Safety of Automotive Multi-Product Lines: A Search-Based Approach
In The Genetic and Evolutionary Computation Conference (GECCO). ACM, 2019.Status: Published
Stability Analysis for Safety of Automotive Multi-Product Lines: A Search-Based Approach
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | The Genetic and Evolutionary Computation Conference (GECCO) |
| Pagination | 1241-1249 |
| Publisher | ACM |
Journal Article
Empirical Research in Software Engineering - a Literature Survey
Journal of Computer Science and Technology 33, no. 5 (2018): 876-899.Status: Published
Empirical Research in Software Engineering - a Literature Survey
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | Journal of Computer Science and Technology |
| Volume | 33 |
| Issue | 5 |
| Pagination | 876-899 |
| Date Published | 09/2018 |
| Publisher | Springer |
| DOI | 10.1007/s11390-018-1864-x |
Proceedings, refereed
Model- Based Personalized Visualization System for Monitoring Evolving Industrial Cyber-Physical System
In The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) . IEEE, 2018.Status: Published
Model- Based Personalized Visualization System for Monitoring Evolving Industrial Cyber-Physical System
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2018 |
| Conference Name | The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) |
| Publisher | IEEE |
Tool Support for Restricted Use Case Specification: Findings from a Controlled Experiment
In The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) . IEEE, 2018.Status: Published
Tool Support for Restricted Use Case Specification: Findings from a Controlled Experiment
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2018 |
| Conference Name | The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) |
| Publisher | IEEE |
Automatic Support of the Generation and Maintenance of Assurance Cases
In Symposium on Dependable Software Engineering: Theories, Tools and Applications. Cham: Springer International Publishing, 2018.Status: Published
Automatic Support of the Generation and Maintenance of Assurance Cases
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2018 |
| Conference Name | Symposium on Dependable Software Engineering: Theories, Tools and Applications |
| Pagination | 11-28 |
| Date Published | 08/2018 |
| Publisher | Springer International Publishing |
| Place Published | Cham |
| DOI | 10.1007/978-3-319-99933-3_2 |
Talks, contributed
Automated Refactoring of OCL Constraints with Search
In ICSE 2018, Gothenburg, Sweden, 2018.Status: Published
Automated Refactoring of OCL Constraints with Search
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Talks, contributed |
| Year of Publication | 2018 |
| Location of Talk | ICSE 2018, Gothenburg, Sweden |
REMAP: Using Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
In IEEE Conference on Software Testing, Validation and Verification (ICST), Västerås, Sweden, 2018.Status: Published
REMAP: Using Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2018 |
| Location of Talk | IEEE Conference on Software Testing, Validation and Verification (ICST), Västerås, Sweden |
MBT4CPS: Model-Based Testing For Cyber-Physical Systems
The complexity of Cyber Physical Systems presents unprecedented challenges for testing both nominal functionality and its associated extra-functional properties in unexpected situations. Thus, we need basic research to handle difficulties imposed by testing. Given the breadth and depth of the topic, as a starting point, we emphasize exclusively on dealing with the following two types of uncertain and risky situations. First, we focus on testing security features together in unpredictable and risky situations. Second, we concentrate on testing self-healing features, i.e., the ability of a CPS to recover from faults itself in risk and unpredictable situations. The successful completion of the project will produce distinct testing methods to test systems. Once such well-tested systems start operating in real life, these will be secure and safe.
Publications for the MBT4CPS project can be found in Simula's publication database.
Funding source:
Research Council of Norway
All partners:
Simula Research Laboratory
Project leaders:
Shaukat Ali (PI), Tao Yue (Co-PI)
Publications for MBT4CPS: Model-Based Testing For Cyber-Physical Systems
Journal Article
Testing Self-Healing Cyber-Physical Systems under Uncertainty with Reinforcement Learning: An Empirical Study
Empirical Software Engineering (2021).Status: Accepted
Testing Self-Healing Cyber-Physical Systems under Uncertainty with Reinforcement Learning: An Empirical Study
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | Empirical Software Engineering |
| Publisher | Springer |
Journal Article
Quality Indicators in Search-based Software Engineering
ACM Transactions on Software Engineering and Methodology 29, no. 2 (2020): 1-29.Status: Published
Quality Indicators in Search-based Software Engineering
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Software Engineering and Methodology |
| Volume | 29 |
| Issue | 2 |
| Pagination | 1 - 29 |
| Date Published | May-04-2020 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 1049-331X |
| URL | https://dl.acm.org/doi/10.1145/3375636 |
| DOI | 10.1145/3375636 |
Journal Article
Using multi-objective search and machine learning to infer rules constraining product configurations
Automated Software Engineering (2019): 1-62.Status: Published
Using multi-objective search and machine learning to infer rules constraining product configurations
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Automated Software Engineering |
| Pagination | 1-62 |
| Publisher | Springer |
| DOI | 10.1007/s10515-019-00266-2 |
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
Journal of Systems and Software 153 (2019).Status: Published
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Journal of Systems and Software |
| Volume | 153 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
Testing Self-Healing Cyber-Physical Systems under Uncertainty: A Fragility-Oriented Approach
Software Quality Journal 27, no. 2 (2019): 615-649.Status: Published
Testing Self-Healing Cyber-Physical Systems under Uncertainty: A Fragility-Oriented Approach
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Software Quality Journal |
| Volume | 27 |
| Issue | 2 |
| Pagination | 615–649 |
| Date Published | 03/2019 |
| Publisher | Springer |
| URL | https://link.springer.com/article/10.1007/s11219-018-9437-3 |
Proceedings, refereed
Towards a Framework for the Analysis of Multi-Product Lines in the Automotive Domain
In Proceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems. New York, NY, USA: ACM, 2019.Status: Published
Towards a Framework for the Analysis of Multi-Product Lines in the Automotive Domain
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | Proceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems |
| Publisher | ACM |
| Place Published | New York, NY, USA |
Edited books
Editorial to the Theme Issue on Model-based Testing
Software & Systems Modeling: Springer, 2018.Status: Published
Editorial to the Theme Issue on Model-based Testing
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Edited books |
| Year of Publication | 2018 |
| Publisher | Springer |
| Place Published | Software & Systems Modeling |
First International Workshop on Verification and Validation of Internet of Things
IEEE, 2018.Status: Published
First International Workshop on Verification and Validation of Internet of Things
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Edited books |
| Year of Publication | 2018 |
| Publisher | IEEE |
Journal Article
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
Software and Systems Modeling (2018): 1-31.Status: Published
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
Self-healing Cyber-Physical Systems (SH-CPSs) detect and recover from faults by themselves at runtime. Testing such systems is challenging due to the complex implementation of self-healing behaviors and their interaction with the physical environment, both of which are uncertain. To this end, we propose an executable model-based approach to test self-healing behaviors under environmental uncertainties. The approach consists of a Modeling Framework of SH-CPSs (MoSH) and an accompanying Test Model Executor (TM-Executor). MoSH provides a set of modeling constructs and a methodology to specify executable test models, which capture expected system behaviors and environmental uncertainties. TM-Executor executes the test models together with the systems under test, to dynamically test their self-healing behaviors under uncertainties. We demonstrated the successful application of MoSH to specify 11 self-healing behaviors and 17 uncertainties for three SH-CPSs. The time spent by TM-Executor to perform testing activities was in the order of milliseconds, though the time spent was strongly correlated with the complexity of test models.
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | Software and Systems Modeling |
| Pagination | 1-31 |
| Date Published | 11/2018 |
| Publisher | Springer |
| Place Published | Berlin Heidelberg |
| ISSN | 1619-1374 |
| DOI | 10.1007/s10270-018-00703-y |
Employing Multi-Objective Search to Enhance Reactive Test Case Generation and Prioritization for Testing Industrial Cyber Physical Systems
IEEE Transactions on Industrial Informatics (TII) 14, no. 3 (2018): 1055-1066.Status: Published
Employing Multi-Objective Search to Enhance Reactive Test Case Generation and Prioritization for Testing Industrial Cyber Physical Systems
The test case generation and prioritization of industrial Cyber-Physical Systems (CPSs) face critical challenges and simulation-based testing is one of the most commonly used techniques for testing these complex systems. However, simulation models of industrial CPSs are usually very complex and executing the simulations becomes computationally expensive, which often make it infeasible to execute all the test cases. To address these challenges, this paper proposes a multi-objective test generation and prioritization approach for testing industrial CPSs by defining a fitness function with four objectives and designing different crossover and mutation operators. We empirically evaluated our fitness function and designed operators along with five multi-objective search algorithms (e.g., Non-dominated Sorting Genetic Algorithm (NSGA-II)) using four case studies. The evaluation results demonstrated that NSGA-II achieved significantly better performance than the other algorithms and managed to improve Random Search for on average 43.80% for each objective and 49.25% for the quality indicator Hypervolume (HV).
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | IEEE Transactions on Industrial Informatics (TII) |
| Volume | 14 |
| Issue | 3 |
| Pagination | 1055-1066 |
| Publisher | IEEE |
| URL | http://ieeexplore.ieee.org/abstract/document/8241845/ |
| DOI | 10.1109/TII.2017.2788019 |
Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems
The primary objective of the Co-evolver project is to explore and exploit the coevolution design of a self-adaptive Cyber-physical Systems (CPSs) to a given level of maturity before deployment and enable the self-evolution of its coevolution strategy during operation, by drawing on theories and technologies from model-based engineering, evolutionary computation, and machine learning. The key scientific outcomes are 1) a multi-paradigm modeling framework for developing executable coevolution design models, 2) novel (co-)evolutionary algorithms and advanced applied studies on uncertainty-related theories and machine learning techniques to enable the continuous exploration and exploitation of coevolution designs, and 3) a comprehensive platform for evolving coevolution design models. The secondary objective is to apply the outcomes to at least one self-CPS application domain, opening a new stream of research in the domain of uncertainty-aware coevolution designs of self-CPSs.
Funding source:
Research Council of Norway
Project leaders:
Simula Research Laboratory
Publications for Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems
Journal Article
Uncertainty-wise Requirements Prioritization with Search
ACM Transactions on Software Engineering and Methodology 30, no. 1 (2020): 1-54.Status: Published
Uncertainty-wise Requirements Prioritization with Search
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Software Engineering and Methodology |
| Volume | 30 |
| Issue | 1 |
| Pagination | 1-54 |
| Date Published | December 2020 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 1049-331X |
| URL | https://doi.org/10.1145/3408301 |
| DOI | 10.1145/3408301 |
Pattern-based Interactive Configuration Derivation for Cyber-Physical System Product Lines
ACM Transactions on Cyber-Physical Systems 4, no. 4 (2020): 24.Status: Published
Pattern-based Interactive Configuration Derivation for Cyber-Physical System Product Lines
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Cyber-Physical Systems |
| Volume | 4 |
| Issue | 4 |
| Pagination | 24 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 2378-962X |
| URL | https://doi.org/10.1145/3389397 |
| DOI | 10.1145/3389397 |
Proceedings, refereed
Do Quality Indicators Prefer Particular Multi-Objective Search Algorithms in Search-Based Software Engineering?
In 12th Symposium on Search-Based Software Engineering. LNCS, 2020.Status: Published
Do Quality Indicators Prefer Particular Multi-Objective Search Algorithms in Search-Based Software Engineering?
In Search-Based Software Engineering (SBSE), users typically select a set of Multi-Objective Search Algorithms (MOSAs) for their experiments without any justification, or they simply choose an MOSA because of its popularity (e.g., NSGA-II). On the other hand, users know certain characteristics of solutions they are interested in. Such characteristics are typically measured with Quality Indicators (QIs) that are commonly used to evaluate the quality of solutions produced by an MOSA. Consequently, these QIs are often employed to empirically evaluate a set of MOSAs for a particular search problem to find the best MOSA. Thus, to guide SBSE users in choosing an MOSA that represents the solutions measured by a specific QI they are interested in, we present an empirical evaluation with a set of SBSE problems to study the relationships among commonly used QIs and MOSAs in SBSE. Our aim, by studying such relationships, is to identify whether there are certain characteristics of a QI because of which it prefers a certain MOSA. Such preferences are then used to provide insights and suggestions to SBSE users in selecting an MOSA, given that they know which quality aspects of solutions they are looking for.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | 12th Symposium on Search-Based Software Engineering |
| Publisher | LNCS |
Simultaneously Searching and Solving Multiple Avoidable Collisions for Testing Autonomous Driving Systems
In The Genetic and Evolutionary Computation Conference (GECCO). ACM, 2020.Status: Published
Simultaneously Searching and Solving Multiple Avoidable Collisions for Testing Autonomous Driving Systems
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | The Genetic and Evolutionary Computation Conference (GECCO) |
| Publisher | ACM |
Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems
In IEEE International Conference on Software Testing, Verification and Validation (ICST) 2020. IEEE, 2020.Status: Published
Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | IEEE International Conference on Software Testing, Verification and Validation (ICST) 2020 |
| Pagination | 375-386 |
| Publisher | IEEE |
Talks, contributed
Empowering Model-based Engineering with the CynefinFramework for Systematic Uncertainty Thinking
In 1st Uncertainty in Modeling Workshop 2020, Co-located with the ACM/IEEE 213rd International Conference on Model Driven Engineering Languages and Systems, 2020.Status: Published
Empowering Model-based Engineering with the CynefinFramework for Systematic Uncertainty Thinking
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | 1st Uncertainty in Modeling Workshop 2020, Co-located with the ACM/IEEE 213rd International Conference on Model Driven Engineering Languages and Systems |
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
In IEEE International Conference on Software Testing, Verification and Validation (ICST 2020), 2020.Status: Published
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | IEEE International Conference on Software Testing, Verification and Validation (ICST 2020) |
| Type of Talk | Journal First Presentation |
Specifying Uncertainty in Use Case Models
In International Conference on Model-Driven Engineering and Software Development (MODELSWARD 2020). INSTICC , 2020.Status: Published
Specifying Uncertainty in Use Case Models
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | International Conference on Model-Driven Engineering and Software Development (MODELSWARD 2020) |
| Publisher | INSTICC |
Edited books
Proceedings of the 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS)
IEEE, 2019.Status: Published
Proceedings of the 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS)
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Edited books |
| Year of Publication | 2019 |
| Publisher | IEEE |
The 15th European Conference on Modelling Foundations and Applications (ECMFA)
2nd ed. Vol. 18. The Journal of Object Technology, 2019.Status: Published
The 15th European Conference on Modelling Foundations and Applications (ECMFA)
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Edited books |
| Year of Publication | 2019 |
| Volume | 18 |
| Edition | 2 |
| Date Published | 09/2019 |
| Publisher | The Journal of Object Technology |
| URL | http://www.jot.fm/contents/issue_2019_02.html |
| DOI | 10.5381/jot.2019.18.2.e1 |
Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs
It is well-known that Quantum Computing (QC) has the potential to solve complex problems in various domains and bring breakthroughs in science and technology. Nowadays, quantum applications span over algorithms addressing optimization problems such as radiotherapy optimization, machine learning techniques, e.g., for detecting objects from images, as well as modeling and simulations, e.g., for handling uncertainties when predicting the future. The development of QC is also driven by the urgent need to solve ever-complex and large-scale problems, which current (super)computers cannot solve. QC comes right on time to bring revolutionary computation power to handle such complexity. Testing such applications, however, is a big challenge due to their radically different characteristics from their classical counterparts. This includes superposition, entanglement, and probabilistic nature of qubits. The overall ambition is to develop fundamentally new methods for automated and systematic testing of quantum programs, based on a rigorous theoretical foundation with the ultimate goal of supporting future ubiquitous services and data related to QC applications to guarantee their dependability. Also, to allow for testing complex quantum programs with a minimal amount of QC resources, we will develop novel quantum optimization algorithms. The cost-effectiveness of our methods will be demonstrated by testing quantum programs written in quantum high-level programming languages (e.g., Q#).
Funding source:
IKTPLUSS
Research Council of Norway
Partners:
The University of Malaga, Spain
The University of Maryland, USA
Durham University, UK
Publications for Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs
Proceedings, refereed
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
In IEEE International Conference on Software Testing, Verification and Validation (ICST)). IEEE, 2021.Status: Accepted
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | IEEE International Conference on Software Testing, Verification and Validation (ICST)) |
| Publisher | IEEE |
Proceedings, refereed
Modeling Quantum Programs: Challenges, Initial Results, and Research Directions
In 1st International Workshop on Architectures and Paradigms for Engineering Quantum Software (APEQS). ACM, 2020.Status: Published
Modeling Quantum Programs: Challenges, Initial Results, and Research Directions
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | 1st International Workshop on Architectures and Paradigms for Engineering Quantum Software (APEQS) |
| Publisher | ACM |
Talks, invited
Testing Quantum Programs
In Oslo Metropolitan University, Oslo, Norway, 2020.Status: Published
Testing Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Talks, invited |
| Year of Publication | 2020 |
| Location of Talk | Oslo Metropolitan University, Oslo, Norway |
Talks, invited
Digital Twins for Cyber-Physical Systems, and Quantum Software Engineering: Research Agenda
In National Institute of Informatics, Tokyo, Japan, 2019.Status: Published
Digital Twins for Cyber-Physical Systems, and Quantum Software Engineering: Research Agenda
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Talks, invited |
| Year of Publication | 2019 |
| Location of Talk | National Institute of Informatics, Tokyo, Japan |
| Type of Talk | Colloquium |
Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems
The ADEPTNESS project seeks to implement and investigate a streamlined and automatic workflow that makes methods and tools to be seamlessly used during design phases as well as in operation. We will explore the generation and reuse of test cases and oracles from initial phases of the development to the system in operation and back to the laboratory for reproduction. Integrated into this workflow, unforeseen situations will also be detected in operation to enhance development models for increasing resilience. We will consider several aspects of uncertainties (such as uncertainties in the environment, uncertainty produced due to timing aspects of CPSoS, uncertainty in networks, etc.). Additionally, automatic and synchronized deployment techniques will be investigated to improve the agility of the whole workflow that covers the design-operation continuum.
Coordinator:
Mondragon Goi Eskola Politeknikoa Jose Maria Arizmendiarrieta S Coop
Funding:
Horizon 2020
Publications for Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems
Proceedings, refereed
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
In th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation. LNCS, 2021.Status: Accepted
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation |
| Publisher | LNCS |
Talks, contributed
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
In 5th Workshop on Monitoring and Testing of Cyber-Physical Systems (MT-CPS), 2020.Status: Published
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | 5th Workshop on Monitoring and Testing of Cyber-Physical Systems (MT-CPS) |
Technical reports
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
Simula Research Laboratory, 2020.Status: Published
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
It is a well-recognized fact that a Cyber-Physical System (CPS) experiences uncertain (including unknown) situations during their operations. Some of such uncertainties could potentially lead to failures of CPS operations. Factors contribute to such uncertainties include 1) the intrinsically unpredictable physical environment of a CPS, 2) the use of communication networks continuously experiencing problems (e.g., slower connection than expected), and 3) the increasing use of machine learning algorithms in CPSs which introduce inherent uncertainties to these CPSs. No matter how meticulously a CPS is designed and developed, it is impossible to predict all possible uncertain situations it will experience during its operation. Thus, there is a need for new methods for discovering and handling uncertain situations during the CPS operation to prevent it from failure. In this paper, we present our ideas on how digital twins, i.e., "live models" of CPSs can help in discovering and handling potentially unsafe situations during its operation. We present the research challenges and potential solutions to develop, deploy, and operate such digital twins. The presented work is planned to be performed in a recently accepted European research-based innovation project, ADEPTNESS.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Technical reports |
| Year of Publication | 2020 |
| Publisher | Simula Research Laboratory |
| Other Numbers | 2020-01 |
Department of Engineering Complex Software Systems
There is an increased reliance on our daily life on Complex Software Systems such as smart Cyber-Physical Systems and Internet of Things. Such systems are used in many safety/mission-critical such as healthcare, autonomous cars, boats and underwater vehicles, maritime, energy, and smart roads, grids, buildings, and cities. Such systems are progressively turning into complex, heterogeneous, open, networked and “smart” systems that are composed of agents, sensors, actuators, information networks, and dedicated middleware and infrastructures. Given these complexities and high interaction with the physical environment, new, smart and efficient Software Engineering (SE) paradigms are needed to design, develop, test, and maintain such systems to ensure their dependability (e.g., safety) as well as other relevant quality attributes such as security and privacy.
Such systems are typically built by integrating various physical units, with known and uncertain assumptions on their physical operating environments, future deployments, middleware, and infrastructures. However, as the first dimension of the complexity and challenges, during a real operation of such a system, there is no guarantee that these (often flawed) assumptions will hold, which makes such a system vulnerable to unforeseen loopholes concerning one or more of the quality attributes. Therefore, a novel SE paradigm is expected to manage and handle uncertainty information (e.g., uncertain assumptions) throughout complex system development lifecycles in an intelligent manner. The second dimension is due to the increased deployment of Artificial Intelligence (AI) components for all kinds of reasoning and recognition tasks in modern complex systems. Current system development tools and techniques are not sufficient to cope with this complexity and challenge, and therefore a radical shift moving toward a novel SE paradigm is urgently expected. The third dimension concerns with the multi-disciplinary nature of developing, maintaining, and operating complex software systems, which concerns not only technical aspects but also ethical principles and address societal concerns especially when developing (AI-enabled) smart systems.
The aim of the Department of Engineering Complex Software Systems (ComplexSE) is to develop novel SE paradigms to address the challenges mentioned above to facilitate engineering of modern complex software systems. The ComplexSE department establishes itself with several well-established SE sub-disciplines including Model-Based Engineering, Model-Based Testing, Search-based Software Engineering, and Empirical Software Engineering.
Publications for Department of Engineering Complex Software Systems
Book Chapter
Artificial Intelligence Paradigms for Smart Cyber-Physical SystemsMalicious - URL Detection Using Machine Learning
160-180. Vol. 488. USA: IGI Global, 2021.Status: Published
Artificial Intelligence Paradigms for Smart Cyber-Physical SystemsMalicious - URL Detection Using Machine Learning
Recently, with the increase in Internet usage, cybersecurity has been a significant challenge for computer systems. Different malicious URLs emit different malicious software and try to capture user information. Signature-based approaches have often been used to detect such websites and detected malicious URLs have been attempted to restrict access by using various security components. This chapter proposes using host-based and lexical features of the associated URLs to better improve the performance of classifiers for detecting malicious web sites. Random forest models and gradient boosting classifier are applied to create a URL classifier using URL string attributes as features. The highest accuracy was achieved by random forest as 98.6%. The results show that being able to identify malicious websites based on URL alone and classify them as spam URLs without relying on page content will result in significant resource savings as well as safe browsing experience for the user.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Book Chapter |
| Year of Publication | 2021 |
| Volume | 488 |
| Pagination | 160 - 180 |
| Date Published | 11/2020 |
| Publisher | IGI Global |
| Place Published | USA |
| ISBN | 2327-3453 |
| URL | https://www.igi-global.com/book/artificial-intelligence-paradigms-smart-... |
| DOI | 10.4018/ASASEHPC10.4018/978-1-7998-5101-110.4018/978-1-7998-5101-1.ch008 |
Journal Article
Data augmentation based malware detection using convolutional neural networks
PeerJ Computer Science 7 (2021): e346.Status: Published
Data augmentation based malware detection using convolutional neural networks
Due to advancements in malware competencies, cyber-attacks have been broadly observed in the digital world. Cyber-attacks can hit an organization hard by causing several damages such as data breach, financial loss, and reputation loss. Some of the most prominent examples of ransomware attacks in history are WannaCry and Petya, which impacted companies’ finances throughout the globe. Both WannaCry and Petya caused operational processes inoperable by targeting critical infrastructure. It is quite impossible for anti-virus applications using traditional signature-based methods to detect this type of malware because they have different characteristics on each contaminated computer. The most important feature of this type of malware is that they change their contents using their mutation engines to create another hash representation of the executable file as they propagate from one computer to another. To overcome this method that attackers use to camouflage malware, we have created three-channel image files of malicious software. Attackers make different variants of the same software because they modify the contents of the malware. In the solution to this problem, we created variants of the images by applying data augmentation methods. This article aims to provide an image augmentation enhanced deep convolutional neural network (CNN) models for detecting malware families in a metamorphic malware environment. The main contributions of the article consist of three components, including image generation from malware samples, image augmentation, and the last one is classifying the malware families by using a CNN model. In the first component, the collected malware samples are converted into binary file to 3-channel images using the windowing technique. The second component of the system create the augmented version of the images, and the last part builds a classification model. This study uses five different deep CNN model for malware family detection. The results obtained by the classifier demonstrate accuracy up to 98%, which is quite satisfactory.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | PeerJ Computer Science |
| Volume | 7 |
| Pagination | e346 |
| Date Published | 01/2021 |
| Publisher | PeerJ |
| ISSN | 2376-5992 |
| Keywords | convolutional neural networks, Cybersecurity, Image augmentation, Malware analysis |
| URL | https://doi.org/10.7717/peerj-cs.346 |
| DOI | 10.7717/peerj-cs.346 |
Testing Self-Healing Cyber-Physical Systems under Uncertainty with Reinforcement Learning: An Empirical Study
Empirical Software Engineering (2021).Status: Accepted
Testing Self-Healing Cyber-Physical Systems under Uncertainty with Reinforcement Learning: An Empirical Study
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | Empirical Software Engineering |
| Publisher | Springer |
Proceedings, refereed
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
In IEEE International Conference on Software Testing, Verification and Validation (ICST)). IEEE, 2021.Status: Accepted
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | IEEE International Conference on Software Testing, Verification and Validation (ICST)) |
| Publisher | IEEE |
Anomaly Detection with Digital Twin in Cyber-Physical Systems
In IEEE International Conference on Software Testing, 2021.Status: Accepted
Anomaly Detection with Digital Twin in Cyber-Physical Systems
Cyber-Physical Systems (CPSs) are susceptible to various anomalies during their operations. Thus, it is important to detect such anomalies. Detecting such anomalies is challenging since it is uncertain when and where anomalies can happen. To this end, we present a novel approach called Anomaly deTection with digiTAl twIN (ATTAIN), which continuously and automatically builds a digital twin with live data obtained from a CPS for anomaly detection. ATTAIN builds a Timed Automaton Machine (TAM) as the digital representation of the CPS, and implements a Generative Adversarial Network (GAN) to detect anomalies. GAN uses a GCN-LSTM-based module as a generator, which can capture temporal and spatial characteristics of the input data and learn to produce realistic unlabeled fake samples. TAM labels these fake samples, which are then fed into a discriminator along with real labeled samples. After training, the discriminator is capable of distinguishing anomalous data from normal data with a high F1 score. To evaluate our approach, we used three publicly available datasets collected from three CPS testbeds. Evaluation results show that ATTAIN improved the performance of two state-of-art anomaly detection methods by 2.413\%, 8.487\% and 5.438\% on average on the three datasets, respectively. Moreover, ATTAIN achieved on average 8.39\% increase in the anomaly detection capability with digital twins as compared with an approach of not using digital twins.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | IEEE International Conference on Software Testing |
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
In th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation. LNCS, 2021.Status: Accepted
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation |
| Publisher | LNCS |
Journal Article
Uncertainty-wise Requirements Prioritization with Search
ACM Transactions on Software Engineering and Methodology 30, no. 1 (2020): 1-54.Status: Published
Uncertainty-wise Requirements Prioritization with Search
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Software Engineering and Methodology |
| Volume | 30 |
| Issue | 1 |
| Pagination | 1-54 |
| Date Published | December 2020 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 1049-331X |
| URL | https://doi.org/10.1145/3408301 |
| DOI | 10.1145/3408301 |
Pattern-based Interactive Configuration Derivation for Cyber-Physical System Product Lines
ACM Transactions on Cyber-Physical Systems 4, no. 4 (2020): 24.Status: Published
Pattern-based Interactive Configuration Derivation for Cyber-Physical System Product Lines
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Cyber-Physical Systems |
| Volume | 4 |
| Issue | 4 |
| Pagination | 24 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 2378-962X |
| URL | https://doi.org/10.1145/3389397 |
| DOI | 10.1145/3389397 |
Quality Indicators in Search-based Software Engineering
ACM Transactions on Software Engineering and Methodology 29, no. 2 (2020): 1-29.Status: Published
Quality Indicators in Search-based Software Engineering
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Software Engineering and Methodology |
| Volume | 29 |
| Issue | 2 |
| Pagination | 1 - 29 |
| Date Published | May-04-2020 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 1049-331X |
| URL | https://dl.acm.org/doi/10.1145/3375636 |
| DOI | 10.1145/3375636 |
Master's thesis
Assessing Instability caused by Multiple Parameters of Automotive Multi-product Lines with Search Algorithms
In UiT The Arctic University of Norway. UiT The Arctic University of Norway, 2020.Status: Published
Assessing Instability caused by Multiple Parameters of Automotive Multi-product Lines with Search Algorithms
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Master's thesis |
| Year of Publication | 2020 |
| Degree awarding institution | UiT The Arctic University of Norway |
| Publisher | UiT The Arctic University of Norway |
| URL | https://europe.wiseflow.net/participant/display.php?id=1923034&licenseId=6 |
Publications
Proceedings, refereed
Anomaly Detection with Digital Twin in Cyber-Physical Systems
In IEEE International Conference on Software Testing, 2021.Status: Accepted
Anomaly Detection with Digital Twin in Cyber-Physical Systems
Cyber-Physical Systems (CPSs) are susceptible to various anomalies during their operations. Thus, it is important to detect such anomalies. Detecting such anomalies is challenging since it is uncertain when and where anomalies can happen. To this end, we present a novel approach called Anomaly deTection with digiTAl twIN (ATTAIN), which continuously and automatically builds a digital twin with live data obtained from a CPS for anomaly detection. ATTAIN builds a Timed Automaton Machine (TAM) as the digital representation of the CPS, and implements a Generative Adversarial Network (GAN) to detect anomalies. GAN uses a GCN-LSTM-based module as a generator, which can capture temporal and spatial characteristics of the input data and learn to produce realistic unlabeled fake samples. TAM labels these fake samples, which are then fed into a discriminator along with real labeled samples. After training, the discriminator is capable of distinguishing anomalous data from normal data with a high F1 score. To evaluate our approach, we used three publicly available datasets collected from three CPS testbeds. Evaluation results show that ATTAIN improved the performance of two state-of-art anomaly detection methods by 2.413\%, 8.487\% and 5.438\% on average on the three datasets, respectively. Moreover, ATTAIN achieved on average 8.39\% increase in the anomaly detection capability with digital twins as compared with an approach of not using digital twins.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | IEEE International Conference on Software Testing |
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
In IEEE International Conference on Software Testing, Verification and Validation (ICST)). IEEE, 2021.Status: Accepted
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | IEEE International Conference on Software Testing, Verification and Validation (ICST)) |
| Publisher | IEEE |
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
In th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation. LNCS, 2021.Status: Accepted
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation |
| Publisher | LNCS |
Journal Article
Testing Self-Healing Cyber-Physical Systems under Uncertainty with Reinforcement Learning: An Empirical Study
Empirical Software Engineering (2021).Status: Accepted
Testing Self-Healing Cyber-Physical Systems under Uncertainty with Reinforcement Learning: An Empirical Study
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | Empirical Software Engineering |
| Publisher | Springer |
Journal Article
A framework for automated multi‑stage and multi‑step product confguration of cyber‑physical systems
Software and Systems Modeling (SoSym) 19, no. 4 (2020): 1-55.Status: Published
A framework for automated multi‑stage and multi‑step product confguration of cyber‑physical systems
Product line engineering (PLE) has been employed to large-scale cyber-physical systems (CPSs) to provide customization based on users’ needs. A PLE methodology can be characterized by its support for capturing and managing the abstractions as commonalities and variabilities and the automation of the confguration process for efective selection and customization of reusable artifacts. The automation of a confguration process heavily relies on the captured abstractions and formally specifed constraints using a well-defned modeling methodology. Based on the results of our previous work and a thorough literature review, in this paper, we propose a conceptual framework to support multi-stage and multi-step automated product confguration of CPSs, including a comprehensive classifcation of constraints and a list of automated functionalities of a CPS confguration solution. Such a framework can serve as a guide for researchers and practitioners to evaluate an existing CPS PLE solution or devise a novel CPS PLE solution. To validate the framework, we conducted three real-world case studies. Results show that the framework fulflls all the requirements of the case studies in terms of capturing and managing variabilities and constraints. Results of the literature review indicate that the framework covers all the functionalities concerned by the literature, suggesting that the framework is complete for enabling the maximum automation of confguration in CPS PLE.
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | Software and Systems Modeling (SoSym) |
| Volume | 19 |
| Issue | 4 |
| Pagination | 1-55 |
| Date Published | 06/2020 |
| Publisher | springer |
| Keywords | Automated configuration, Constraint classification, Cyber-Physical Systems, Multi-stage and multi-step configuration process, Product Line Engineering, Real-world case studies, Variability Modeling |
| DOI | 10.1007/s10270-020-00803-8 |
Pattern-based Interactive Configuration Derivation for Cyber-Physical System Product Lines
ACM Transactions on Cyber-Physical Systems 4, no. 4 (2020): 24.Status: Published
Pattern-based Interactive Configuration Derivation for Cyber-Physical System Product Lines
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Cyber-Physical Systems |
| Volume | 4 |
| Issue | 4 |
| Pagination | 24 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 2378-962X |
| URL | https://doi.org/10.1145/3389397 |
| DOI | 10.1145/3389397 |
Uncertainty-wise Requirements Prioritization with Search
ACM Transactions on Software Engineering and Methodology 30, no. 1 (2020): 1-54.Status: Published
Uncertainty-wise Requirements Prioritization with Search
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Software Engineering and Methodology |
| Volume | 30 |
| Issue | 1 |
| Pagination | 1-54 |
| Date Published | December 2020 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 1049-331X |
| URL | https://doi.org/10.1145/3408301 |
| DOI | 10.1145/3408301 |
Proceedings, refereed
Do Quality Indicators Prefer Particular Multi-Objective Search Algorithms in Search-Based Software Engineering?
In 12th Symposium on Search-Based Software Engineering. LNCS, 2020.Status: Published
Do Quality Indicators Prefer Particular Multi-Objective Search Algorithms in Search-Based Software Engineering?
In Search-Based Software Engineering (SBSE), users typically select a set of Multi-Objective Search Algorithms (MOSAs) for their experiments without any justification, or they simply choose an MOSA because of its popularity (e.g., NSGA-II). On the other hand, users know certain characteristics of solutions they are interested in. Such characteristics are typically measured with Quality Indicators (QIs) that are commonly used to evaluate the quality of solutions produced by an MOSA. Consequently, these QIs are often employed to empirically evaluate a set of MOSAs for a particular search problem to find the best MOSA. Thus, to guide SBSE users in choosing an MOSA that represents the solutions measured by a specific QI they are interested in, we present an empirical evaluation with a set of SBSE problems to study the relationships among commonly used QIs and MOSAs in SBSE. Our aim, by studying such relationships, is to identify whether there are certain characteristics of a QI because of which it prefers a certain MOSA. Such preferences are then used to provide insights and suggestions to SBSE users in selecting an MOSA, given that they know which quality aspects of solutions they are looking for.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | 12th Symposium on Search-Based Software Engineering |
| Publisher | LNCS |
Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems
In IEEE International Conference on Software Testing, Verification and Validation (ICST) 2020. IEEE, 2020.Status: Published
Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | IEEE International Conference on Software Testing, Verification and Validation (ICST) 2020 |
| Pagination | 375-386 |
| Publisher | IEEE |
Modeling Quantum Programs: Challenges, Initial Results, and Research Directions
In 1st International Workshop on Architectures and Paradigms for Engineering Quantum Software (APEQS). ACM, 2020.Status: Published
Modeling Quantum Programs: Challenges, Initial Results, and Research Directions
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | 1st International Workshop on Architectures and Paradigms for Engineering Quantum Software (APEQS) |
| Publisher | ACM |
Simultaneously Searching and Solving Multiple Avoidable Collisions for Testing Autonomous Driving Systems
In The Genetic and Evolutionary Computation Conference (GECCO). ACM, 2020.Status: Published
Simultaneously Searching and Solving Multiple Avoidable Collisions for Testing Autonomous Driving Systems
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | The Genetic and Evolutionary Computation Conference (GECCO) |
| Publisher | ACM |
Talks, contributed
Empowering Model-based Engineering with the CynefinFramework for Systematic Uncertainty Thinking
In 1st Uncertainty in Modeling Workshop 2020, Co-located with the ACM/IEEE 213rd International Conference on Model Driven Engineering Languages and Systems, 2020.Status: Published
Empowering Model-based Engineering with the CynefinFramework for Systematic Uncertainty Thinking
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | 1st Uncertainty in Modeling Workshop 2020, Co-located with the ACM/IEEE 213rd International Conference on Model Driven Engineering Languages and Systems |
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
In 5th Workshop on Monitoring and Testing of Cyber-Physical Systems (MT-CPS), 2020.Status: Published
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | 5th Workshop on Monitoring and Testing of Cyber-Physical Systems (MT-CPS) |
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
In IEEE International Conference on Software Testing, Verification and Validation (ICST 2020), 2020.Status: Published
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | IEEE International Conference on Software Testing, Verification and Validation (ICST 2020) |
| Type of Talk | Journal First Presentation |
Specifying Uncertainty in Use Case Models
In International Conference on Model-Driven Engineering and Software Development (MODELSWARD 2020). INSTICC , 2020.Status: Published
Specifying Uncertainty in Use Case Models
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | International Conference on Model-Driven Engineering and Software Development (MODELSWARD 2020) |
| Publisher | INSTICC |
Technical reports
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
Simula Research Laboratory, 2020.Status: Published
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
It is a well-recognized fact that a Cyber-Physical System (CPS) experiences uncertain (including unknown) situations during their operations. Some of such uncertainties could potentially lead to failures of CPS operations. Factors contribute to such uncertainties include 1) the intrinsically unpredictable physical environment of a CPS, 2) the use of communication networks continuously experiencing problems (e.g., slower connection than expected), and 3) the increasing use of machine learning algorithms in CPSs which introduce inherent uncertainties to these CPSs. No matter how meticulously a CPS is designed and developed, it is impossible to predict all possible uncertain situations it will experience during its operation. Thus, there is a need for new methods for discovering and handling uncertain situations during the CPS operation to prevent it from failure. In this paper, we present our ideas on how digital twins, i.e., "live models" of CPSs can help in discovering and handling potentially unsafe situations during its operation. We present the research challenges and potential solutions to develop, deploy, and operate such digital twins. The presented work is planned to be performed in a recently accepted European research-based innovation project, ADEPTNESS.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Technical reports |
| Year of Publication | 2020 |
| Publisher | Simula Research Laboratory |
| Other Numbers | 2020-01 |
Talks, invited
Testing Quantum Programs
In Oslo Metropolitan University, Oslo, Norway, 2020.Status: Published
Testing Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Talks, invited |
| Year of Publication | 2020 |
| Location of Talk | Oslo Metropolitan University, Oslo, Norway |
Proceedings, refereed
Big data from the cloud to the edge: the aggregate computing solution
In Proceedings of the 13th European Conference on Software Architecture. Vol. 2. New York, NY, USA: ACM, 2019.Status: Published
Big data from the cloud to the edge: the aggregate computing solution
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | Proceedings of the 13th European Conference on Software Architecture |
| Volume | 2 |
| Pagination | 177-180 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| URL | https://dl.acm.org/citation.cfm?id=3344988&dl=ACM&coll=DL |
| DOI | 10.1145/3344948.3344988 |
Experiences of studying Attention through EEG in the Context of Review Tasks
In EASE '19: Proceedings of the Evaluation and Assessment on Software Engineering. New York, NY, USA: ACM Press, 2019.Status: Published
Experiences of studying Attention through EEG in the Context of Review Tasks
Context: Electroencephalograms (EEG) have been used in a few cases in the context of software engineering (SE). EEGs allow capturing emotions and cognitive functioning. Such human factors have already shown to be important to understand software engineering tasks. Therefore, it is essential to gain experience in the community to utilize EEG as a research tool. Objective: To report experiences of using EEG in the context of a software engineering education (review of master theses proposals). We provide our reflections and lessons learned of (1) how to plan an EEG study, (2) how to conduct and execute (e.g., tools), (3) how to analyze. Method: We carried out an experiment using an EEG headset to measure the participants' attention rate. The experiment task includes reviewing three master thesis project plans. Results: We describe how we evolved our understanding of experimentation practices to collect and analyze psychological and cognitive data. We also provide a set of lessons learned regarding the application of EEG technology for research. Conclusions: We believe that that EEG could benefit software engineering research to collect cognitive information under certain conditions. The lessons learned reported here should be used as inputs for future experiments in software engineering, where human aspects are of interest.
| Afilliation | Software Engineering |
| Project(s) | Department of IT Management, Simula Metropolitan Center for Digital Engineering |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | EASE '19: Proceedings of the Evaluation and Assessment on Software Engineering |
| Pagination | 313-318 |
| Publisher | ACM Press |
| Place Published | New York, NY, USA |
| ISBN Number | 9781450371452 |
| Keywords | attention, experiment, human subjects, lectroencephalogram |
| URL | http://dl.acm.org/citation.cfm?doid=3319008http://dl.acm.org/citation.cf... |
| DOI | 10.1145/331900810.1145/3319008.3319357 |
Stability Analysis for Safety of Automotive Multi-Product Lines: A Search-Based Approach
In The Genetic and Evolutionary Computation Conference (GECCO). ACM, 2019.Status: Published
Stability Analysis for Safety of Automotive Multi-Product Lines: A Search-Based Approach
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | The Genetic and Evolutionary Computation Conference (GECCO) |
| Pagination | 1241-1249 |
| Publisher | ACM |
Towards a Framework for the Analysis of Multi-Product Lines in the Automotive Domain
In Proceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems. New York, NY, USA: ACM, 2019.Status: Published
Towards a Framework for the Analysis of Multi-Product Lines in the Automotive Domain
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | Proceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems |
| Publisher | ACM |
| Place Published | New York, NY, USA |
Talks, invited
Digital Twins for Cyber-Physical Systems, and Quantum Software Engineering: Research Agenda
In National Institute of Informatics, Tokyo, Japan, 2019.Status: Published
Digital Twins for Cyber-Physical Systems, and Quantum Software Engineering: Research Agenda
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Talks, invited |
| Year of Publication | 2019 |
| Location of Talk | National Institute of Informatics, Tokyo, Japan |
| Type of Talk | Colloquium |
Practical Cyber-Physical Systems Testing with Artificial Intelligence Techniques
In Artificial Intelligence Lab, Oslo Metropolitan University, Norway, 2019.Status: Published
Practical Cyber-Physical Systems Testing with Artificial Intelligence Techniques
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Talks, invited |
| Year of Publication | 2019 |
| Location of Talk | Artificial Intelligence Lab, Oslo Metropolitan University, Norway |
Uncertainty in Requirements
In Nanjing University of Aeronautics and Astronautics, Nanjing, China, 2019.Status: Published
Uncertainty in Requirements
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Talks, invited |
| Year of Publication | 2019 |
| Location of Talk | Nanjing University of Aeronautics and Astronautics, Nanjing, China |
Journal Article
Employing Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
Journal of Systems and Software 153 (2019): 86-104.Status: Published
Employing Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
Test case prioritization (TP) is widely used in regression testing for optimal reordering of test cases to achieve specific criteria (e.g., higher fault detection capability) as early as possible. In our earlier work, we proposed an approach for black-box dynamic TP using rule mining and multi-objective search (named as REMAP) by defining two objectives (fault detection capability and test case reliance score) and considering test case execution results at runtime. In this paper, we conduct an extensive empirical evaluation of REMAP by employing three different rule mining algorithms and three different multi-objective search algorithms, and we also evaluate REMAP with one additional objective (estimated execution time) for a total of 18 different configurations (i.e., 3 rule mining algorithms × 3 search algorithms × 2 different set of objectives) of REMAP. Specifically, we empirically evaluated the 18 variants of REMAP with 1) two variants of random search while using two objectives and three objectives, 2) three variants of greedy algorithm based on one objective, two objectives, and three objectives, 3) 18 variants of static search-based prioritization approaches, and 4) six variants of rule-based prioritization approaches using two industrial and three open source case studies. Results showed that the two best variants of REMAP with two objectives and three objectives significantly outperformed the best variants of competing approaches by 84.4% and 88.9%, and managed to achieve on average 14.2% and 18.8% higher Average Percentage of Faults Detected per Cost (APFDc) scores.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Journal of Systems and Software |
| Volume | 153 |
| Pagination | 86-104 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
Search-Based Test Case Implantation for Testing Untested Configurations
Information and Software Technology 111 (2019): 22-36.Status: Published
Search-Based Test Case Implantation for Testing Untested Configurations
Context: Modern large-scale software systems are highly configurable, and thus require a large number of test cases to be implemented and revised for testing a variety of system configurations. This makes testing highly configurable systems very expensive and time-consuming.
Objective: Driven by our industrial collaboration with a video conferencing company, we aim to automatically analyze and implant existing test cases (i.e., an original test suite) to test the untested configurations.
Method: We propose a search-based test case implantation approach (named as SBI) consisting of two key components: 1) Test case analyzer that statically analyzes each test case in the original test suite to obtain the program dependence graph for test case statements and 2) Test case implanter that uses multi-objective search to select suitable test cases for implantation using three operators, i.e., selection, crossover, and mutation (at the test suite level) and implants the selected test cases using a mutation operator at the test case level including three operations (i.e., addition, modification, and deletion).
Conclusion: SBI can be applied to automatically implant a test suite with the aim of testing untested configurations and thus achieving higher configuration coverage.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Information and Software Technology |
| Volume | 111 |
| Pagination | 22-36 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
| Keywords | genetic algorithms, Multi-objective optimization, Search, test case implantation |
Testing Self-Healing Cyber-Physical Systems under Uncertainty: A Fragility-Oriented Approach
Software Quality Journal 27, no. 2 (2019): 615-649.Status: Published
Testing Self-Healing Cyber-Physical Systems under Uncertainty: A Fragility-Oriented Approach
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Software Quality Journal |
| Volume | 27 |
| Issue | 2 |
| Pagination | 615–649 |
| Date Published | 03/2019 |
| Publisher | Springer |
| URL | https://link.springer.com/article/10.1007/s11219-018-9437-3 |
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
Journal of Systems and Software 153 (2019).Status: Published
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Journal of Systems and Software |
| Volume | 153 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
Using multi-objective search and machine learning to infer rules constraining product configurations
Automated Software Engineering (2019): 1-62.Status: Published
Using multi-objective search and machine learning to infer rules constraining product configurations
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Automated Software Engineering |
| Pagination | 1-62 |
| Publisher | Springer |
| DOI | 10.1007/s10515-019-00266-2 |
Talks, contributed
Enabling automated requirements reuse and configuration
In 23rd International Systems and Software Product Line Conference, Paris, France. ACM, 2019.Status: Published
Enabling automated requirements reuse and configuration
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2019 |
| Location of Talk | 23rd International Systems and Software Product Line Conference, Paris, France |
| Publisher | ACM |
| Type of Talk | Paris, France |
| DOI | 10.1145/3336294.3342370 |
Search-based test case implantation for testing untested configurations
In 34th IEEE/ACM International Conference on Automated Software Engineering (ASE 2019), San Diego (CA), USA. ACM, 2019.Status: Published
Search-based test case implantation for testing untested configurations
| Afilliation | Software Engineering |
| Project(s) | No Simula project |
| Publication Type | Talks, contributed |
| Year of Publication | 2019 |
| Location of Talk | 34th IEEE/ACM International Conference on Automated Software Engineering (ASE 2019), San Diego (CA), USA |
| Publisher | ACM |
| Type of Talk | Journal First Presentation |
Uncertainty-wise Test Case Generation and Minimization for Cyber-Physical Systems
In 34th IEEE/ACM International Conference on Automated Software Engineering (ASE 2019), San Diego (CA), USA. San Diego (CA), USA: ACM, 2019.Status: Published
Uncertainty-wise Test Case Generation and Minimization for Cyber-Physical Systems
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2019 |
| Location of Talk | 34th IEEE/ACM International Conference on Automated Software Engineering (ASE 2019), San Diego (CA), USA |
| Publisher | ACM |
| Place Published | San Diego (CA), USA |
| Type of Talk | Journal First Presentation |
Edited books
Evaluation and Assessment in Software Engineering 2019
ACM, 2019.Status: Published
Evaluation and Assessment in Software Engineering 2019
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Edited books |
| Year of Publication | 2019 |
| Publisher | ACM |
Second International Workshop on Verification and Validation of Internet of Things
IEEE, 2019.Status: Published
Second International Workshop on Verification and Validation of Internet of Things
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Edited books |
| Year of Publication | 2019 |
| Publisher | IEEE |
The 15th European Conference on Modelling Foundations and Applications (ECMFA)
2nd ed. Vol. 18. The Journal of Object Technology, 2019.Status: Published
The 15th European Conference on Modelling Foundations and Applications (ECMFA)
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Edited books |
| Year of Publication | 2019 |
| Volume | 18 |
| Edition | 2 |
| Date Published | 09/2019 |
| Publisher | The Journal of Object Technology |
| URL | http://www.jot.fm/contents/issue_2019_02.html |
| DOI | 10.5381/jot.2019.18.2.e1 |
Technical reports
Automated Test Case Implantation to Test Untested Configurations: A Cost-Effective Search-Based Approach
Simula Research Laboratory, 2018.Status: Published
Automated Test Case Implantation to Test Untested Configurations: A Cost-Effective Search-Based Approach
Modern large-scale software systems are highly configurable, and thus require a large number of test cases to be implemented and revised for testing a variety of system configurations. This makes testing highly configurable systems very expensive and time-consuming. Driven by our industrial collaboration with a video conferencing company, we aim to automatically analyze and implant existing test cases (i.e., an original test suite) to test the untested configurations. We propose a search-based test case implantation approach (named as SBI) consisting of two key components: 1) Test case analyzer that statically analyzes each test case in the original test suite to obtain the program dependence graph for test case statements and 2) Test case implanter that uses multi-objective search to select suitable test cases for implantation using three operators, i.e., selection, crossover, and mutation (at the test suite level) and implants the selected test cases using a mutation operator at the test case level including three operations (i.e., addition, modification, and deletion). We empirically evaluated SBI with an industrial case study and an open source case study by comparing the implanted test suites produced by SBI with the original test suite using evaluation metrics such as statement coverage (SC), branch coverage (BC), mutation score (MS). Results show that for both the case studies, the implanted test suites performed significantly better than the original test suites with on average 21.9% higher coverage of configuration variable values. For the open source case study, SBI managed to improve SC, BC, and MS with 4.8%, 7.5%, and 2.6%, respectively. SBI can be applied to automatically implant an existing test suite with the aim of testing untested configurations and thus achieving higher configuration coverage.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Technical reports |
| Year of Publication | 2018 |
| Publisher | Simula Research Laboratory |
| Keywords | Genetic Algorithm, Multi-objective optimization, Search, test case implantation |
| Notes | This research was supported by the Research Council of Norway (RCN) funded Certus SFI. Shuai Wang is also supported by RFF Hovedstaden funded MBE-CR project and COST Action CA15140 (ImAppNIO). Tao Yue and Shaukat Ali are supported by RCN funded Zen-Configurator project, RCN funded MBTCPS project, and COST Action IC1404. |
| URL | https://www.simula.no/sites/default/files/publications/files/automated_t... |
Employing Multi-Objective Search and Machine Learning to Mine Cross Product Line Rules: A Technical Report
Oslo, Norway: Simula Research Laboratory, 2018.Status: Published
Employing Multi-Objective Search and Machine Learning to Mine Cross Product Line Rules: A Technical Report
Modern systems are being developed by integrating multiple products within/across product lines that communicate with each other through information networks. Runtime behaviors of such systems are related to product configurations and information networks. Cost-effectively supporting Product Line Engineering (PLE) of such systems is challenging mainly because of lacking the support of automation of the configuration process. Capturing rules is the key for automating the configuration process in PLE. However, there does not exist explicitly-specified rules constraining configurable parameter values of such products and product lines. Manually specifying such rules is tedious and time-consuming. To address this challenge, in this paper, we present an improved version (named as SBRM+) of our previously proposed Search-based Rule Mining (SBRM) approach. SBRM+ incorporates two machine learning algorithms (i.e., C4.5 and PART) and two multi-objective search algorithms (i.e., NSGA-II and NSGA-III), employs a clustering algorithm (i.e., k-means) for classifying rules as high or low confidence rules, which are used for defining three objectives to guide the search. To evaluate SBRM+ (i.e., SBRM+NSGA-II-C45, SBRM+NSGA-III-C45, SBRM+NSGA-II-PART, and SBRM+NSGA-III-PART), we performed two case studies (Cisco and Jitsi) and conducted three types of analyses of results: difference analysis, correlation analysis, and trend analysis. Results of the analyses show that all the SBRM+ approaches performed significantly better than two Random Search-based approaches (RBRM+-C45 and RBRM+-PART) in terms of fitness values, six quality indicators, and 17 machine learning quality measurements (MLQMs). As compared to RBRM+ approaches, SBRM+ approaches have improved the quality of rules based on MLQMs up to 27% for the Cisco case study and 28% for the Jitsi case study.
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Technical reports |
| Year of Publication | 2018 |
| Pagination | 1-59 |
| Date Published | 05/2018 |
| Publisher | Simula Research Laboratory |
| Place Published | Oslo, Norway |
| Keywords | Configuration, Interacting Products, Machine learning, Multi-Objective Search, Product Line, Rule Mining |
| Notes | This work was supported by the Zen-Configurator project funded by the Research Council of Norway (Grant No. 240024/F20) under the category of Young Research Talents of the FRIPO funding scheme. Tao Yue and Shaukat Ali are also supported by the Co-evolver project funded by the Research Council of Norway (grant no. 286898/LIS) under the category of Researcher Projects of the FRIPO funding scheme. |
| URL | https://www.simula.no/sites/default/files/publications/files/employing_m... |
Employing Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
Simula Research Laboratory, 2018.Status: Published
Employing Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
Test case prioritization (TP) is widely used in regression testing for optimal reordering of test cases to achieve specific criteria (e.g., higher fault detection capability) as early as possible. In our earlier work, we proposed an approach for black-box dynamic TP using rule mining and multi-objective search (named as REMAP) by defining two objectives (fault detection capability and test case reliance score) by considering test case execution results at runtime. In this paper, we conduct an extensive empirical evaluation of REMAP by employing three different rule mining algorithms and three different multi-objective search algorithms, and we also evaluate REMAP with one additional objective (estimated execution time) for a total of 18 different configurations of REMAP. Specifically, we empirically evaluated the 18 variants of REMAP with 1) two variants of random search while using two objectives and three objectives, 2) three variants of greedy algorithm based on one objective, two objectives, and three objectives, 3) 18 variants of static search-based prioritization approaches, and 4) six variants of rule-based prioritization approaches using two industrial and three open source case studies. Results showed that the two best variants of REMAP with two objectives and three objectives significantly outperformed the best variants of competing approaches by 84.4% and 88.9%, and managed to achieve on average 14.2% and 18.8% higher Average Percentage of Faults Detected per Cost (APFDc) scores.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Technical reports |
| Year of Publication | 2018 |
| Publisher | Simula Research Laboratory |
| Keywords | Black-box regression testing, Dynamic test case prioritization, Multi-objective optimization, Rule Mining, Search |
| Notes | This research was supported by the Research Council of Norway (RCN) funded Certus SFI (grant no. 203461/O30). Tao Yue and Shaukat Ali are also supported by RCN funded Zen-Configurator project (grant no. 240024/F20) and RCN funded MBT4CPS project. |
| URL | https://www.simula.no/sites/default/files/publications/files/employing_r... |
Journal Article
CBGA-ES+: A Cluster-Based Genetic Algorithm with Non-Dominated Elitist Selection for Supporting Multi-Objective Test Optimization
IEEE Transactions on Software Engineering (2018).Status: Published
CBGA-ES+: A Cluster-Based Genetic Algorithm with Non-Dominated Elitist Selection for Supporting Multi-Objective Test Optimization
Many real-world test optimization problems (e.g., test case prioritization) are multi-objective intrinsically and can be tackled using various multi-objective search algorithms (e.g., Non-dominated Sorting Genetic Algorithm (NSGA-II)). However, existing multi-objective search algorithms have certain randomness when selecting parent solutions for producing offspring solutions. In a worse case, suboptimal parent solutions may result in offspring solutions with bad quality, and thus affect the overall quality of the solutions in the next generation. To address such a challenge, we propose CBGA-ES+, a novel cluster-based genetic algorithm with non-dominated elitist selection to reduce the randomness when selecting the parent solutions to support multi-objective test optimization. We empirically compared CBGA-ES+ with random search and greedy (as baselines), four commonly used multi-objective search algorithms (i.e., Multi-objective Cellular genetic algorithm (MOCell), NSGA-II, Pareto Archived Evolution Strategy (PAES), and Strength Pareto Evolutionary Algorithm (SPEA2)), and the predecessor of CBGA-ES+ (named CBGA-ES) using five multi-objective test optimization problems with eight subjects (two industrial, one real world, and five open source). The results showed that CBGA-ES+ managed to significantly outperform the selected search algorithms for a majority of the experiments. Moreover, for the solutions in the same search space, CBGA-ES+ managed to perform better than CBGA-ES, MOCell, NSGA-II, PAES, and SPEA2 for 2.2%, 13.6%, 14.5%, 17.4%, and 9.9%, respectively. Regarding the running time of the algorithm, CBGA-ES+ was faster than CBGA-ES for all the experiments.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | IEEE Transactions on Software Engineering |
| Publisher | IEEE |
| ISSN | 0098-5589 |
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
Software and Systems Modeling (2018): 1-31.Status: Published
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
Self-healing Cyber-Physical Systems (SH-CPSs) detect and recover from faults by themselves at runtime. Testing such systems is challenging due to the complex implementation of self-healing behaviors and their interaction with the physical environment, both of which are uncertain. To this end, we propose an executable model-based approach to test self-healing behaviors under environmental uncertainties. The approach consists of a Modeling Framework of SH-CPSs (MoSH) and an accompanying Test Model Executor (TM-Executor). MoSH provides a set of modeling constructs and a methodology to specify executable test models, which capture expected system behaviors and environmental uncertainties. TM-Executor executes the test models together with the systems under test, to dynamically test their self-healing behaviors under uncertainties. We demonstrated the successful application of MoSH to specify 11 self-healing behaviors and 17 uncertainties for three SH-CPSs. The time spent by TM-Executor to perform testing activities was in the order of milliseconds, though the time spent was strongly correlated with the complexity of test models.
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | Software and Systems Modeling |
| Pagination | 1-31 |
| Date Published | 11/2018 |
| Publisher | Springer |
| Place Published | Berlin Heidelberg |
| ISSN | 1619-1374 |
| DOI | 10.1007/s10270-018-00703-y |
Specifying Uncertainty in Use Case Models
Journal of Systems and Software 144 (2018): 573-603.Status: Published
Specifying Uncertainty in Use Case Models
Context: Latent uncertainty in the context of software-intensive systems (e.g., Cyber-Physical Systems (CPSs)) demands explicit attention right from the start of development. Use case modeling—a commonly used method for specifying requirements in practice, should also be extended for explicitly specifying uncertainty.
Objective: Since uncertainty is a common phenomenon in requirements engineering, it is best to address it explicitly by identifying, qualifying, and, where possible, quantifying uncertainty at the beginning stage. The ultimate aim, though not within the scope of this paper, was to use these use cases as the starting point to create test-ready models to support automated testing of CPSs under uncertainty.
Method: We extend the Restricted Use Case Modeling (RUCM) methodology and its supporting tool to specify uncertainty as part of system requirements. Such uncertainties include those caused by insufficient domain expertise of stakeholders, disagreements among them, and known uncertainties about assumptions about the environment of the system. The extended RUCM, called U-RUCM, inherits the features of RUCM, such as automated analyses and generation of models, to mention but a few. Consequently, U-RUCM provides all the key benefits offered by RUCM (i.e., reducing ambiguities in requirements), but also, it allows specification of uncertainties with the possibilities of reasoning and refining existing ones and even uncovering unknown ones.
Results: We evaluated U-RUCM with two industrial CPS case studies. After refining RUCM models (specifying initial requirements), by applying the U-RUCM methodology, we successfully identified and specified additional 306% and 512% (previously unknown) uncertainty requirements, as compared to the initial requirements specified in RUCM. This showed that, with U-RUCM, we were able to get a significantly better and more precise characterization of uncertainties in requirement engineering.
Conclusion: Evaluation results show that U-RUCM is an effective methodology (with tool support) for dealing with uncertainty in requirements engineering. We present our experience, lessons learned, and future challenges, based on the two industrial case studies.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | Journal of Systems and Software |
| Volume | 144 |
| Pagination | 573-603 |
| Publisher | Elsevier |
| Keywords | Belief, Uncertainty, Use Case Modeling |
| DOI | 10.1016/j.jss.2018.06.075 |
Edited books
Editorial to the Theme Issue on Model-based Testing
Software & Systems Modeling: Springer, 2018.Status: Published
Editorial to the Theme Issue on Model-based Testing
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Edited books |
| Year of Publication | 2018 |
| Publisher | Springer |
| Place Published | Software & Systems Modeling |
First International Workshop on Verification and Validation of Internet of Things
IEEE, 2018.Status: Published
First International Workshop on Verification and Validation of Internet of Things
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Edited books |
| Year of Publication | 2018 |
| Publisher | IEEE |
Proceedings, refereed
Model- Based Personalized Visualization System for Monitoring Evolving Industrial Cyber-Physical System
In The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) . IEEE, 2018.Status: Published
Model- Based Personalized Visualization System for Monitoring Evolving Industrial Cyber-Physical System
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2018 |
| Conference Name | The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) |
| Publisher | IEEE |
REMAP: Using Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
In 11th IEEE Conference on Software Testing, Validation and Verification (ICST). IEEE, 2018.Status: Published
REMAP: Using Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
Test case prioritization (TP) prioritizes test cases into an optimal order for achieving specific criteria (e.g., higher fault detection capability) as early as possible. However, the existing TP techniques usually only produce a static test case order before the execution without taking runtime test case execution results into account. In this paper, we propose an approach for dynamic TP using rule mining and multi-objective search (named as REMAP). REMAP has three key components: 1) Rule Miner, which mines execution relations among test cases from historical execution data; 2) Static Prioritizer, which defines two objectives (i.e., fault detection capability (FDC) and test case reliance score (TRS)) and applies multi-objective search to prioritize test cases statically; and 3) Dynamic Executor and Prioritizer, which executes statically-prioritized test cases and dynamically updates the test case order based on the runtime test case execution results. We empirically evaluated REMAP with random search, greedy based on FDC, greedy based on FDC and TRS, static search-based prioritization, and rule-based prioritization using two industrial and three open source case studies. Results showed that REMAP significantly outperformed the other approaches for 96% of the case studies and managed to achieve on average 18% higher Average Percentage of Faults Detected (APFD).
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2018 |
| Conference Name | 11th IEEE Conference on Software Testing, Validation and Verification (ICST) |
| Publisher | IEEE |
Tool Support for Restricted Use Case Specification: Findings from a Controlled Experiment
In The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) . IEEE, 2018.Status: Published
Tool Support for Restricted Use Case Specification: Findings from a Controlled Experiment
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2018 |
| Conference Name | The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) |
| Publisher | IEEE |
Uncovering Unknown System Behaviors in Uncertain Networks with Model and Search-based Testing
In 11th IEEE Conference on Software Testing, Validation and Verification. IEEE, 2018.Status: Published
Uncovering Unknown System Behaviors in Uncertain Networks with Model and Search-based Testing
Modern software systems rely on information networks for communication. Such information networks are inherently unpredictable and unreliable. Consequently, software systems behave in an unstipulated manner in uncertain network conditions. Discovering unknown behaviors of these software systems in uncertain network conditions is essential to ensure their correct behaviors. Such discovery requires the development of systematic and automated methods. We propose an online and iterative Model-Based Testing approach to evolve test models with search algorithms. Our ultimate aim is to discover unknown expected behaviors that can only be observed in uncertain network conditions. Also, we implement an adaptive search-based test case generation strategy to generate test cases that are executed on the system under test. We evaluated our approach with an open source video conference application—Jitsi with four search algorithms. Results show that our approach was effective in discovering unknown system behaviors. In particular, (1+1) Evolutionary Algorithm outperformed the other three search algorithms including Random Search.
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2018 |
| Conference Name | 11th IEEE Conference on Software Testing, Validation and Verification |
| Publisher | IEEE |
Talk, keynote
Model-Based Testing of Cyber-Physical Systems with Machine Learning and Search Algorithms
In International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018. SCITEPRESS Digital Library, 2018.Status: Published
Model-Based Testing of Cyber-Physical Systems with Machine Learning and Search Algorithms
Testing of Cyber-Physical Systems (CPS) is manifesting unprecedented challenges due to the tremendous complexity and interdisciplinary nature of CPS. Thus, CPS testing must be driven by automated, scalable, optimised, and intelligent solutions. This keynote will focus on the key results from such CPS testing solutions that we developed in the last several years. These solutions rely on model-based testing, machine learning, and search algorithms. The results from the application of these testing solutions in industrial case studies and real-world systems will be presented together with the future research directions.
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Talk, keynote |
| Year of Publication | 2018 |
| Location of Talk | International Conference on Model-Driven Engineering and Software Development, MODELSWARD 2018 |
| Publisher | SCITEPRESS Digital Library |
Talks, invited
Model-Based Testing of Cyber-Physical Systems with Machine Learning and Search Algorithms
In DNV-GL, Trondheim, Norway, 2018.Status: Published
Model-Based Testing of Cyber-Physical Systems with Machine Learning and Search Algorithms
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Talks, invited |
| Year of Publication | 2018 |
| Location of Talk | DNV-GL, Trondheim, Norway |
Practical Cyber-Physical Systems Testing: Results and Future Directions
In The 7th Workshop of Advanced Software Engineering, Gold Coast, Australia, 2018.Status: Published
Practical Cyber-Physical Systems Testing: Results and Future Directions
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Talks, invited |
| Year of Publication | 2018 |
| Location of Talk | The 7th Workshop of Advanced Software Engineering, Gold Coast, Australia |
Talks, contributed
Presentation on Symposium on Search-Based Software Engineering (SSBSE 2019)
In Symposium on Search-Based Software Engineering (SSBSE 2018), Montpellier, France, 2018.Status: Published
Presentation on Symposium on Search-Based Software Engineering (SSBSE 2019)
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2018 |
| Location of Talk | Symposium on Search-Based Software Engineering (SSBSE 2018), Montpellier, France |
REMAP: Using Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
In IEEE Conference on Software Testing, Validation and Verification (ICST), Västerås, Sweden, 2018.Status: Published
REMAP: Using Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2018 |
| Location of Talk | IEEE Conference on Software Testing, Validation and Verification (ICST), Västerås, Sweden |
Search-based Test Optimization for Software Systems
In GECCO 2018, Kyoto, Japan, 2018.Status: Published
Search-based Test Optimization for Software Systems
Test optimization is a crucial activity to select, generate, minimize, or prioritize test cases to test software systems cost-effectively. Test optimization is a complex problem that requires often solving contradictory relationships among cost (e.g., test execution time), effectiveness (e.g., fault detection ability), and efficiency (fault detection per execution time) objectives. Search algorithms including both single objective (e.g., Genetic Algorithms) and multi-objective (e.g., NSGA-II) have been extensively used for solving a variety of test optimization problems in diverse contexts. This tutorial will provide an introduction to test optimization with both single and multi-objective search algorithms. Mainly, it will focus on a variety of real industrial test selection, test minimization, and test prioritization problems. The tutorial will focus on the following aspects:1) How to optimally encoding test optimization problems for search algorithms? 2) How to define effective fitness functions that can guide search algorithms to find optimal solutions cost-effectively? 3) How to pick a suitable search algorithm to solve test optimization problem? 4) How to select and configure proper parameter settings for the selected search algorithms? 5) How to choose appropriate quality indicators to assess the quality of solutions generated by multi-objective algorithms? 6) How to choose appropriate statistical tests to evaluate search algorithms for a particular test optimization problem empirically?
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2018 |
| Location of Talk | GECCO 2018, Kyoto, Japan |
Search-based Test Optimization: A Very Short Introduction
In ERATO MMSD Summer Camp, National Institute of Informatics, Japan, 2018.Status: Published
Search-based Test Optimization: A Very Short Introduction
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2018 |
| Location of Talk | ERATO MMSD Summer Camp, National Institute of Informatics, Japan |
Testing Cyber-Physical Systems with Machine Learning and Search Algorithms
In National Institute of Informatics, Tokyo, Japan, 2018.Status: Published
Testing Cyber-Physical Systems with Machine Learning and Search Algorithms
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2018 |
| Location of Talk | National Institute of Informatics, Tokyo, Japan |
Proceedings, refereed
A Multi-objective and Cost-Aware Optimization of Requirements Assignment
In EEE Congress on Evolutionary Computation 2017 (CEC). IEEE, 2017.Status: Published
A Multi-objective and Cost-Aware Optimization of Requirements Assignment
A typical way to improve the quality of requirements is to assign them to suitable stakeholders for reviewing. Due to different characteristics of requirements and diverse background of stakeholders, it is needed to find an optimal solution for requirements assignment. Existing search-based requirements assignment solutions focus on maximizing stakeholders' familiarities to assigned requirements and balancing the overall workload of each stakeholder. However, a cost-effective requirements assignment solution should also take into account another two optimization objectives: 1) minimizing required time for re-viewing requirements, and 2) minimizing the monetary cost required for performing reviewing tasks. We formulated the requirements assignment problem as a search problem and defined a fitness function considering all the five optimization objectives. We conducted an empirical evaluation to assess the fitness function together with six search algorithms using a real-world case study and 120 artificial problems to assess the scalability of the proposed fitness function. Results show that overall, our optimization problem is complex and further justifies the use for multi-objectives search algorithms, and the Speed-constrained Multi-Objective Particle Swarm Optimization (SMPSO) algorithm performed the best among all the search algorithms.
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | EEE Congress on Evolutionary Computation 2017 (CEC) |
| Publisher | IEEE |
A Restricted Natural Language based Use Case Modeling Methodology for Real-time Systems
In 9th Workshop on Modelling in Software Engineering (MiSE'2017). IEEE, 2017.Status: Published
A Restricted Natural Language based Use Case Modeling Methodology for Real-time Systems
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | 9th Workshop on Modelling in Software Engineering (MiSE'2017) |
| Publisher | IEEE |
An Empirical Evaluation of Mutation and Crossover Operators for Multi-Objective Uncertainty-Wise Test Minimization
In 10th International Workshop on Search-based Software Testing. IEEE, 2017.Status: Published
An Empirical Evaluation of Mutation and Crossover Operators for Multi-Objective Uncertainty-Wise Test Minimization
Multi-objective uncertainty-wise test case minimization focuses on selecting a minimum number of test cases to execute out of all available ones while maximizing effectiveness (e.g., coverage), minimizing cost (e.g., time to execute test cases), and at the same time optimizing uncertainty-related objectives. In our previous work, we developed four uncertainty-wise test case minimization strategies relying on Uncertainty Theory and multi-objective search (NSGA-II with default settings), which were evaluated with one real Cyber-Physical System (CPS) with inherent uncertainty. However, a fundamental question to answer is whether these default settings of NSGA-II are good enough to provide optimized solutions. In this direction, we report one of the preliminary empirical evaluations, where we performed an experiment with three different mutation operators and three crossover operators, i.e., in total nine combinations with NSGA-II for the four uncertainty-wise test case minimization strategies using a real CPS case study. Results show that the Blend Alpha crossover operator together with the polynomial mutation operator permits NSGA-II achieving the best performance in terms of solving our uncertainty-wise test minimization problems.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | 10th International Workshop on Search-based Software Testing |
| Publisher | IEEE |
| Keywords | Cyber-Physical Systems, Multi-Objective Search, Test Case Minimization, Uncertainty-Wise Testing |
| DOI | 10.1109/SBST.2017.9 |
CBGA-ES: A Cluster-Based Genetic Algorithm with Elitist Selection for Supporting Multi-objective Test Optimization
In 10th IEEE International Conference on Software Testing, Verification and Validation (ICST 2017). IEEE, 2017.Status: Published
CBGA-ES: A Cluster-Based Genetic Algorithm with Elitist Selection for Supporting Multi-objective Test Optimization
Multi-objective search algorithms (e.g., non- dominated sorting genetic algorithm II (NSGA-II)) have been frequently applied to address various testing problems requiring multi-objective optimization such as test case selection. However, existing multi-objective search algorithms have certain randomness when selecting parent solutions for producing offspring solutions. In the worse case, suboptimal parent solutions may result in offspring solutions with bad quality, and thus affect the overall quality of the next generation. To address such a challenge, we propose a cluster-based genetic algorithm with elitist selection (CBGA-ES) with the aim to reduce such randomness for supporting multi-objective test optimization. We empirically compared CBGA-ES with random search, greedy (as baselines) and four commonly used multi-objective search algorithms (e.g., NSGA-II) using three industrial test optimization problems, i.e., test suite minimization, test case prioritization, and test case selection. The results showed that CBGA-ES significantly outperformed the baseline algorithms (e.g., greedy), and the four selected search algorithms for all the three test optimization problems. CBGA-ES managed to outperform more than 75% of the objectives for all the four algorithms in each test optimization problem. Moreover, CBGA- ES was able to improve the quality of the solutions for an average of 32.5% for each objective as compared to the four algorithms for the three test optimization problems.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | 10th IEEE International Conference on Software Testing, Verification and Validation (ICST 2017) |
| Publisher | IEEE |
| DOI | 10.1109/ICST.2017.40 |
Empowering Testing Activities with Modeling: Achievements and Insights from Nine Years of Collaboration with Cisco
In The International Conference on Model-Driven Engineering and Software Development (MODELSWARD). SCITEPRESS – Science and Technology Publications, 2017.Status: Published
Empowering Testing Activities with Modeling: Achievements and Insights from Nine Years of Collaboration with Cisco
Research-Based Innovation (RBI) aims at bringing research-driven innovative solutions to the industrial problems identified from the industry in close collaboration with researchers. This paper focuses on presenting one such instance of RBI between Cisco Systems, Norway and the Software Engineering department of Simula Research Laboratory, Norway over the period of last nine years. The main topic of the collaboration was related to improving the current testing practice at Cisco with the use of models. We present a brief overview of various model-driven testing projects, and lessons learned from such RBI collaboration.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | The International Conference on Model-Driven Engineering and Software Development (MODELSWARD) |
| Pagination | 581-589 |
| Publisher | SCITEPRESS – Science and Technology Publications |
Fragility-Oriented Testing with Model Execution and Reinforcement Learning
In The 29th International Conference on Testing Software and Systems. LNCS, 2017.Status: Published
Fragility-Oriented Testing with Model Execution and Reinforcement Learning
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | The 29th International Conference on Testing Software and Systems |
| Publisher | LNCS |
Mining Cross Product Line Rules with Multi-Objective Search and Machine Learning
In Genetic and Evolutionary Computation Conference (GECCO) . ACM, 2017.Status: Published
Mining Cross Product Line Rules with Multi-Objective Search and Machine Learning
Nowadays, an increasing number of systems are being developed by integrating products (belonging to different products lines) that communicate with each other through information networks. Cost-effectively supporting Product Line Engineering (PLE) and in particular enabling automation of configuration in PLE is a challenge. Capturing rules is the key for enabling automation of configuration. Product configuration has a direct impact on runtime interactions of communicating products. Such products might be within or across product lines and there usually don’t exist explicitly specified rules constraining configurable parameter values of such products. Manually specifying such rules is tedious, time-consuming, and requires expert’s knowledge of the domain and the product lines. To address this challenge, we propose an approach named as SBRM that combines multi- objective search with machine learning to mine rules. To evaluate the proposed approach, we performed a real case study of two communicating Video Conferencing Systems belonging to two different product lines. Results show that SBRM performed significantly better than Random Search in terms of fitness values, Hyper-Volume, and machine learning quality measurements. When comparing with rules mined with real data, SBRM performed significantly better particularly in terms of Failed Precision (18%), Failed Recall (72%), and Failed F-measure (59%).
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | Genetic and Evolutionary Computation Conference (GECCO) |
| Publisher | ACM |
| Keywords | Rule Mining; Multi-Objective Search; Configuration; Machine Learning; Product Line. |
| DOI | 10.1145/3071178.307126 |
Product Line Engineering of Monitoring Functionality in Industrial Cyber-Physical Systems: A Domain Analysis
In The 21st International Systems and Software Product Line Conference. ACM, 2017.Status: Published
Product Line Engineering of Monitoring Functionality in Industrial Cyber-Physical Systems: A Domain Analysis
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | The 21st International Systems and Software Product Line Conference |
| Publisher | ACM |
RCIA: Automated Change Impact Analysis to Facilitate a Practical Cancer Registry System
In The International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2017.Status: Published
RCIA: Automated Change Impact Analysis to Facilitate a Practical Cancer Registry System
The Cancer Registry of Norway (CRN) employs a cancer registry system to collect cancer patient data (e.g., diagnosis and treatments) from various medical entities (e.g., clinic hospitals). The collected data are then checked for validity (i.e., validation) and assembled as cancer cases (i.e., aggregation) based on more than 1000 cancer coding rules in the system. However, it is frequent in practice that the collected cancer data changes due to various reasons (e.g., different treatments) and the cancer coding rules can also change/evolve due to new medical knowledge. Thus, such a cancer registry system requires an efficient means to automatically analyze these changes and provide consequent impacts to medical experts for further actions. This paper proposes an automated Rule-based Change Impact Analysis (CIA) approach named RCIA that includes: 1) a change classification to capture the potential changes that can occur at CRN; 2) in total 80 change impact analysis rules including 50 dependency rules and 30 impact rules; and 3) an efficient algorithm to analyze changes and produce consequent impacts. We evaluate RCIA via a case study with 12 real change sets from CRN and a conducted interview. The results showed that RCIA managed to produce 100% actual change impacts and the medical expert at CRN is quite positive to apply RCIA to facilitate their cancer registry system. We also shared a set of lessons learned based on the collaboration with CRN.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | The International Conference on Software Maintenance and Evolution (ICSME) |
| Pagination | 603-612 |
| Publisher | IEEE |
Search-based Uncertainty-wise Requirements Prioritization
In The 22nd International Conference on Engineering of Complex Computer Systems. IEEE, 2017.Status: Published
Search-based Uncertainty-wise Requirements Prioritization
To ensure the quality of requirements, a common practice, especially in critical domains, is to review requirements within a limited time and monetary budgets. A requirement with higher importance, larger number of dependencies with other requirements, and higher implementation cost should be re-viewed with the highest priority. However, requirements are inherently uncertain in terms of their impact on the requirements implementation cost. Such cost is typically estimated by stakeholders as an interval, though an exact value is often used in the literature for requirements optimization (e.g., prioritization). Such a practice, therefore, ignores uncertainty inherent in the estimation of requirements implementation cost. This paper explicitly taken into account such uncertainty for requirement prioritization and formulates four objectives for uncertainty-wise requirements prioritization with the aim of maximizing 1) the importance of requirements, 2) requirements dependencies, 3) the implementation cost of requirements, and 4) cost over-run probability. We evaluated the multi-objective search algorithm NSGA-II together with Random Search (RS) using the RALIC dataset and 19 artificial problems. Results show that NSGA-II can solve the requirements prioritization problem with a significantly better performance than RS. Moreover, NSGA-II can prioritize requirements with higher priority earlier in the prioritization sequence. For example, in the case of the RALIC dataset, the first 10% of prioritized requirements in the prioritization sequence are on average 50% better than RS in terms of prioritization effectiveness.
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | The 22nd International Conference on Engineering of Complex Computer Systems |
| Publisher | IEEE |
Technical reports
A Pilot Experiment to Assess Interactive OCL Specification in a Real Setting
Simula Research Laboratory, 2017.Status: Published
A Pilot Experiment to Assess Interactive OCL Specification in a Real Setting
The Object Constraint Language (OCL) is a formal, declarative, and side-effect free language, standardized by the Object Management Group, for specifying constraints or queries on models specified in the Unified Modeling Language (UML). OCL was designed with the aim to bridge the gap between natural language and traditional formal languages requiring a strong mathematical background to understand and apply. OCL, along with UML, have been applied in practice for various purposes such as facilitating automated model-based testing. In most of such contexts of OCL, engineers with software engineering backgrounds specify OCL constraints. However, it is still a challenge for constraint authors (e.g., medical coders) who have no such background to apply OCL for other purposes (e.g., specifying medical rules). In this direction, in our previous work, we proposed a user-interactive specification framework, named iOCL, for facilitating OCL constraint specification and validation. The aim was to ease its adoption in practice in a wider application scope. In this paper, we present a pilot experiment that was conducted to assess the practical applicability of iOCL in Cancer Registry of Norway with real users of iOCL in terms of specifying medical cancer coding rules with iOCL. Results of the pilot experiment showed that, with iOCL, time to specify OCL constraints can be significantly reduced as compared to directly specifying OCL constraints without the tool support. In addition, participants of the experiment found that iOCL is easy to use.
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Technical reports |
| Year of Publication | 2017 |
| Publisher | Simula Research Laboratory |
An Empirical Evaluation of Mutation and Crossover Operators for Multi-Objective Uncertainty-Wise Test Minimization
Simula Research Laboratory, 2017.Status: Published
An Empirical Evaluation of Mutation and Crossover Operators for Multi-Objective Uncertainty-Wise Test Minimization
Multi-objective uncertainty-wise test case minimization focuses on selecting a minimum number of test cases to execute out of all available ones while maximizing effectiveness (e.g., coverage), minimizing cost (e.g., time to execute test cases), and at the same time optimizing uncertainty-related objectives. In our previous work, we developed four uncertainty-wise test case minimization strategies relying on Uncertainty Theory and multi-objective search (NSGA-II with default settings), which were evaluated with one real Cyber-Physical System (CPS) with inherent uncertainty. However, a fundamental question to answer is whether these default settings of NSGA-II are good enough to provide optimized solutions. In this direction, we report one of the preliminary empirical evaluations, where we performed an experiment with three different mutation operators and three crossover operators, i.e., in total nine combinations with NSGA-II for the four uncertainty-wise test case minimization strategies using a real CPS case study. Results show that the Blend Alpha crossover operator together with the polynomial mutation operator permits NSGA-II achieving the best performance in terms of solving our uncertainty-wise test minimization problems.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Technical reports |
| Year of Publication | 2017 |
| Publisher | Simula Research Laboratory |
| Keywords | Cyber-Physical Systems, Multi-Objective Search, Test Case Minimization, Uncertainty-Wise Testing |
Fragility-Oriented Testing with Model Execution and Reinforcement Learning
Simula Research Laboratory, 2017.Status: Published
Fragility-Oriented Testing with Model Execution and Reinforcement Learning
Self-healing is becoming an intrinsic feature of Cyber-Physical Systems (CPSs), which allows them to recover from faults in an autonomous manner during their operation. It becomes even more challenging when testing self-healing behaviors of CPSs in the presence of environment uncertainty. Such uncertainty makes system behaviors even more unpredictable in addition to the autonomous nature of self-healing behaviors. To this end, we propose Fragility-Oriented Testing (FOT) relying on model execution and reinforcement learning to efficiently test self-healing behaviors of a CPS in the presence of environment uncertainty. To evaluate the efficiency of FOT, we compared it with random testing (RT) and coverage-based testing (CBT). Results show that FOT significantly outperformed the two baselines for 7 out of 10 experiments in terms of fault revelation. Compared with RT and CBT, FOT respectively reduces 50% and 35% test execution time to find a fault.
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , The Certus Centre (SFI) |
| Publication Type | Technical reports |
| Year of Publication | 2017 |
| Publisher | Simula Research Laboratory |
Search-based Uncertainty-wise Requirements Prioritization
Simula Research Laboratory, 2017.Status: Submitted
Search-based Uncertainty-wise Requirements Prioritization
To ensure the quality of requirements, a common practice, especially in critical domains, is to review requirements within a limited time and monetary budgets. A requirement with higher importance, larger number of dependencies with other requirements, and higher implementation cost should be reviewed with the highest priority. However, requirements are inherently uncertain in terms of their impact on the requirements implementation cost. Such cost is typically estimated by stakeholders as an interval, though an exact value is often used in the literature for requirements optimization (e.g., prioritization). Such a practice, therefore, ignores uncertainty inherent in the estimation of requirements implementation cost. This paper explicitly taken into account such uncertainty for requirement prioritization and formulates four objectives for uncertainty-wise requirements prioritization with the aim of maximizing 1) the importance of requirements, 2) requirements dependencies, 3) the implementation cost of requirements, and 4) cost overrun probability. We evaluated the multi-objective search algorithm NSGA-II together with Random Search (RS) using the RALIC dataset and 19 artificial problems. Results show that NSGA-II can solve the requirements prioritization problem with a significantly better performance than RS. Moreover, NSGA-II can prioritize requirements with higher priority earlier in the prioritization sequence. For example, in the case of the RALIC dataset, the first 10% of prioritized requirements in the prioritization sequence are on average 50% better than RS in terms of prioritization effectiveness.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Technical reports |
| Year of Publication | 2017 |
| Publisher | Simula Research Laboratory |
Uncertainty-based Test Case Generation and Minimization for Cyber-Physical Systems: A Multi-Objective Search-based Approach
Simula Research Laboratory, 2017.Status: Published
Uncertainty-based Test Case Generation and Minimization for Cyber-Physical Systems: A Multi-Objective Search-based Approach
Cyber-Physical Systems (CPSs) typically operate in highly indeterminate environmental conditions, which require the development of testing methods that must explicitly consider uncertainty in test design, test generation, and test optimization. Towards this direction, we propose uncertainty-wise test case generation and test case minimization strategies that rely on test ready models explicitly specifying subjective uncertainty. We propose two test case generation strategies and four test case minimization strategies based on Uncertainty Theory and multi-objective search. These strategies include a novel methodology for designing and introducing indeterminacy sources in the environment during test execution and a novel set of uncertainty-wise test verdicts. We performed an extensive empirical study to select the best algorithm out of eight commonly used multi-objective search algorithms, for each of the four minimization strategies, with five use cases of two industrial CPS case studies. The minimized set of test cases obtained with the best algorithm for each minimization strategy were executed on the two real CPSs. The results showed that our best test strategy managed to observe 51% more uncertainties due to unknown indeterminate behaviors of the physical environment of the CPSs as compared to the rest of the test strategies. In addition, the same test strategy managed to observed 118% more unknown uncertainties as compared to the unique number of known uncertainties
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Technical reports |
| Year of Publication | 2017 |
| Publisher | Simula Research Laboratory |
| Keywords | Cyber-Physical Systems, Multi-Objective Search, Test Case Generation, Test Case Minimization, Uncertainty |
| Notes | This work is supported by U-Test project (Testing Cyber-Physical Systems under Uncertainty). |
Uncertainty-Wise and Time-Aware Test Case Prioritization with Multi-Objective Search
Simula Research Laboratory, 2017.Status: Published
Uncertainty-Wise and Time-Aware Test Case Prioritization with Multi-Objective Search
Context: Complex systems (e.g., Cyber-Physical Systems) that interact with the real world, behave in an unstipulated manner while operating in uncertain environments. Testing such systems in uncertainty is a big challenge. Devising uncertainty-wise testing solutions can be considered as a mandate for dealing with this challenge. Though uncertainty-wise testing is gaining attention in the last few years, industry-strengthening solutions are still missing.
Objective: Our objective is to propose uncertainty-wise test case prioritization approaches that can significantly improve the cost-effectiveness of test case execution to maximize the occurrence of uncertainty.
Method: In this paper, we identified and defined twenty uncertainty-wise, search-based, multi-objective test case prioritization problems. To solve these problems with multi-objective search algorithms, we defined twenty corresponding fitness functions based on various cost, effectiveness, and uncertainty measures.
Results: We performed a large-scale empirical evaluation to evaluate the well-known multi-objective search algorithms, i.e., NSGA-II, MOCell, SPEA2 and CellDE, and Random Search (RS), with five different use cases from two industrial CPS case studies to solve the twenty prioritization problems. Results showed that all the selected search algorithms significantly outperformed RS signifying that our identified test prioritization problems are complex. When comparing search algorithms, we observed that different search algorithms performed best regarding providing the best solutions for different uncertainty-wise prioritization problems. Based on the results, we provide recommendations to select search algorithms for different prioritization problems under different time budgets.
Conclusion: This paper presented uncertainty-wise and time-aware test case prioritization approaches, which were specially developed to improve the cost and effectiveness of test case execution and at the same time maximizing the occurrence of uncertainties during CPS testing
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Technical reports |
| Year of Publication | 2017 |
| Publisher | Simula Research Laboratory |
| Keywords | Uncertainty-wise Testing; Test Case Prioritization; Multi-objective Search |
Talk, keynote
Advancing Testing Methods with the Explicit Consideration of Environmental Uncertainty
In MBSE Seminar on Uncertainty, Nanjing University, China, 2017.Status: Published
Advancing Testing Methods with the Explicit Consideration of Environmental Uncertainty
As compared with classical software/system testing, uncertainty-wise testing explicitly addresses known uncertainty about the behavior of a System Under Test (SUT), its operating environment, and interactions between the SUT and its operational environment, across all testing phases, including test design, test generation, test optimization, and test execution, with the aim to mainly achieve the following two goals. First, uncertainty-wise testing aims to ensure that the SUT deals with known uncertainty adequately. Second, uncertainty-wise testing should also be capable of learning new (previously unknown) uncertainties such that the SUT’s implementation can be improved to guard against newly learned uncertainties during its operation. The necessity to integrate uncertainty in testing is becoming imperative because of the emergence of new types of intelligent and communicating software-based systems such as Cyber-Physical Systems (CPSs). Intrinsically, such systems are exposed to uncertainty because of their interactions with highly indeterminate physical environments. In this talk, I will provide the experience of applying uncertainty-wise testing techniques that rely on modeling and search algorithms in several industrial contexts. Furthermore, the vision about this new testing paradigm and its plausible future research directions will be presented.
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Talk, keynote |
| Year of Publication | 2017 |
| Location of Talk | MBSE Seminar on Uncertainty, Nanjing University, China |
Uncertainty-Wise Testing
In Advances in Model-Based Testing (A-MOST), Tokyo, Japan, 2017.Status: Published
Uncertainty-Wise Testing
Uncertainty-Wise testing explicitly integrates known uncertainty about the behavior of the System Under Test (SUT), its operating environment, and interactions between them in test design, generation, optimization, and execution with the following two key objectives. First, to ensure that the SUT deals with known uncertainty appropriately. Second, to learn new uncertainties such that the SUT’s implementation can be improved to guard against these uncertainties, when it is operational. The necessity to integrate uncertainty in testing is becoming imperative because of the emergence of new types of intelligent and communicating software-based systems, e.g., Cyber-Physical Systems (CPSs). Intrinsically, such systems are exposed to uncertainty because of their interactions with highly indeterminate physical environment including human. On top of that, these systems are becoming more and more autonomous, i.e., making decisions themselves at runtime (e.g., self-healing and self-configuration) and thus introduce an extra layer of complexity to test these systems. This keynote first focuses on novel challenges posed by uncertainty-wise testing of CPSs. Second, it presents some research results from the applications of novel approaches for uncertainty-wise testing of CPSs that were devised based on model-based testing, search-based testing, and machine learning techniques.
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , The Certus Centre (SFI) |
| Publication Type | Talk, keynote |
| Year of Publication | 2017 |
| Location of Talk | Advances in Model-Based Testing (A-MOST), Tokyo, Japan |
Talks, contributed
An Empirical Evaluation of Mutation and Crossover Operators for Multi-Objective Uncertainty-Wise Test Minimization
In Proceedings of the IEEE/ACM 10th International Workshop on Search-Based Software Testing under the 2017 IEEE/ACM 39th International Conference on Software Engineering, Buenos Aires, Argentina. Buenos Aires, Argentina: ACM IEEE, 2017.Status: Published
An Empirical Evaluation of Mutation and Crossover Operators for Multi-Objective Uncertainty-Wise Test Minimization
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Talks, contributed |
| Year of Publication | 2017 |
| Location of Talk | Proceedings of the IEEE/ACM 10th International Workshop on Search-Based Software Testing under the 2017 IEEE/ACM 39th International Conference on Software Engineering, Buenos Aires, Argentina |
| Publisher | ACM IEEE |
| Place Published | Buenos Aires, Argentina |
Empowering Testing Activities with Modeling
In The International Conference on Model-Driven Engineering and Software Development (MODELSWARD), Porto, Portugal, 2017.Status: Published
Empowering Testing Activities with Modeling
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , The Certus Centre (SFI) |
| Publication Type | Talks, contributed |
| Year of Publication | 2017 |
| Location of Talk | The International Conference on Model-Driven Engineering and Software Development (MODELSWARD), Porto, Portugal |
Introduction to U-Test: Uncertain CPS behaviour and reliability
In Exploitation Event ULMA Handling Systems, Spain, 2017.Status: Published
Introduction to U-Test: Uncertain CPS behaviour and reliability
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Talks, contributed |
| Year of Publication | 2017 |
| Location of Talk | Exploitation Event ULMA Handling Systems, Spain |
Uncertainty modeling (UM) – Progress Summary
In OMG Technical Meeting at Brussels, Belgium, 2017.Status: Published
Uncertainty modeling (UM) – Progress Summary
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Talks, contributed |
| Year of Publication | 2017 |
| Location of Talk | OMG Technical Meeting at Brussels, Belgium |
Journal Article
Automated Refactoring of OCL Constraints with Search
IEEE Transactions on Software Engineering (TSE) (2017).Status: Published
Automated Refactoring of OCL Constraints with Search
Object Constraint Language (OCL) constraints are typically used to provide precise semantics to models developed with the Unified Modeling Language (UML). When OCL constraints evolve regularly, it is essential that they are easy to understand and maintain. For instance, in cancer registries, to ensure the quality of cancer data, more than one thousand medical rules are defined and evolve regularly. Such rules can be specified with OCL. It is, therefore, important to ensure the understandability and maintainability of medical rules specified with OCL. To tackle such a challenge, we propose an automated search-based OCL constraint refactoring approach (SBORA) by defining and applying four semantics-preserving refactoring operators (i.e., Context Change, Swap, Split and Merge) and three OCL quality metrics (Complexity, Coupling, and Cohesion) to measure the understandability and maintainability of OCL constraints. We evaluate SBORA along with six commonly used multi-objective search algorithms (e.g., Indicator-Based Evolutionary Algorithm (IBEA)) by employing four case studies from different domains: healthcare (i.e., cancer registry system from Cancer Registry of Norway (CRN)), Oil&Gas (i.e., subsea production systems), warehouse (i.e., handling systems), and an open source case study named SEPA. Results show: 1) IBEA achieves the best performance among all the search algorithms and 2) the refactoring approach along with IBEA can manage to reduce on average 29.25% Complexity and 39% Coupling and improve 47.75% Cohesion, as compared to the original OCL constraint set from CRN. Furthermore, we conducted a controlled experiment with 96 subjects and results show that the understandability and maintainability of the original constraint set can be improved significantly from the perspectives of the 96 participants of the controlled experiment.
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | IEEE Transactions on Software Engineering (TSE) |
| Publisher | IEEE |
| URL | http://ieeexplore.ieee.org/document/8114267/ |
| DOI | 10.1109/TSE.2017.2774829 |
Enabling Automated Requirements Reuse and Configuration
Software and Systems Modeling (2017).Status: Published
Enabling Automated Requirements Reuse and Configuration
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Software and Systems Modeling |
| Publisher | Springer |
| Place Published | Berlin Heidelberg |
| ISSN | 1619-1374 |
| DOI | 10.1007/s10270-017-0641-6 |
Integrating Weight Assignment Strategies with NSGA-II for Supporting User Preference Multi-Objective Optimization
IEEE Transactions on Evolutionary Computation (TEVC) (2017).Status: Published
Integrating Weight Assignment Strategies with NSGA-II for Supporting User Preference Multi-Objective Optimization
Driven by the needs of several industrial projects on the applications of multi-objective search algorithms, we observed that user preferences must be properly incorporated into optimization objectives. However, existing algorithms usually treat all the objectives with equal priorities and do not provide a mechanism to reflect user preferences. To address this, we propose an extension—User-Preference Multi-Objective Optimization Algorithm (UPMOA), to the most commonly applied, non-dominated sorting genetic algorithm II (NSGA-II) by introducing a user preference indicator !, based on existing weight assignment strategies (e.g., Uniformly Distributed Weights (UDW)). We empirically evaluated UPMOA using four industrial problems from three diverse domains (i.e., Communication, Maritime and Subsea Oil&Gas). We also performed a sensitivity analysis for UPMOA with 625 algorithm parameter settings. To further assess the performance and scalability, 103500 artificial problems were created and evaluated representing 207 sets of user preferences. Results show that the UDW strategy with UPMOA achieves the best performance and UPMOA significantly outperformed other three multi-objective search algorithms, and has the ability to solve problems with a wide range of complexity. We also observed that different parameter settings led to the varied performance of UPMOA, thus suggesting that configuring proper parameters is highly problem-specific.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | IEEE Transactions on Evolutionary Computation (TEVC) |
| Publisher | IEEE |
| DOI | 10.1109/TEVC.2017.2778560 |
IOCL: An Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
Science of Computer Programming (SCP) 149 (2017): 3-8.Status: Published
IOCL: An Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
The Object Constraint Language (OCL) is commonly used for specifying additional constraints on models, in addition, to the ones enforced by the semantics of the models. However, a lot of practitioners and even researchers are reluctant in using OCL to some extent due to the lack of sufficient familiarity with OCL. To facilitate practitioners and researchers in specifying OCL constraints, we designed and developed a web-based tool called interactive OCL (iOCL) for interactively specifying constraints on a given model. The core idea behind iOCL is to present and display only relevant details (e.g., operations) of OCL to users at a given step of constraint specification process, in addition to helping modelers with its syntax. We evaluated iOCL using a real-world case study from Cancer Registry of Norway and the results showed that iOCL can significantly reduce the time required to specify OCL constraints and decrease the possibility of making syntactic errors during the specification process. Thus, we conclude that iOCL can facilitate the process of OCL constraint specification.
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Science of Computer Programming (SCP) |
| Volume | 149 |
| Pagination | 3-8 |
| Date Published | 08/2017 |
| Publisher | Elsevier |
Reliability-Redundancy-Location Allocation with Maximum Reliability and Minimum Cost Using Search Technique
Information and Software Techonology 82 (2017): 36-54.Status: Published
Reliability-Redundancy-Location Allocation with Maximum Reliability and Minimum Cost Using Search Technique
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Information and Software Techonology |
| Volume | 82 |
| Pagination | 36-54 |
| Publisher | Elsevier |
| ISSN | 0950-5849 |
| DOI | 10.1016/j.infsof.2016.09.010 |
Search and similarity based selection of use case scenarios: An empirical study
Empirical Software Engineering (2017): 1-78.Status: Published
Search and similarity based selection of use case scenarios: An empirical study
Use case modeling is a well-known requirements specification method and has been widely applied in practice. Use case scenarios of use case models are input elements for requirements inspection and analysis, requirements-based testing, and other downstream activities. It is, however, a practical challenge to inspect all use case scenarios that can be obtained from any non-trivial use case model, as such an inspection activity is often performed manually by domain experts. Therefore, it is needed to propose an automated solution for selecting a subset of use case scenarios with the ultimate aim of enabling cost-effective requirements (use case) inspection, analysis, and other relevant activities. Our solution is built on a natural language based, restricted use case modeling methodology (named as RUCM), in the sense that requirements specifications are specified as RUCM use case models. Use case scenarios can be automatically derived from RUCM use case models with the already established Zen-RUCM framework. In this paper, we propose a search-based and similarity-based approach called S3RCUM, through an empirical study, to select most diverse use case scenarios to enable cost-effective use case inspections. The empirical study was designed to evaluate the performance of three search algorithms together with eight similarity functions, through one real-world case study and six case studies from literature. Results show that (1+1) Evolutionary Algorithm together with Needleman-Wunsch similarity function significantly outperformed the other 31 combinations of the search algorithms and similarity functions. The combination managed to select 50% of all the generated RUCM use case scenarios for all the case studies to detect all the seeded defects.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems , U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Empirical Software Engineering |
| Pagination | 1-78 |
| Date Published | 04/2017 |
| Publisher | Springer |
Uncertainty-Wise Cyber-Physical System Test Modeling
Software & Systems Modeling (2017).Status: Published
Uncertainty-Wise Cyber-Physical System Test Modeling
It is important that a Cyber-Physical System (CPS) deals with uncertainty in its behavior caused by its unpredictable operating environment, to ensure its reliable operation. One method to ensure that the CPS will handle such uncertainty during its operation is by testing the CPS with Model-based Testing (MBT) techniques. However, existing MBT techniques do not explicitly capture uncertainty in test ready models i.e., capturing the uncertain expected behavior of a CPS in the presence of environment uncertainty. To fill this gap, we present an Uncertainty-Wise test-modeling framework, named as Uncertum, to create test ready models to support MBT of CPSs facing uncertainty. Uncertum relies on the definition of a UML profile (the UML Uncertainty Profile (UUP)) and a set of UML model libraries extending the UML profile for Modeling and Analysis of Real-Time and Embedded Systems (MARTE). Uncertum also benefits from the UML Testing Profile (UTP) V.2 to support standard-based MBT. Uncertum was evaluated with two industrial CPS case studies, one real-world case study, and one open source CPS case study from the following four perspectives: 1) Completeness and Coverage of the profiles and model libraries in terms of concepts defined in their underlying uncertainty conceptual model for CPSs (i.e., U-Model and MARTE, 2) Effort required to model uncertainty with Uncertum, and 3) Correctness of the developed test ready models, which was assessed via model execution. Based on the evaluation, we can conclude that we were successful in modeling all the uncertainties identified in the four case studies, which gives us an indication that Uncertum is sufficiently complete. In terms of modeling effort, we concluded that on average Uncertum requires18.5% more time to apply stereotypes from UUP on test ready models.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Software & Systems Modeling |
| Publisher | Springer |
| ISSN | 1619-1374 |
| Keywords | Cyber-Physical System; UML, Model-based Testing, Uncertainty |
| DOI | 10.1007/s10270-017-0609-6 |
Uncertainty-Wise Evolution of Test Ready Models
Information and Software Technology (IST) 87 (2017): 140-159.Status: Published
Uncertainty-Wise Evolution of Test Ready Models
Context: Cyber-Physical Systems (CPSs), when deployed for operation, are inherently prone to uncertainty. Considering their applications in critical domains (e.g., healthcare), it is important that such CPSs are tested sufficiently, with the explicit consideration of uncertainty. Model-based testing (MBT) involves creating test ready models capturing the expected behavior of a CPS and its operating environment. These test ready models are then used for generating executable test cases. It is, therefore, necessary to develop methods that can continuously evolve, based on real operational data collected during the operation of CPSs, test ready models and uncertainty captured in them, all together termed as Belief Test Ready Models (BMs)
Objective: Our objective is to propose a model evolution framework that can interactively improve the quality of BMs, based on operational data. Such BMs are developed by one or more test modelers (belief agents) with their assumptions about the expected behavior of a CPS, its expected physical environment, and potential future deployments. Thus, these models explicitly contain subjective uncertainty of the test modelers.
Method: We propose a framework (named as UncerTolve) for interactively evolving BMs (specified with extended UML notations) of CPSs with subjective uncertainty developed by test modelers. The key inputs of UncerTolve include initial BMs of CPSs with known subjective uncertainty and real data collected from the operation of CPSs. UncerTolve has three key features: 1) Validating the syntactic correctness and conformance of BMs against real operational data via model execution, 2) Evolving objective uncertainty measurements of BMs via model execution, and 3) Evolving state invariants (modeling test oracles) and guards of transitions (modeling constraints for test data generation) of BMs with a machine learning technique.
Results: As a proof-of-concept, we evaluated UncerTolve with one industrial CPS case study, i.e., GeoSports from the healthcare domain. Using UncerTolve, we managed to evolve 51% of belief elements, 18% of states, and 21% of transitions as compared to the initial BM developed in an industrial setting.
Conclusion: UncerTolve can successfully evolve model elements of the initial BM, in addition to objective uncertainty measurements using real operational data. The evolved model can be used to generate additional test cases covering evolved model elements and objective uncertainty. These additional test cases can be used to test the current and future deployments of a CPS to ensure that it will handle uncertainty gracefully during its operations.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Information and Software Technology (IST) |
| Volume | 87 |
| Pagination | 140-159 |
| Publisher | Elsevier |
| ISSN | 0950-5849 |
| Keywords | Belief Model, Belief Test Ready Model, Model Evolution, Model-based Testing, Uncertainty |
| DOI | 10.1016/j.infsof.2017.03.003 |
Talks, invited
Reusable Use Case and Test Case Specification Modeling
In The 16th International Conference on Software Reuse, Salvador, Brazil, 2017.Status: Published
Reusable Use Case and Test Case Specification Modeling
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , The Certus Centre (SFI) |
| Publication Type | Talks, invited |
| Year of Publication | 2017 |
| Location of Talk | The 16th International Conference on Software Reuse, Salvador, Brazil |
| Type of Talk | Tutorial |
Testing Cyber-Physical Systems under Uncertainty
In CPS Concertation Event, Brussels, Belgium, 2017.Status: Published
Testing Cyber-Physical Systems under Uncertainty
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Talks, invited |
| Year of Publication | 2017 |
| Location of Talk | CPS Concertation Event, Brussels, Belgium |
Uncertainty-wise Test Case Generation and Minimization for Cyber-Physical Systems: A Multi-Objective Search-based Approach
In National Institute of Informatics, Tokyo, Japan, 2017.Status: Published
Uncertainty-wise Test Case Generation and Minimization for Cyber-Physical Systems: A Multi-Objective Search-based Approach
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Talks, invited |
| Year of Publication | 2017 |
| Location of Talk | National Institute of Informatics, Tokyo, Japan |
Uncertainty-wise Testing of Cyber-Physical Systems
In 2017 IEEE International Symposium on Systems Engineering, Vienna, Austria, 2017.Status: Published
Uncertainty-wise Testing of Cyber-Physical Systems
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Talks, invited |
| Year of Publication | 2017 |
| Location of Talk | 2017 IEEE International Symposium on Systems Engineering, Vienna, Austria |
| Type of Talk | Tutorial |
Poster
SUnCPS: A Taxonomy of Security-related Uncertainty in Cyber-Physical Systems
International Symposium on Engineering Secure Software and Systems (ESSoS'17), Bonn, Germany, 2017.Status: Published
SUnCPS: A Taxonomy of Security-related Uncertainty in Cyber-Physical Systems
Cyber-Physical Systems (CPS) are driving the fourth industrial revolution. In CPS, uncertainty is inherent that may cause security issue either directly or indirectly. To understand security together with uncertainty, we propose SUnCPS, a taxonomy of security-related uncertainty in the context of CPS. SUnCPS can provide a structured representation of security-related uncertainty in CPS as the basis for different (early) security engineering activities for CPS such as security risk analysis and management, vulnerability/attack analysis, and security testing for CPS.
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , The Certus Centre (SFI) |
| Publication Type | Poster |
| Year of Publication | 2017 |
| Place Published | International Symposium on Engineering Secure Software and Systems (ESSoS'17), Bonn, Germany |
Uncertainty-wise and Model-based Testing of Industrial Cyber-Physical Systems
5th User Conference and Advanced Automated Testing (UCAAT), Berlin, Germany, 2017.Status: Published
Uncertainty-wise and Model-based Testing of Industrial Cyber-Physical Systems
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Poster |
| Year of Publication | 2017 |
| Place Published | 5th User Conference and Advanced Automated Testing (UCAAT), Berlin, Germany |
Miscellaneous
Uncertainty Modeling Framework Version 2
None, 2017.Status: Published
Uncertainty Modeling Framework Version 2
This a U-Test public deliverable.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Miscellaneous |
| Year of Publication | 2017 |
| Publisher | None |
Uncertainty Testing Framework V.2
None, 2017.Status: Published
Uncertainty Testing Framework V.2
This is a public deliverable for the U-Test project.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Miscellaneous |
| Year of Publication | 2017 |
| Publisher | None |
Uncertainty Testing Framework V.3
None, 2017.Status: Published
Uncertainty Testing Framework V.3
This is a public deliverable for U-Test project.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Miscellaneous |
| Year of Publication | 2017 |
| Publisher | None |
Book Chapter
Uncertainty-wise Testing of Cyber-Physical Systems
In Advances in Computers, 23-94. Vol. 107. Elsevier, 2017.Status: Published
Uncertainty-wise Testing of Cyber-Physical Systems
As compared with classical software/system testing, uncertainty-wise testing explicitly addresses known uncertainty about the behavior of a System Under Test (SUT), its operating environment, and interactions between the SUT and its operational environment, across all testing phases, including test design, test generation, test optimization, and test execution, with the aim to mainly achieve the following two goals. First, uncertainty-wise testing aims to ensure that the SUT deals with known uncertainty adequately. Second, uncertainty-wise testing should be also capable of learning new (previously unknown) uncertainties such that the SUT’s implementation can be improved to guard against newly learned uncertainties during its operation. The necessity to integrate uncertainty in testing is becoming imperative because of the emergence of new types of intelligent and communicating software-based systems such as Cyber-Physical Systems (CPSs). Intrinsically, such systems are exposed to uncertainty because of their interactions with highly indeterminate physical environments. In this chapter, we provide our understanding and experience of uncertainty-wise testing from the aspects of uncertainty-wise model-based testing, uncertainty-wise modeling and evolution of test ready models, and uncertainty-wise multi-objective test optimization, in the context of testing CPSs under uncertainty. Furthermore, we present our vision about this new testing paradigm and its plausible future research directions.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Book Chapter |
| Year of Publication | 2017 |
| Book Title | Advances in Computers |
| Volume | 107 |
| Chapter | 2 |
| Pagination | 23-94 |
| Publisher | Elsevier |
Proceedings, refereed
A Model-Based Approach with Tool Support to Facilitate the Cancer Registration Process in Cancer Registry of Norway
In European Telemedicine Conference (ETC), 2016.Status: Published
A Model-Based Approach with Tool Support to Facilitate the Cancer Registration Process in Cancer Registry of Norway
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | European Telemedicine Conference (ETC) |
A Practical Guide to Select Quality Indicators for Assessing Pareto-Based Search Algorithms in Search-Based Software Engineering
In the 38th International Conference on Software Engineering (ICSE), 2016.Status: Published
A Practical Guide to Select Quality Indicators for Assessing Pareto-Based Search Algorithms in Search-Based Software Engineering
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | the 38th International Conference on Software Engineering (ICSE) |
| Pagination | 631-642 |
A Practical Use Case Modeling Approach to Specify Crosscutting Concerns: Industrial Applications
In International Conference on Software Reuse (ICSR), 2016.Status: Published
A Practical Use Case Modeling Approach to Specify Crosscutting Concerns: Industrial Applications
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | International Conference on Software Reuse (ICSR) |
Enhancing Test Case Prioritization in an Industrial Setting with Resource Awareness and Multi-Objective Search
In The 38th International Conference on Software Engineering (ICSE), Software Engineering in Practice (SEIP) track , 2016.Status: Published
Enhancing Test Case Prioritization in an Industrial Setting with Resource Awareness and Multi-Objective Search
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | The 38th International Conference on Software Engineering (ICSE), Software Engineering in Practice (SEIP) track |
| Pagination | 182-191 |
Evaluating Variability Modeling Techniques for Supporting Cyber-Physical System Product Line Engineering
In System Analysis and Modelling (SAM) Conference. 9th ed. Vol. 9959. Saint Malo, France: Springer International Publishing, 2016.Status: Published
Evaluating Variability Modeling Techniques for Supporting Cyber-Physical System Product Line Engineering
Modern society is increasingly dependent on Cyber-Physical Systems (CPSs) in diverse domains such as aerospace, energy and healthcare. Employing Product Line Engineering (PLE) in CPSs is cost-effective in terms of reducing production cost, and achieving high productivity of a CPS development process as well as higher quality of produced CPSs. To apply CPS PLE in practice, one needs to first select an appropriate variability modeling technique (VMT), with which variabilities of a CPS Product Line (PL) can be specified. In this paper, we proposed a set of basic and CPS-specific variation point (VP) types and modeling requirements for proposing CPS-specific VMTs. Based on the proposed set of VP types (basic and CPS-specific) and modeling requirements, we evaluated four VMTs: Feature Modeling, Cardinality Based Feature Modeling, Common Variability Language, and SimPL (a variability modeling technique dedicated to CPS PLE), with a real-world case study. Evaluation results show that none of the selected VMTs can capture all the basic and CPS-specific VP and meet all the modeling requirements. Therefore, there is a need to extend existing techniques or propose new ones to satisfy all the requirements.
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | System Analysis and Modelling (SAM) Conference |
| Volume | 9959 |
| Edition | 9 |
| Pagination | 1-19 |
| Date Published | 10/2016 |
| Publisher | Springer International Publishing |
| Place Published | Saint Malo, France |
| Keywords | and Cyber- Physical Systems, Product Line Engineering, Variability Modeling |
Generating Boundary Values from OCL Constraints using Constraints Rewriting and Search Algorithms
In IEEE World Congress on Computational Intelligence, 2016.Status: Published
Generating Boundary Values from OCL Constraints using Constraints Rewriting and Search Algorithms
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | IEEE World Congress on Computational Intelligence |
iOCL: A Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
In Tool Demonstrations Track, ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MODELS), 2016.Status: Published
iOCL: A Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
The Object Constraint Language (OCL) is frequently used to specify additional constraints on models, in addition, to the ones enforced by semantics of the models. It is a well- known fact that due to the lack of familiarity with OCL, practitioners and even researcher to some extent are reluctant in using OCL. To help practitioners and researchers in writing OCL constraints for their specific problem at hand, we developed a tool called interactive OCL (iOCL) for interactively specifying constraints on a given model. The basic philosophy behind the tool is to present only those details (e.g., operations) of OCL to modelers that are valid at a given step of constraint specification process, in addition to helping modelers with its syntax. Our ultimate aim is to reduce the effort required to specify constraints, subsequently lowering down training cost and increasing the correctness of the constraints. iOCL is a web-based ap- plication that integrates other tools including Eclipse OCL for validation and evaluation of OCL constraints, and EsOCL for automatically generating valid instances of models that satisfy the specified constraints.
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | Tool Demonstrations Track, ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MODELS) |
| Pagination | 1-7 |
| Date Published | 09/2016 |
MBF4CR: A Model-Based Framework for Supporting An Automated Cancer Registry System
In 12th European Conference on Modelling Foundations and Applications (ECMFA 2016). Lecture Notes in Computer Science, Springer Verlag, 2016.Status: Published
MBF4CR: A Model-Based Framework for Supporting An Automated Cancer Registry System
The Cancer Registry of Norway (CRN) collects medical information (e.g., laboratory results, clinical procedures and treatment) of cancer patients from different medical entities, for all cancer patients in Norway. The collected data are checked for validity and correctness (i.e., validation) and is the basis for the registration of cancer cases (i.e., aggregation) by employing more than a thousand of medical rules. However, the current practice of CRN lacks of a systematic way to capture the domain knowledge and maintain medical rules at a proper level of abstraction.
To tackle these challenges, this paper proposes a model-based framework (named as MBF4CR) for capturing the domain knowledge, formalizing medi- cal rules, automating rule selection, and enabling data (cancer messages and cancer cases) validation and aggregation using Unified Modeling Language (UML) and Object Constraint Language (OCL). MBF4CR systematically cap- tures domain knowledge (e.g., cancer messages) as a UML class diagram and formally specifies medical rules as OCL constraints. By associating tags to OCL constraints, MBF4CR enables an automated rule selection process with tool support. We employed a case study from CRN that consists of 187 medical rules to evaluate MBF4CR from two aspects: Performance in terms of se- lecting and executing rules, and Correctness in terms of producing correct validation and aggregation results. Results show that MBF4CR can facilitate the current practice by complying with the medical domain knowledge with an acceptable performance, while reducing the maintenance effort.
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | 12th European Conference on Modelling Foundations and Applications (ECMFA 2016) |
| Pagination | 191-204 |
| Publisher | Lecture Notes in Computer Science, Springer Verlag |
Nonconformity Resolving Recommendations for Product Line Configuration
In IEEE International Conference on Software Testing, Verification and Validation (ICST), 2016.Status: Published
Nonconformity Resolving Recommendations for Product Line Configuration
In the context of large-scale system product line engineering, manual configuration is often mandatory and therefore inevitably introduces nonconformities: violating pre-defined constraints for conformance checking. Resolving nonconformities without proper tool support is more or less random, as there are usually hundreds and thousands of configurable parameters and conformance constraints, in the context of configuring a large-scale and directly deployable system. Moreover, inter-connections among constraints and configurable parameters worsen the feasibility of manual resolving nonconformities without proper tool support. In this paper, we present an automatic approach (named as Zen-FIX) to optimally recommend solutions to resolve nonconformities by combining multi-objective search and constraint solving techniques. Solutions recommended by Zen-FIX conform to all pre-defined constraints and are optimal in terms of maximizing the overall efficiency of an interactive product configuration process. We evaluated Zen-FIX with a real- world case study containing 52454 optimization problems, with which we evaluated seven multi-objective search algorithms. Results show that MoCell significantly outperformed all the others: CellDE, IBEA, NSGA-II, PESA2, Random, SPEA2, for most of the problems, in terms of Efficiency (a combined metric of finding optimal solutions and time performance).
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | IEEE International Conference on Software Testing, Verification and Validation (ICST) |
| Date Published | 04/2016 |
Search-Based Cost-Effective Test Case Selection within a Time Budget: An Empirical Study
In Genetic and Evolutionary Computation Conference (GECCO). Denver, Colorado, USA, 2016.Status: Published
Search-Based Cost-Effective Test Case Selection within a Time Budget: An Empirical Study
Due to limited time and resources available for execution, test case selection always remains crucial for cost-effective testing. It is even more prominent when test cases require manual steps, e.g., operating physical equipment. Thus, test case selection must consider complicated trade-offs between cost (e.g., execution time) and effectiveness (e.g., fault detection capability). Based on our industrial collaboration within the Maritime domain, we identified a real-world and multi-objective test case selection problem in the context of robustness testing, where test case execution requires human involvement in certain steps, such as turning on the power supply to a device. The high-level goal is to select test cases for execution within a given time budget, where test engineers provide weights for a set of objectives, depending on testing requirements, standards, and regulations. To address the identified test case selection problem, we defined a fitness function including one cost measure, i.e., Time Difference (TD) and three effectiveness measures, i.e., Mean Priority (MPR), Mean Probability (MPO) and Mean Consequence (MC) that were identified together with test engineers. We further empirically evaluated eight multi-objective search algorithms, which include three weight-based search algorithms (e.g., Alternating Variable Method) and five Pareto-based search algorithms (e.g., Strength Pareto Evolutionary Algorithm 2 (SPEA2)) using two weight assignment strategies (WASs). Notice that Random Search (RS) was used as a comparison baseline. We conducted two sets of empirical evaluations: 1) Using a real world case study that was developed based on our industrial collaboration; 2) Simulating the real world case study to a larger scale to assess the scalability of the search algorithms. Results show that SPEA2 with either of the WASs performed the best for both the studies. Overall, SPEA2 managed to improve on average 32.7%, 39% and 33% in terms of MPR, MPO and MC respectively as compared to RS.
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | Genetic and Evolutionary Computation Conference (GECCO) |
| Pagination | 1085-1092 |
| Date Published | 07/2016 |
| Place Published | Denver, Colorado, USA |
| Keywords | Search-Based Testing |
Search-based Decision Ordering to Facilitate Product Line Engineering of Cyber-Physical System
In International Conference on Model-Driven Engineering and Software Development. IEEE, 2016.Status: Published
Search-based Decision Ordering to Facilitate Product Line Engineering of Cyber-Physical System
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | International Conference on Model-Driven Engineering and Software Development |
| Publisher | IEEE |
STIPI: Using Search to Prioritize Test Cases based on Multi-Objectives Derived from Industrial Practice
In The 28th International Conference on Testing Software and Systems (ICTSS) . Graz, Austria: Lecture Notes in Computer Science, Springer Verlag, 2016.Status: Published
STIPI: Using Search to Prioritize Test Cases based on Multi-Objectives Derived from Industrial Practice
The importance of cost-effective test case prioritization is undeniable in automated testing practice in industry. Such prioritization typically relies on various cost and effective objectives. This paper focuses on prioritizing test cases developed to test product lines of Video Conferencing Systems (VCSs) at Cisco Systems, Norway. Each test case requires setting up configurations of a set of VCSs, invoking a set of test APIs with specific inputs, and checking the status of the VCSs. Based on these characteristics and information available about the execution of test cases (e.g., number of faults detected), we identified that the test case prioritization problem in our particular context should focus on achieving high coverage of configurations, test APIs, statuses, and high fault detection capability as fast as possible. We propose a search-based test case prioritization approach (named STIPI) to solve this problem by defining a fitness function with four objectives and integrating it with the widely applied multi-objective Non-dominated Sorting Genetic Algorithm II. We compared STIPI with random search (RS), Greedy algorithm, and three approaches adapted from literature, using three real sets of test cases from Cisco with four time budgets (25%, 50%, 75% and 100%). Results show that STIPI significantly outperformed the selected approaches and managed to achieve better performance than RS for on average 39.9%, 18.6%, 32.7% and 43.9% for the coverage of configurations, test APIs, statuses and fault detection capability, respectively.
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | The 28th International Conference on Testing Software and Systems (ICTSS) |
| Pagination | 172-190 |
| Date Published | 10/2016 |
| Publisher | Lecture Notes in Computer Science, Springer Verlag |
| Place Published | Graz, Austria |
Towards Mutation Analysis for Use Cases
In ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MODELS). ACM/IEEE, 2016.Status: Published
Towards Mutation Analysis for Use Cases
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems , MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MODELS) |
| Publisher | ACM/IEEE |
Understanding Uncertainty in Cyber-Physical Systems: A Conceptual Model
In European Conference on Modelling Foundations and Applications(ECMFA), 2016.Status: Published
Understanding Uncertainty in Cyber-Physical Systems: A Conceptual Model
Uncertainty is intrinsic in most technical systems, including Cyber-Physical Systems (CPS). Therefore, handling uncertainty in a graceful manner during the real operation of CPS is critical. Since designing, developing, and testing modern and highly sophisticated CPS is an expanding field, a step towards dealing with uncertainty is to identify, define, and classify uncertainties at various levels of CPS. This will help develop a systematic and comprehensive understanding of uncertainty. To that end, we propose a conceptual model for uncertainty specifically designed for CPS. Since the study of uncertainty in CPS development and testing is still irrelatively unexplored, this conceptual model was derived in a large part by reviewing existing work on uncertainty in other fields, including philosophy, physics, statistics, and healthcare. The conceptual model is mapped to the three logical levels of CPS: Application, Infrastructure, and Integration. It is captured using UML class diagrams, including relevant OCL constraints. To validate the conceptual model, we identified, classified, and specified uncertainties in two distinct industrial case studies
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | European Conference on Modelling Foundations and Applications(ECMFA) |
| Keywords | Cyber-Physical Systems, Testing, Uncertainty |
| Notes | This work is supported by U-Test project (Testing Cyber-Physical Systems under Uncertainty). |
Journal Article
A Systematic Test Case Selection Methodology for Product Lines: Results and Insights From an Industrial Case Study
Empirical Software Engineering 21, no. 4 (2016): 1586-1622.Status: Published
A Systematic Test Case Selection Methodology for Product Lines: Results and Insights From an Industrial Case Study
In the context of product lines, test case selection aims at obtaining a set of relevant test cases for a product from the entire set of test cases available for a product line. While working on a research-based innovation project on automated testing of product lines of Video Conferencing Systems (VCSs) developed by Cisco, we felt the need to devise a cost-effective way of selecting relevant test cases for a product. To fulfill such need, we propose a systematic and automated test selection methodology using: 1) Feature Model for Testing (FM\_T) to capture commonalities and variabilities of a product line; 2) Component Family Model for Testing (CFM\_T) to model the structure of test case repository; 3) A tool to automatically build restrictions from CFM\_T to FM\_T and traces from CFM\_T to the actual test cases. Using our methodology, a test engineer is only required to select relevant features through FM\_T at a higher level of abstraction for a product and the corresponding test cases will be obtained automatically. We evaluate our methodology by applying it to a VCS product line called Saturn with seven commercial products and the results show that our methodology can significantly reduce cost measured as test selection time and at the same time achieves higher effectiveness (feature coverage, feature pairwise coverage and fault detection) as compared with the current manual process. Moreover, we conduct a questionnaire-based study to solicit the views of test engineers who are involved in developing FM\_T and CFM\_T. The results show that test engineers are positive about adapting our methodology in their current practice. Finally, we present a set of lessons learnt while applying product line engineering at Cisco for test case selection.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2016 |
| Journal | Empirical Software Engineering |
| Volume | 21 |
| Issue | 4 |
| Pagination | 1586-1622 |
| Date Published | 08/2016 |
| Publisher | Springer |
Assessing the Quality of Industrial Avionics Software: An Extensive Empirical Evaluation
Empirical Software Engineering (2016).Status: Published
Assessing the Quality of Industrial Avionics Software: An Extensive Empirical Evaluation
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2016 |
| Journal | Empirical Software Engineering |
| Publisher | Springer |
Model-based Incremental Conformance Checking to Enable Interactive Product Configuration
Information and Software Technology 72 (2016): 68-89.Status: Published
Model-based Incremental Conformance Checking to Enable Interactive Product Configuration
Context: Model-based product line engineering (PLE) is a paradigm that can enable automated product configuration of large- scale software systems, in which models are used as an abstract specification of commonalities and variabilities of products of a product line.
Objective: In the context of PLE, providing immediate feedback on the correctness of a manual configuration step to users has a practical impact on whether a configuration process with tool support can be successfully adopted in practice.
Method: In an existing work, an UML-based variability modeling methodology named as SimPL and an interactive configuration process was proposed. Based on the existing work, we propose an automated, incremental and efficient conformance checking approach to ensure that the manual configuration of a variation point conforms to a set of pre-defined conformance rules specified in the Object Constraint Language (OCL). The proposed approach, named as Zen-CC, has been implemented as an integrated part of our product configuration and derivation tool: Zen-Configurator.
Results: The performance and scalability of Zen-CC have been evaluated with a real-world case study. Results show that Zen- CC significantly outperformed two baseline engines in terms of performance. Besides, the performance of Zen-CC remains stable during the configuration of all the 10 products of the product line and its efficiency also remains un-impacted even with the growing product complexity, which is not the case for both of the baseline engines.
Conclusion: The results suggest that Zen-CC performs practically well and is much more scalable than the two baseline engines and is scalable for configuring products with a larger number of variation points.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2016 |
| Journal | Information and Software Technology |
| Volume | 72 |
| Pagination | 68-89 |
| Date Published | 04/2016 |
| Publisher | Elsevier |
Model-Based Security Engineering for Cyber-Physical Systems: A Systematic Mapping Study
Information and Software Technology 83 (2016): 116-135.Status: Published
Model-Based Security Engineering for Cyber-Physical Systems: A Systematic Mapping Study
Context: Cyber-physical systems (CPSs) have emerged to be the next generation of engineered systems driving the so-called fourth industrial revolution. CPSs are becoming more complex, open and more prone to security threats, which urges security to be engineered systematically into CPSs. Model-Based Security Engineering (MBSE) could be a key means to tackle this challenge via security by design, abstraction and automation.
Objective: We aim at providing an initial assessment on the state of the art in MBSE for CPSs (MBSE4CPS). Specifically, this work focuses on finding out 1) the publication statistics of MBSE4CPS studies; 2) the characteristics of MBSE4CPS studies; and 3) the open issues of MBSE4CPS research.
Method: We conducted a systematic mapping study (SMS) following a rigorous protocol that was developed based on the state-of-the-art SMS and systematic review guidelines. From thousands of relevant publications, we systematically identified 34 primary MBSE4CPS studies for data extraction and synthesis to answer predefined research questions.
Results: SMS results show that for two recent years (2014-2015) the number of primary MBSE4CPS studies has increased significantly. Within the primary studies, the popularity of using Domain-Specific Languages (DSLs) is comparable with the use of the standardized UML modeling notation. Most primary studies do not explicitly address specific security concerns (e.g., confidentiality, integrity) but rather focus on security analyses in general on threats, attacks or vulnerabilities. Few primary studies propose to engineer security solutions for CPSs. Many focus on the early stages of development lifecycle such as security requirement engineering or analysis.
Conclusion: The SMS does not only provide the state of the art in MBSE4CPS, but also points out several open issues that would deserve more investigation, e.g., the lack of engineering security solutions for CPSs, limited tool support, too few industrial case studies, and the challenge of bridging DSLs in engineering secure CPSs.
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2016 |
| Journal | Information and Software Technology |
| Volume | 83 |
| Pagination | 116–135 |
| Date Published | 11/2016 |
| Publisher | Elsevier |
| Keywords | Cyber-Physical Systems, Security Testing, Testing, Uncertainty |
| URL | http://dx.doi.org/10.1016/j.infsof.2016.11.004 |
Tackling Uncertainty in Cyber-Physical Systems with Automated Testing
ADA User Journal 37, no. 4 (2016).Status: Published
Tackling Uncertainty in Cyber-Physical Systems with Automated Testing
The U-Test-EU project aims at developing new methods and techniques for testing Cyber-Physical Systems (CPSs) under uncertainty. This paper aims to provide the current status of the results achieved in the project during the first one and half years. Our ultimate aim is to enable collaboration among several Horizon2020 projects focusing on CPSs. This paper focuses on the research results from the following four perspectives in the context of the project: 1) Understanding uncertainty in CPSs, 2) Modeling uncertainty in CPSs to support automated testing, 3) Discovering unspecified uncertainties, 4) Testing CPSs under the specified and discovered uncertainties. In addition to the research results, we also present a set of standardization activities that are planned in the project with the final goal of bringing results to a wider audience than the targeted projects and consortium of the project.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Journal Article |
| Year of Publication | 2016 |
| Journal | ADA User Journal |
| Volume | 37 |
| Issue | 4 |
| Publisher | ADA |
| Keywords | Cyber-Physical Systems, Search-Based Software Engineering, Testing, Uncertainty |
| Notes |
This work is supported by EU project (Testing Cyber-Physical Systems under Uncertainty). |
Zen-ReqOptimizer: A Search-based Approach for Requirements Assignment Optimization
Empirical Software Engineering 22, no. 1 (2016): 175-234.Status: Published
Zen-ReqOptimizer: A Search-based Approach for Requirements Assignment Optimization
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2016 |
| Journal | Empirical Software Engineering |
| Volume | 22 |
| Issue | 1 |
| Pagination | 175-234 |
| Date Published | 03/2016 |
| Publisher | Springer |
Technical reports
An Integrated Modeling Framework to Facilitate Model-Based Testing of Cyber-Physical Systems under Uncertainty
Simula Research Laboratory, 2016.Status: Published
An Integrated Modeling Framework to Facilitate Model-Based Testing of Cyber-Physical Systems under Uncertainty
Undertaking innate uncertainty during the operation of Cyber-Physical Systems (CPSs) is crucial for their intact behavior. One method to handle such uncertainty is using Model-based Testing (MBT); however, existing MBT techniques do not explicitly capture uncertainty in test ready models that capture expected behavior of a CPS and its operating environment. To fill this gap, we present an Uncertainty-Wise test-modeling framework, named as Uncertum, to create test ready models to support MBT of CPSs facing uncertainty. Uncertum relies on the definition of a UML profile (the UML Uncertainty Profile (UUP)) and a set of UML model libraries extending the UML profile for Modeling and Analysis of Real-Time and Embedded Systems (MARTE). Uncertum also benefits from the UML Testing Profile (UTP) V.2 to support standard-based MBT. Uncertum was evaluated with two industrial CPS case studies, one real world and one open source CPS case study from the following four perspectives: 1) Completeness and Coverage of the profiles and model libraries in terms of concepts defined in their underlying uncertainty conceptual model for CPSs (i.e., U-Model and MARTE, 2) Effort required to model uncertainty with Uncertum, and 3) Correctness of the developed test ready models, which was assessed via model execution.
| Afilliation | Software Engineering |
| Pr |