Projects
MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
The innovation planned in this project is an add-on to the digitization project currently being undertaken by the Cancer Registry of Norway (CR). The project started in 2009 and aims to transform the current paper-based/manual system into an ICT-based Automated Cancer Registry System (ACRS). The planned innovation project aims to develop systematic, automated and cost-effective model-based approaches for ensuring the quality of the evolving ACRS system and therefore significantly improving the efficiency of the patient history registration process. This will positively affect all its end users, including researchers, patients, doctors, and government officials.
Funding source:
Regionale forskningsfond
All partners:
- Simula Research Laboratory
- Cancer Registry of Norway
Project leaders:
- Simula: Tao Yue (PI) and Shaukat Ali (Co-PI),
- Cancer Registry: Jan F. Nygård
Publications for MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System
Journal Article
Automated Refactoring of OCL Constraints with Search
IEEE Transactions on Software Engineering (TSE) (2017).Status: Published
Automated Refactoring of OCL Constraints with Search
Object Constraint Language (OCL) constraints are typically used to provide precise semantics to models developed with the Unified Modeling Language (UML). When OCL constraints evolve regularly, it is essential that they are easy to understand and maintain. For instance, in cancer registries, to ensure the quality of cancer data, more than one thousand medical rules are defined and evolve regularly. Such rules can be specified with OCL. It is, therefore, important to ensure the understandability and maintainability of medical rules specified with OCL. To tackle such a challenge, we propose an automated search-based OCL constraint refactoring approach (SBORA) by defining and applying four semantics-preserving refactoring operators (i.e., Context Change, Swap, Split and Merge) and three OCL quality metrics (Complexity, Coupling, and Cohesion) to measure the understandability and maintainability of OCL constraints. We evaluate SBORA along with six commonly used multi-objective search algorithms (e.g., Indicator-Based Evolutionary Algorithm (IBEA)) by employing four case studies from different domains: healthcare (i.e., cancer registry system from Cancer Registry of Norway (CRN)), Oil&Gas (i.e., subsea production systems), warehouse (i.e., handling systems), and an open source case study named SEPA. Results show: 1) IBEA achieves the best performance among all the search algorithms and 2) the refactoring approach along with IBEA can manage to reduce on average 29.25% Complexity and 39% Coupling and improve 47.75% Cohesion, as compared to the original OCL constraint set from CRN. Furthermore, we conducted a controlled experiment with 96 subjects and results show that the understandability and maintainability of the original constraint set can be improved significantly from the perspectives of the 96 participants of the controlled experiment.
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | IEEE Transactions on Software Engineering (TSE) |
| Publisher | IEEE |
| URL | http://ieeexplore.ieee.org/document/8114267/ |
| DOI | 10.1109/TSE.2017.2774829 |
IOCL: An Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
Science of Computer Programming (SCP) 149 (2017): 3-8.Status: Published
IOCL: An Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
The Object Constraint Language (OCL) is commonly used for specifying additional constraints on models, in addition, to the ones enforced by the semantics of the models. However, a lot of practitioners and even researchers are reluctant in using OCL to some extent due to the lack of sufficient familiarity with OCL. To facilitate practitioners and researchers in specifying OCL constraints, we designed and developed a web-based tool called interactive OCL (iOCL) for interactively specifying constraints on a given model. The core idea behind iOCL is to present and display only relevant details (e.g., operations) of OCL to users at a given step of constraint specification process, in addition to helping modelers with its syntax. We evaluated iOCL using a real-world case study from Cancer Registry of Norway and the results showed that iOCL can significantly reduce the time required to specify OCL constraints and decrease the possibility of making syntactic errors during the specification process. Thus, we conclude that iOCL can facilitate the process of OCL constraint specification.
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Science of Computer Programming (SCP) |
| Volume | 149 |
| Pagination | 3-8 |
| Date Published | 08/2017 |
| Publisher | Elsevier |
Search and similarity based selection of use case scenarios: An empirical study
Empirical Software Engineering (2017): 1-78.Status: Published
Search and similarity based selection of use case scenarios: An empirical study
Use case modeling is a well-known requirements specification method and has been widely applied in practice. Use case scenarios of use case models are input elements for requirements inspection and analysis, requirements-based testing, and other downstream activities. It is, however, a practical challenge to inspect all use case scenarios that can be obtained from any non-trivial use case model, as such an inspection activity is often performed manually by domain experts. Therefore, it is needed to propose an automated solution for selecting a subset of use case scenarios with the ultimate aim of enabling cost-effective requirements (use case) inspection, analysis, and other relevant activities. Our solution is built on a natural language based, restricted use case modeling methodology (named as RUCM), in the sense that requirements specifications are specified as RUCM use case models. Use case scenarios can be automatically derived from RUCM use case models with the already established Zen-RUCM framework. In this paper, we propose a search-based and similarity-based approach called S3RCUM, through an empirical study, to select most diverse use case scenarios to enable cost-effective use case inspections. The empirical study was designed to evaluate the performance of three search algorithms together with eight similarity functions, through one real-world case study and six case studies from literature. Results show that (1+1) Evolutionary Algorithm together with Needleman-Wunsch similarity function significantly outperformed the other 31 combinations of the search algorithms and similarity functions. The combination managed to select 50% of all the generated RUCM use case scenarios for all the case studies to detect all the seeded defects.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems , U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Empirical Software Engineering |
| Pagination | 1-78 |
| Date Published | 04/2017 |
| Publisher | Springer |
Master's thesis
A Large-Scale OCL Constraint Repository And Comprehensive Analysis For Supporting Automated Cancer Registry System
In The Department of Informatics, University of Oslo, 2017.Status: Published
A Large-Scale OCL Constraint Repository And Comprehensive Analysis For Supporting Automated Cancer Registry System
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Master's thesis |
| Year of Publication | 2017 |
| Degree awarding institution | The Department of Informatics, University of Oslo |
A Rule-Based Framework for Supporting Automated Change Impact Analysis in the Cancer Registry of Norway
In The Department of Informatics, University of Oslo, 2017.Status: Published
A Rule-Based Framework for Supporting Automated Change Impact Analysis in the Cancer Registry of Norway
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Master's thesis |
| Year of Publication | 2017 |
| Degree awarding institution | The Department of Informatics, University of Oslo |
Proceedings, refereed
RCIA: Automated Change Impact Analysis to Facilitate a Practical Cancer Registry System
In The International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2017.Status: Published
RCIA: Automated Change Impact Analysis to Facilitate a Practical Cancer Registry System
The Cancer Registry of Norway (CRN) employs a cancer registry system to collect cancer patient data (e.g., diagnosis and treatments) from various medical entities (e.g., clinic hospitals). The collected data are then checked for validity (i.e., validation) and assembled as cancer cases (i.e., aggregation) based on more than 1000 cancer coding rules in the system. However, it is frequent in practice that the collected cancer data changes due to various reasons (e.g., different treatments) and the cancer coding rules can also change/evolve due to new medical knowledge. Thus, such a cancer registry system requires an efficient means to automatically analyze these changes and provide consequent impacts to medical experts for further actions. This paper proposes an automated Rule-based Change Impact Analysis (CIA) approach named RCIA that includes: 1) a change classification to capture the potential changes that can occur at CRN; 2) in total 80 change impact analysis rules including 50 dependency rules and 30 impact rules; and 3) an efficient algorithm to analyze changes and produce consequent impacts. We evaluate RCIA via a case study with 12 real change sets from CRN and a conducted interview. The results showed that RCIA managed to produce 100% actual change impacts and the medical expert at CRN is quite positive to apply RCIA to facilitate their cancer registry system. We also shared a set of lessons learned based on the collaboration with CRN.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2017 |
| Conference Name | The International Conference on Software Maintenance and Evolution (ICSME) |
| Pagination | 603-612 |
| Publisher | IEEE |
Talks, contributed
RCIA: Automated Change Impact Analysis to Facilitate a Practical Cancer Registry System
In The International Conference on Software Maintenance and Evolution (ICSME), Shanghai, China, 2017.Status: Published
RCIA: Automated Change Impact Analysis to Facilitate a Practical Cancer Registry System
The Cancer Registry of Norway (CRN) employs a cancer registry system to collect cancer patient data (e.g., diagnosis and treatments) from various medical entities (e.g., clinic hospitals). The collected data are then checked for validity (i.e., validation) and assembled as cancer cases (i.e., aggregation) based on more than 1000 cancer coding rules in the system. However, it is frequent in practice that the collected cancer data changes due to various reasons (e.g., different treatments) and the cancer coding rules can also change/evolve due to new medical knowledge. Thus, such a cancer registry system requires an efficient means to automatically analyze these changes and provide consequent impacts to medical experts for further actions. This paper proposes an automated Rule-based Change Impact Analysis (CIA) approach named RCIA that includes: 1) a change classification to capture the potential changes that can occur at CRN; 2) in total 80 change impact analysis rules including 50 dependency rules and 30 impact rules; and 3) an efficient algorithm to analyze changes and produce consequent impacts. We evaluate RCIA via a case study with 12 real change sets from CRN and a conducted interview. The results showed that RCIA managed to produce 100% actual change impacts and the medical expert at CRN is quite positive to apply RCIA to facilitate their cancer registry system. We also shared a set of lessons learned based on the collaboration with CRN.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Talks, contributed |
| Year of Publication | 2017 |
| Location of Talk | The International Conference on Software Maintenance and Evolution (ICSME), Shanghai, China |
Technical reports
A Pilot Experiment to Assess Interactive OCL Specification in a Real Setting
Simula Research Laboratory, 2017.Status: Published
A Pilot Experiment to Assess Interactive OCL Specification in a Real Setting
The Object Constraint Language (OCL) is a formal, declarative, and side-effect free language, standardized by the Object Management Group, for specifying constraints or queries on models specified in the Unified Modeling Language (UML). OCL was designed with the aim to bridge the gap between natural language and traditional formal languages requiring a strong mathematical background to understand and apply. OCL, along with UML, have been applied in practice for various purposes such as facilitating automated model-based testing. In most of such contexts of OCL, engineers with software engineering backgrounds specify OCL constraints. However, it is still a challenge for constraint authors (e.g., medical coders) who have no such background to apply OCL for other purposes (e.g., specifying medical rules). In this direction, in our previous work, we proposed a user-interactive specification framework, named iOCL, for facilitating OCL constraint specification and validation. The aim was to ease its adoption in practice in a wider application scope. In this paper, we present a pilot experiment that was conducted to assess the practical applicability of iOCL in Cancer Registry of Norway with real users of iOCL in terms of specifying medical cancer coding rules with iOCL. Results of the pilot experiment showed that, with iOCL, time to specify OCL constraints can be significantly reduced as compared to directly specifying OCL constraints without the tool support. In addition, participants of the experiment found that iOCL is easy to use.
| Afilliation | Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Technical reports |
| Year of Publication | 2017 |
| Publisher | Simula Research Laboratory |
Proceedings, refereed
A Model-Based Approach with Tool Support to Facilitate the Cancer Registration Process in Cancer Registry of Norway
In European Telemedicine Conference (ETC), 2016.Status: Published
A Model-Based Approach with Tool Support to Facilitate the Cancer Registration Process in Cancer Registry of Norway
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | European Telemedicine Conference (ETC) |
iOCL: A Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
In Tool Demonstrations Track, ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MODELS), 2016.Status: Published
iOCL: A Interactive Tool for Specifying, Validating and Evaluating OCL Constraints
The Object Constraint Language (OCL) is frequently used to specify additional constraints on models, in addition, to the ones enforced by semantics of the models. It is a well- known fact that due to the lack of familiarity with OCL, practitioners and even researcher to some extent are reluctant in using OCL. To help practitioners and researchers in writing OCL constraints for their specific problem at hand, we developed a tool called interactive OCL (iOCL) for interactively specifying constraints on a given model. The basic philosophy behind the tool is to present only those details (e.g., operations) of OCL to modelers that are valid at a given step of constraint specification process, in addition to helping modelers with its syntax. Our ultimate aim is to reduce the effort required to specify constraints, subsequently lowering down training cost and increasing the correctness of the constraints. iOCL is a web-based ap- plication that integrates other tools including Eclipse OCL for validation and evaluation of OCL constraints, and EsOCL for automatically generating valid instances of models that satisfy the specified constraints.
| Afilliation | Software Engineering, Software Engineering, Software Engineering |
| Project(s) | MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System, The Certus Centre (SFI) |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2016 |
| Conference Name | Tool Demonstrations Track, ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MODELS) |
| Pagination | 1-7 |
| Date Published | 09/2016 |
U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty is intrinsic in Cyber-Physical Systems (CPS) owning to novel interactions among software, embedded systems, networking equipment, cloud infrastructures, and agents (e.g., humans). Such systems have become predominantly visible in critical industrial domains (e.g., healthcare and transportation) and oblige the implementation of proper mechanisms to deal with uncertainty during their real operation. One way to ensure the correct implementation of such mechanisms is with automated testing. The U-Test project aims at ensuring that CPS are tested adequately under uncertainty using systematic and automated techniques such as model and search-based testing to facilitate their reliable operation.
U-Test keeps a full catalog of the project's publications, including publications from other academic institutions than Simula.
Final goal
To improve the dependability of Cyber-Physical Systems (CPS) via cost-effective model-based and search-based testing of CPS under uncertainty, by defining an Uncertainty Taxonomy and holistic modeling and testing frameworks with considerable reliance on standards.
Funding source
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. 645463. (ICT-01-2014 - Smart Cyber-Physical Systems)
All partners
- Oslo Medtech (Norway)
- Simula (Norway)
- Technical University of Vienna (Austria)
- Fraunhofer FOKUS (Germany)
- Future Position X (Sweden)
- ULMA Handling Systems (Spain)
- Nordic MedTest (Sweden)
- Easy Global Market (France)
- Ikerlan (Spain)
Technical project leader
Shaukat Ali (PI)
Standardization leader
Tao Yue (Co-PI)
Media presence
Publications for U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies
Journal Article
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
Journal of Systems and Software 153 (2019).Status: Published
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Journal of Systems and Software |
| Volume | 153 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Journal Article
Specifying Uncertainty in Use Case Models
Journal of Systems and Software 144 (2018): 573-603.Status: Published
Specifying Uncertainty in Use Case Models
Context: Latent uncertainty in the context of software-intensive systems (e.g., Cyber-Physical Systems (CPSs)) demands explicit attention right from the start of development. Use case modeling—a commonly used method for specifying requirements in practice, should also be extended for explicitly specifying uncertainty.
Objective: Since uncertainty is a common phenomenon in requirements engineering, it is best to address it explicitly by identifying, qualifying, and, where possible, quantifying uncertainty at the beginning stage. The ultimate aim, though not within the scope of this paper, was to use these use cases as the starting point to create test-ready models to support automated testing of CPSs under uncertainty.
Method: We extend the Restricted Use Case Modeling (RUCM) methodology and its supporting tool to specify uncertainty as part of system requirements. Such uncertainties include those caused by insufficient domain expertise of stakeholders, disagreements among them, and known uncertainties about assumptions about the environment of the system. The extended RUCM, called U-RUCM, inherits the features of RUCM, such as automated analyses and generation of models, to mention but a few. Consequently, U-RUCM provides all the key benefits offered by RUCM (i.e., reducing ambiguities in requirements), but also, it allows specification of uncertainties with the possibilities of reasoning and refining existing ones and even uncovering unknown ones.
Results: We evaluated U-RUCM with two industrial CPS case studies. After refining RUCM models (specifying initial requirements), by applying the U-RUCM methodology, we successfully identified and specified additional 306% and 512% (previously unknown) uncertainty requirements, as compared to the initial requirements specified in RUCM. This showed that, with U-RUCM, we were able to get a significantly better and more precise characterization of uncertainties in requirement engineering.
Conclusion: Evaluation results show that U-RUCM is an effective methodology (with tool support) for dealing with uncertainty in requirements engineering. We present our experience, lessons learned, and future challenges, based on the two industrial case studies.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | Journal of Systems and Software |
| Volume | 144 |
| Pagination | 573-603 |
| Publisher | Elsevier |
| Keywords | Belief, Uncertainty, Use Case Modeling |
| DOI | 10.1016/j.jss.2018.06.075 |
PhD Thesis
Uncertainty-wise Cyber-Physical Systems Testing
In The University of Oslo. Vol. PhD. Norway: The University of Oslo, 2018.Status: Published
Uncertainty-wise Cyber-Physical Systems Testing
A Cyber-Physical Systems (CPS), as an integration of computing, communication, and control for making intelligent and autonomous systems, has been widely applied in various safety-critical domains, e.g., avionics and automotive. However, uncertainty is inherent in CPSs due to various reasons such as unpredictable environment under which the CPSs are operated. And, uncertainties may cause irreparable accidents once they cannot be handled properly by CPSs. Therefore, it is crucial to identify uncertainties in CPSs and test CPSs under the uncertainties, to ensure that CPSs are capable of handling the uncertainties during their actual operations, i.e., making CPSs less uncertain.
Towards this direction, five contributions were made in the thesis corresponding to five papers respectively: (C1) a conceptual model, named as U-Model, for helping develop a systematic and comprehensive understanding of uncertainty in CPSs; (C2) an use case modeling methodology, named as U-RUCM, for identifying, qualifying, and, where possible, quantifying uncertainty in requirements engineering; (C3) a test modeling methodology, named as UncerTum, for supporting the construction of test ready models with the explicit representation of uncertainties in CPSs; (C4) an evolution framework, named as UncerTolve, for interactively evolving test ready models specified with UncerTum based on real operational data; and (C5) a testing framework, named as UncerTest, for testing CPSs in the presence of uncertainties in their operating environments in a cost-effective manner using model-based and search-based testing techniques.
Based on our evaluations of the five contributions with the industrial CPS case studies, we observed that U-Model, as the foundation for this research, is sufficiently complete for characterizing and classifying uncertainties in CPSs. Then, the U-Model based modeling methodologies U-RUCM and UncerTum offer solutions to enable the identification and specification of uncertainties at two critical phases of a system development lifecycle: requirements engineering and testing. Furthermore, UncerTolve can successfully evolve model elements of the test ready models specified with UncerTum and calculate objective uncertainty measurements based on real operational data. Last, UncerTest managed to cost-effectively test CPSs in the presence of uncertainties and proactively identify unknown uncertainties by introducing the sources of the uncertainties into the test environments during test case execution.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBT4CPS: Model-Based Testing For Cyber-Physical Systems , The Certus Centre (SFI) |
| Publication Type | PhD Thesis |
| Year of Publication | 2018 |
| Degree awarding institution | The University of Oslo |
| Degree | PhD |
| Number of Pages | 292 |
| Date Published | 26/06/2018 |
| Publisher | The University of Oslo |
| Place Published | Norway |
| Keywords | Cyber-Physical System, Model-based Testing, Uncertainty |
Book Chapter
Uncertainty-wise Testing of Cyber-Physical Systems
In Advances in Computers, 23-94. Vol. 107. Elsevier, 2017.Status: Published
Uncertainty-wise Testing of Cyber-Physical Systems
As compared with classical software/system testing, uncertainty-wise testing explicitly addresses known uncertainty about the behavior of a System Under Test (SUT), its operating environment, and interactions between the SUT and its operational environment, across all testing phases, including test design, test generation, test optimization, and test execution, with the aim to mainly achieve the following two goals. First, uncertainty-wise testing aims to ensure that the SUT deals with known uncertainty adequately. Second, uncertainty-wise testing should be also capable of learning new (previously unknown) uncertainties such that the SUT’s implementation can be improved to guard against newly learned uncertainties during its operation. The necessity to integrate uncertainty in testing is becoming imperative because of the emergence of new types of intelligent and communicating software-based systems such as Cyber-Physical Systems (CPSs). Intrinsically, such systems are exposed to uncertainty because of their interactions with highly indeterminate physical environments. In this chapter, we provide our understanding and experience of uncertainty-wise testing from the aspects of uncertainty-wise model-based testing, uncertainty-wise modeling and evolution of test ready models, and uncertainty-wise multi-objective test optimization, in the context of testing CPSs under uncertainty. Furthermore, we present our vision about this new testing paradigm and its plausible future research directions.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Book Chapter |
| Year of Publication | 2017 |
| Book Title | Advances in Computers |
| Volume | 107 |
| Chapter | 2 |
| Pagination | 23-94 |
| Publisher | Elsevier |
Journal Article
Uncertainty-Wise Cyber-Physical System Test Modeling
Software & Systems Modeling (2017).Status: Published
Uncertainty-Wise Cyber-Physical System Test Modeling
It is important that a Cyber-Physical System (CPS) deals with uncertainty in its behavior caused by its unpredictable operating environment, to ensure its reliable operation. One method to ensure that the CPS will handle such uncertainty during its operation is by testing the CPS with Model-based Testing (MBT) techniques. However, existing MBT techniques do not explicitly capture uncertainty in test ready models i.e., capturing the uncertain expected behavior of a CPS in the presence of environment uncertainty. To fill this gap, we present an Uncertainty-Wise test-modeling framework, named as Uncertum, to create test ready models to support MBT of CPSs facing uncertainty. Uncertum relies on the definition of a UML profile (the UML Uncertainty Profile (UUP)) and a set of UML model libraries extending the UML profile for Modeling and Analysis of Real-Time and Embedded Systems (MARTE). Uncertum also benefits from the UML Testing Profile (UTP) V.2 to support standard-based MBT. Uncertum was evaluated with two industrial CPS case studies, one real-world case study, and one open source CPS case study from the following four perspectives: 1) Completeness and Coverage of the profiles and model libraries in terms of concepts defined in their underlying uncertainty conceptual model for CPSs (i.e., U-Model and MARTE, 2) Effort required to model uncertainty with Uncertum, and 3) Correctness of the developed test ready models, which was assessed via model execution. Based on the evaluation, we can conclude that we were successful in modeling all the uncertainties identified in the four case studies, which gives us an indication that Uncertum is sufficiently complete. In terms of modeling effort, we concluded that on average Uncertum requires18.5% more time to apply stereotypes from UUP on test ready models.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Software & Systems Modeling |
| Publisher | Springer |
| ISSN | 1619-1374 |
| Keywords | Cyber-Physical System; UML, Model-based Testing, Uncertainty |
| DOI | 10.1007/s10270-017-0609-6 |
Uncertainty-Wise Evolution of Test Ready Models
Information and Software Technology (IST) 87 (2017): 140-159.Status: Published
Uncertainty-Wise Evolution of Test Ready Models
Context: Cyber-Physical Systems (CPSs), when deployed for operation, are inherently prone to uncertainty. Considering their applications in critical domains (e.g., healthcare), it is important that such CPSs are tested sufficiently, with the explicit consideration of uncertainty. Model-based testing (MBT) involves creating test ready models capturing the expected behavior of a CPS and its operating environment. These test ready models are then used for generating executable test cases. It is, therefore, necessary to develop methods that can continuously evolve, based on real operational data collected during the operation of CPSs, test ready models and uncertainty captured in them, all together termed as Belief Test Ready Models (BMs)
Objective: Our objective is to propose a model evolution framework that can interactively improve the quality of BMs, based on operational data. Such BMs are developed by one or more test modelers (belief agents) with their assumptions about the expected behavior of a CPS, its expected physical environment, and potential future deployments. Thus, these models explicitly contain subjective uncertainty of the test modelers.
Method: We propose a framework (named as UncerTolve) for interactively evolving BMs (specified with extended UML notations) of CPSs with subjective uncertainty developed by test modelers. The key inputs of UncerTolve include initial BMs of CPSs with known subjective uncertainty and real data collected from the operation of CPSs. UncerTolve has three key features: 1) Validating the syntactic correctness and conformance of BMs against real operational data via model execution, 2) Evolving objective uncertainty measurements of BMs via model execution, and 3) Evolving state invariants (modeling test oracles) and guards of transitions (modeling constraints for test data generation) of BMs with a machine learning technique.
Results: As a proof-of-concept, we evaluated UncerTolve with one industrial CPS case study, i.e., GeoSports from the healthcare domain. Using UncerTolve, we managed to evolve 51% of belief elements, 18% of states, and 21% of transitions as compared to the initial BM developed in an industrial setting.
Conclusion: UncerTolve can successfully evolve model elements of the initial BM, in addition to objective uncertainty measurements using real operational data. The evolved model can be used to generate additional test cases covering evolved model elements and objective uncertainty. These additional test cases can be used to test the current and future deployments of a CPS to ensure that it will handle uncertainty gracefully during its operations.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI) |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Information and Software Technology (IST) |
| Volume | 87 |
| Pagination | 140-159 |
| Publisher | Elsevier |
| ISSN | 0950-5849 |
| Keywords | Belief Model, Belief Test Ready Model, Model Evolution, Model-based Testing, Uncertainty |
| DOI | 10.1016/j.infsof.2017.03.003 |
Search and similarity based selection of use case scenarios: An empirical study
Empirical Software Engineering (2017): 1-78.Status: Published
Search and similarity based selection of use case scenarios: An empirical study
Use case modeling is a well-known requirements specification method and has been widely applied in practice. Use case scenarios of use case models are input elements for requirements inspection and analysis, requirements-based testing, and other downstream activities. It is, however, a practical challenge to inspect all use case scenarios that can be obtained from any non-trivial use case model, as such an inspection activity is often performed manually by domain experts. Therefore, it is needed to propose an automated solution for selecting a subset of use case scenarios with the ultimate aim of enabling cost-effective requirements (use case) inspection, analysis, and other relevant activities. Our solution is built on a natural language based, restricted use case modeling methodology (named as RUCM), in the sense that requirements specifications are specified as RUCM use case models. Use case scenarios can be automatically derived from RUCM use case models with the already established Zen-RUCM framework. In this paper, we propose a search-based and similarity-based approach called S3RCUM, through an empirical study, to select most diverse use case scenarios to enable cost-effective use case inspections. The empirical study was designed to evaluate the performance of three search algorithms together with eight similarity functions, through one real-world case study and six case studies from literature. Results show that (1+1) Evolutionary Algorithm together with Needleman-Wunsch similarity function significantly outperformed the other 31 combinations of the search algorithms and similarity functions. The combination managed to select 50% of all the generated RUCM use case scenarios for all the case studies to detect all the seeded defects.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems , U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBE-CR: An Innovative Approach for Longstanding Development and Maintenance of the Automated Cancer Registry System |
| Publication Type | Journal Article |
| Year of Publication | 2017 |
| Journal | Empirical Software Engineering |
| Pagination | 1-78 |
| Date Published | 04/2017 |
| Publisher | Springer |
Miscellaneous
Uncertainty Testing Framework V.3
None, 2017.Status: Published
Uncertainty Testing Framework V.3
This is a public deliverable for U-Test project.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Miscellaneous |
| Year of Publication | 2017 |
| Publisher | None |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty Testing Framework V.2
None, 2017.Status: Published
Uncertainty Testing Framework V.2
This is a public deliverable for the U-Test project.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Miscellaneous |
| Year of Publication | 2017 |
| Publisher | None |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty Modeling Framework Version 2
None, 2017.Status: Published
Uncertainty Modeling Framework Version 2
This a U-Test public deliverable.
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Miscellaneous |
| Year of Publication | 2017 |
| Publisher | None |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
MPM4CPS: Multi-Paradigm Modelling for Cyber-Physical Systems
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
To date there is neither unifying theory, systematic design methods, nor techniques and tools for Cyber-Physical Systems (CPS). Individual engineering disciplines offer only partial solutions. Multi-paradigm Modelling (MPM) proposes to model every part and aspect of a system explicitly, at the most appropriate level of abstraction, using the most appropriate modelling formalism. The MPM4CPS action aims to promote the sharing of foundations, techniques, and tools to academia and industry. By bringing together and disseminating knowledge and experiments on CPS and MPM solutions across disciplines this goal can be achieved.
Funding source:
EU COST Action
Management Committee Members:
Tao Yue, Shaukat Ali
Publications for MPM4CPS: Multi-Paradigm Modelling for Cyber-Physical Systems
Talks, contributed
U-TCsGM: Generating and Minimizing Uncertainty-Based Test Cases for Cyber-Physical Systems (Tool Demo)
In MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016, 2016.Status: Published
U-TCsGM: Generating and Minimizing Uncertainty-Based Test Cases for Cyber-Physical Systems (Tool Demo)
In this tool demo, we will present the implementation of our recent research work on generating and minimizing executable test cases from the test models of a Cyber-Physical System tagged with subjective uncertainty. The algorithms are founded on uncertainty theory and NSGA-II—the most commonly used multi-objective search algorithm. We will demonstrate the complete process starting from creating a test model with uncertainty, generating test cases, minimizing test cases, and finally executing the minimized test cases using a real CPS case study.
| Afilliation | Software Engineering, Software Engineering |
| Project(s) | MPM4CPS: Multi-Paradigm Modelling for Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2016 |
| Location of Talk | MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016 |
| Keywords | Model-based Testing, Search-Based Testing, Uncertainty |
Model-Driven Testing of Cyber-Physical Systems with the Explicit Consideration of Uncertainty
In MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016, 2016.Status: Published
Model-Driven Testing of Cyber-Physical Systems with the Explicit Consideration of Uncertainty
It is a well-recognized fact that Cyber-Physical Systems (CPSs) face both known and unknown uncertainty during their operation. This demands the development of testing techniques that must take into account known uncertainty both in the CPS and its environment with the final goal of discovering unknown uncertainty against which the CPS can be tested. Eventually, the implementation of the CPS can be improved to shield it against the newly discovered uncertainty. In this presentation, we will present some of the results that we have achieved in an EU Horizon2020 project (U-Test) in this regard. We, first, demonstrate the modeling of test ready models of CPSs together with subjective uncertainty using the Uncertainty Modeling Framework. Second, we present two uncertainty-based test case generation and four test case minimization techniques relying on the test ready models founded on Uncertainty theory and multi-objective search. Third, we present the evaluation of the test case generation and minimization techniques that was conducted to select the best strategy to be used in the practice to test a real CPS. Fourth, we present the results of testing a real CPS with the selected best strategy.
| Afilliation | Software Engineering |
| Project(s) | MPM4CPS: Multi-Paradigm Modelling for Cyber-Physical Systems, The Certus Centre (SFI), U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies |
| Publication Type | Talks, contributed |
| Year of Publication | 2016 |
| Location of Talk | MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016 |
| Keywords | Cyber-Physical System, Model-based Testing, Search-Based Testing, Uncertainty |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Integrating Uncertainty Modelling with Use Case Modelling to Discover Unknowns
In MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016, 2016.Status: Published
Integrating Uncertainty Modelling with Use Case Modelling to Discover Unknowns
Use case modeling is a commonly used means for specifying requirements in practice. In the past, we have developed a use case modeling solution (with tool support), named as Restricted Use Case Modeling (RUCM), for the purpose of reducing inherent ambiguities of textual requirements and enabling automated analyses and generations. However, such use case models still contain uncertainties due to various reasons such as insufficient domain expertise and disagreement among stakeholders. We therefore, integrate RUCM with Uncertainty Modeling to provide requirements engineers an integrated platform (named as U-RUCM) for explicitly specifying uncertainties as part of use case models, such that both ambiguities and uncertainties in use case models can be reduced. U-RUCM was devised in the context of the EU Horizon 2020 U-Test project (http://www.u-test.eu/) and has been evaluated with two industrial case studies of two industrial partners of the U-Test consortium.
| Afilliation | Software Engineering, Software Engineering |
| Project(s) | MPM4CPS: Multi-Paradigm Modelling for Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2016 |
| Location of Talk | MPM4CPS WG meetings in Malaga, Spain, 24-25 November 2016 |
| Keywords | Modeling, RUCM, Uncertainty |
Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
The goal of the Zen-Configurator project is to increase the efficiency and effectiveness, and thereby reduce the cost, of configuring large-scale Cyber Physical System (CPS) product lines. To achieve this goal, we maximally automate error-prone and costly manual configuration activities and optimally assist the interactive configuration process. On one hand, the project relies on advanced technologies of constraint solving/evaluation, optimization using search algorithms, and propose state-of-art algorithms to enable automated configuration activities. On the other hand, the project grounds itself to address real challenges faced by industry and propose a practical and applicable solution and apply it in at least one application domain.
Funding source:
Research Council of Norway
All partners:
Simula Research Laboratory
Project leaders:
Tao Yue (PI), Shaukat Ali (Co-PI)
Publications for Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines
Journal Article
Recommending Faulty Configurations for Interacting Systems Under Test Using Multi-Objective Search
ACM Transactions on Software Engineering and Methodology 30, no. 4 (2021): 1-36.Status: Published
Recommending Faulty Configurations for Interacting Systems Under Test Using Multi-Objective Search
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | ACM Transactions on Software Engineering and Methodology |
| Volume | 30 |
| Issue | 4 |
| Pagination | 1-36 |
| Publisher | ACM |
| DOI | 10.1145/3464939 |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
PhD Thesis
Improving Post-Deployment Configuration of Cyber- Physical Systems Using Machine Learning and Multi- Objective Search
In University of Oslo. Vol. PhD. Oslo, Norway: University of Oslo, 2021.Status: Published
Improving Post-Deployment Configuration of Cyber- Physical Systems Using Machine Learning and Multi- Objective Search
Today, Cyber-Physical Systems (CPSs) are increasingly becoming an essential part of our daily lives and can be found in various domains such as energy, communication, and logistics. To accommodate different needs of users and provide customizations, CPS producers often adopt Product Line Engineering (PLE) methodologies. Consequently, CPSs are developed by integrating multiple products within/across product lines (PLs) that communicate with each other through information networks. Several PLE methodologies exist in the literature, however, their suitability for CPS PLs needs to be evaluated because of unique characteristics of CPS PLs (e.g., variabilities corresponding to multiple domains (e.g., electronics, mechanics), complex configuration processes). Hence, we need to identify key requirements of CPS PLE and evaluate existing PLE methodologies to assess their capabilities of supporting CPS PLE. Furthermore, most of the existing studies address challenges related to the pre-deployment configuration (i.e., making configuration decisions at design time) of individual products. There is a need for studies focusing on the post-deployment configuration (i.e., making configuration decisions at runtime) of interacting products.
In this thesis, first, we conducted a systematic domain analysis and proposed a conceptual framework for CPS PLs, based on which we evaluated existing PLE methodologies. Then, we focused on the post-deployment configuration of CPSs and made another two contributions: we proposed 1) an approach to capture patterns of configurations in the form of configuration rules and, and 2) another approach for recommending configurations to improve the post-deployment configuration experience from the perspective of testers and end-users.
To conduct the domain analysis, we analyzed three real-world CPS case studies. Based on the knowledge collected from the domain analysis and a thorough literature review on PLE, we proposed a conceptual framework, in which we 1) clarify the context of CPS PLE by formalizing CPSs, PLE, and configuration process; 2) present classifications of Variation Point (VP), constraint, and view types in addition to other modeling requirements to support the domain engineering of CPS PLs; and 3) formalize various types of automation that can be enabled to support the application engineering of CPS PLs. The completeness of the framework was evaluated using three real-world case studies containing 2161 VPs, 3943 constraints, and 40 views, 11 configuration tools, and an extensive literature review. Furthermore, we also evaluated four representative variability modeling techniques (VMTs): Feature Model (FM), Cardinality- Based Feature Model (CBFM), Common Variability Language (CVL), and SimPL. With the selected VMTs, we modeled a case study to assess if they can capture variabilities of CPS PLs. Results show that using SimPL, CVL, CBFM, and FM, we can capture only 81%, 75%, 50%, and 15% of the total variabilities, respectively.
To capture the configuration patterns in the form of configuration rules, we proposed the Search-Based Rule Mining (SBRM+) approach. SBRM+ combines multi-objective search with machine learning to mine configuration rules in an incremental and iterative way. We evaluated
the performance of SBRM+ using multiple real-world and open-source case studies from the communication domain and compared its performance with Random Search Based Rule Mining (RBRM+). Results show that SBRM+ performed significantly better than RBRM+ in terms of fitness values, six quality indicators, and 17 Machine Learning Quality Measurements MLQMs. As compared to RBRM+, SBRM+ improved the quality of rules up to 28% in terms of MLQMs.
To improve the post-deployment configuration experience, we proposed the Search-Based Configuration Recommendation (SBCR) approach, which recommends faulty configurations for CPSs with interacting products under test, based on mined rules. These configurations can be used to test CPSs and create guidelines for end-users to improve the post-deployment configuration experience. We evaluated SBCR using the same case studies, for which we mined the rules using SBRM+. Results show that SBCR significantly outperformed Random Search- Based Configuration Recommendation (RBCR) in terms of six quality indicators and the percentage of faulty configurations. Overall, SBCR made up to 22% more accurate recommendations than RBCR.
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | PhD Thesis |
| Year of Publication | 2021 |
| Degree awarding institution | University of Oslo |
| Degree | PhD |
| Number of Pages | 254 |
| Date Published | 04/2021 |
| Publisher | University of Oslo |
| Place Published | Oslo, Norway |
| Thesis Type | Collection of papers |
Journal Article
A framework for automated multi‑stage and multi‑step product confguration of cyber‑physical systems
Software and Systems Modeling (SoSym) 19, no. 4 (2020): 1-55.Status: Published
A framework for automated multi‑stage and multi‑step product confguration of cyber‑physical systems
Product line engineering (PLE) has been employed to large-scale cyber-physical systems (CPSs) to provide customization based on users’ needs. A PLE methodology can be characterized by its support for capturing and managing the abstractions as commonalities and variabilities and the automation of the confguration process for efective selection and customization of reusable artifacts. The automation of a confguration process heavily relies on the captured abstractions and formally specifed constraints using a well-defned modeling methodology. Based on the results of our previous work and a thorough literature review, in this paper, we propose a conceptual framework to support multi-stage and multi-step automated product confguration of CPSs, including a comprehensive classifcation of constraints and a list of automated functionalities of a CPS confguration solution. Such a framework can serve as a guide for researchers and practitioners to evaluate an existing CPS PLE solution or devise a novel CPS PLE solution. To validate the framework, we conducted three real-world case studies. Results show that the framework fulflls all the requirements of the case studies in terms of capturing and managing variabilities and constraints. Results of the literature review indicate that the framework covers all the functionalities concerned by the literature, suggesting that the framework is complete for enabling the maximum automation of confguration in CPS PLE.
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | Software and Systems Modeling (SoSym) |
| Volume | 19 |
| Issue | 4 |
| Pagination | 1-55 |
| Date Published | 06/2020 |
| Publisher | Springer |
| Keywords | Automated configuration, Constraint classification, Cyber-Physical Systems, Multi-stage and multi-step configuration process, Product Line Engineering, Real-world case studies, Variability Modeling |
| DOI | 10.1007/s10270-020-00803-8 |
Journal Article
Using multi-objective search and machine learning to infer rules constraining product configurations
Automated Software Engineering (2019): 1-62.Status: Published
Using multi-objective search and machine learning to infer rules constraining product configurations
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Automated Software Engineering |
| Pagination | 1-62 |
| Publisher | Springer |
| DOI | 10.1007/s10515-019-00266-2 |
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
Journal of Systems and Software 153 (2019).Status: Published
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Journal of Systems and Software |
| Volume | 153 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Proceedings, refereed
Stability Analysis for Safety of Automotive Multi-Product Lines: A Search-Based Approach
In The Genetic and Evolutionary Computation Conference (GECCO). ACM, 2019.Status: Published
Stability Analysis for Safety of Automotive Multi-Product Lines: A Search-Based Approach
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | The Genetic and Evolutionary Computation Conference (GECCO) |
| Pagination | 1241-1249 |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Journal Article
Empirical Research in Software Engineering - a Literature Survey
Journal of Computer Science and Technology 33, no. 5 (2018): 876-899.Status: Published
Empirical Research in Software Engineering - a Literature Survey
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | Journal of Computer Science and Technology |
| Volume | 33 |
| Issue | 5 |
| Pagination | 876-899 |
| Date Published | 09/2018 |
| Publisher | Springer |
| DOI | 10.1007/s11390-018-1864-x |
Proceedings, refereed
Model- Based Personalized Visualization System for Monitoring Evolving Industrial Cyber-Physical System
In The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) . IEEE, 2018.Status: Published
Model- Based Personalized Visualization System for Monitoring Evolving Industrial Cyber-Physical System
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2018 |
| Conference Name | The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) |
| Publisher | IEEE |
Tool Support for Restricted Use Case Specification: Findings from a Controlled Experiment
In The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) . IEEE, 2018.Status: Published
Tool Support for Restricted Use Case Specification: Findings from a Controlled Experiment
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2018 |
| Conference Name | The 25th Asia-Pacific Software Engineering Conference (APSEC 2018) |
| Publisher | IEEE |
Automatic Support of the Generation and Maintenance of Assurance Cases
In Symposium on Dependable Software Engineering: Theories, Tools and Applications. Cham: Springer International Publishing, 2018.Status: Published
Automatic Support of the Generation and Maintenance of Assurance Cases
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2018 |
| Conference Name | Symposium on Dependable Software Engineering: Theories, Tools and Applications |
| Pagination | 11-28 |
| Date Published | 08/2018 |
| Publisher | Springer International Publishing |
| Place Published | Cham |
| DOI | 10.1007/978-3-319-99933-3_2 |
MBT4CPS: Model-Based Testing For Cyber-Physical Systems
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
The complexity of Cyber Physical Systems presents unprecedented challenges for testing both nominal functionality and its associated extra-functional properties in unexpected situations. Thus, we need basic research to handle difficulties imposed by testing. Given the breadth and depth of the topic, as a starting point, we emphasize exclusively on dealing with the following two types of uncertain and risky situations. First, we focus on testing security features together in unpredictable and risky situations. Second, we concentrate on testing self-healing features, i.e., the ability of a CPS to recover from faults itself in risk and unpredictable situations. The successful completion of the project will produce distinct testing methods to test systems. Once such well-tested systems start operating in real life, these will be secure and safe.
Publications for the MBT4CPS project can be found in Simula's publication database.
Funding source:
Research Council of Norway
All partners:
Simula Research Laboratory
Project leaders:
Shaukat Ali (PI), Tao Yue (Co-PI)
Publications for MBT4CPS: Model-Based Testing For Cyber-Physical Systems
Journal Article
Testing Self-Healing Cyber-Physical Systems under Uncertainty with Reinforcement Learning: An Empirical Study
Empirical Software Engineering 26, no. 3 (2021): 52.Status: Published
Testing Self-Healing Cyber-Physical Systems under Uncertainty with Reinforcement Learning: An Empirical Study
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | Empirical Software Engineering |
| Volume | 26 |
| Issue | 3 |
| Pagination | 52 |
| Publisher | Springer |
| DOI | 10.1007/s10664-021-09941-z |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Journal Article
Quality Indicators in Search-based Software Engineering
ACM Transactions on Software Engineering and Methodology 29, no. 2 (2020): 1-29.Status: Published
Quality Indicators in Search-based Software Engineering
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Software Engineering and Methodology |
| Volume | 29 |
| Issue | 2 |
| Pagination | 1 - 29 |
| Date Published | May-04-2020 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 1049-331X |
| URL | https://dl.acm.org/doi/10.1145/3375636 |
| DOI | 10.1145/3375636 |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Journal Article
Using multi-objective search and machine learning to infer rules constraining product configurations
Automated Software Engineering (2019): 1-62.Status: Published
Using multi-objective search and machine learning to infer rules constraining product configurations
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Automated Software Engineering |
| Pagination | 1-62 |
| Publisher | Springer |
| DOI | 10.1007/s10515-019-00266-2 |
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
Journal of Systems and Software 153 (2019).Status: Published
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Journal of Systems and Software |
| Volume | 153 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Testing Self-Healing Cyber-Physical Systems under Uncertainty: A Fragility-Oriented Approach
Software Quality Journal 27, no. 2 (2019): 615-649.Status: Published
Testing Self-Healing Cyber-Physical Systems under Uncertainty: A Fragility-Oriented Approach
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Software Quality Journal |
| Volume | 27 |
| Issue | 2 |
| Pagination | 615–649 |
| Date Published | 03/2019 |
| Publisher | Springer |
| URL | https://link.springer.com/article/10.1007/s11219-018-9437-3 |
Proceedings, refereed
Towards a Framework for the Analysis of Multi-Product Lines in the Automotive Domain
In Proceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems. New York, NY, USA: ACM, 2019.Status: Published
Towards a Framework for the Analysis of Multi-Product Lines in the Automotive Domain
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | Proceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems |
| Publisher | ACM |
| Place Published | New York, NY, USA |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Edited books
Editorial to the Theme Issue on Model-based Testing
Software & Systems Modeling: Springer, 2018.Status: Published
Editorial to the Theme Issue on Model-based Testing
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Edited books |
| Year of Publication | 2018 |
| Publisher | Springer |
| Place Published | Software & Systems Modeling |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
First International Workshop on Verification and Validation of Internet of Things
IEEE, 2018.Status: Published
First International Workshop on Verification and Validation of Internet of Things
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Edited books |
| Year of Publication | 2018 |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Journal Article
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
Software and Systems Modeling (2018): 1-31.Status: Published
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
Self-healing Cyber-Physical Systems (SH-CPSs) detect and recover from faults by themselves at runtime. Testing such systems is challenging due to the complex implementation of self-healing behaviors and their interaction with the physical environment, both of which are uncertain. To this end, we propose an executable model-based approach to test self-healing behaviors under environmental uncertainties. The approach consists of a Modeling Framework of SH-CPSs (MoSH) and an accompanying Test Model Executor (TM-Executor). MoSH provides a set of modeling constructs and a methodology to specify executable test models, which capture expected system behaviors and environmental uncertainties. TM-Executor executes the test models together with the systems under test, to dynamically test their self-healing behaviors under uncertainties. We demonstrated the successful application of MoSH to specify 11 self-healing behaviors and 17 uncertainties for three SH-CPSs. The time spent by TM-Executor to perform testing activities was in the order of milliseconds, though the time spent was strongly correlated with the complexity of test models.
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | Software and Systems Modeling |
| Pagination | 1-31 |
| Date Published | 11/2018 |
| Publisher | Springer |
| Place Published | Berlin Heidelberg |
| ISSN | 1619-1374 |
| DOI | 10.1007/s10270-018-00703-y |
Employing Multi-Objective Search to Enhance Reactive Test Case Generation and Prioritization for Testing Industrial Cyber Physical Systems
IEEE Transactions on Industrial Informatics (TII) 14, no. 3 (2018): 1055-1066.Status: Published
Employing Multi-Objective Search to Enhance Reactive Test Case Generation and Prioritization for Testing Industrial Cyber Physical Systems
The test case generation and prioritization of industrial Cyber-Physical Systems (CPSs) face critical challenges and simulation-based testing is one of the most commonly used techniques for testing these complex systems. However, simulation models of industrial CPSs are usually very complex and executing the simulations becomes computationally expensive, which often make it infeasible to execute all the test cases. To address these challenges, this paper proposes a multi-objective test generation and prioritization approach for testing industrial CPSs by defining a fitness function with four objectives and designing different crossover and mutation operators. We empirically evaluated our fitness function and designed operators along with five multi-objective search algorithms (e.g., Non-dominated Sorting Genetic Algorithm (NSGA-II)) using four case studies. The evaluation results demonstrated that NSGA-II achieved significantly better performance than the other algorithms and managed to improve Random Search for on average 43.80% for each objective and 49.25% for the quality indicator Hypervolume (HV).
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2018 |
| Journal | IEEE Transactions on Industrial Informatics (TII) |
| Volume | 14 |
| Issue | 3 |
| Pagination | 1055-1066 |
| Publisher | IEEE |
| URL | http://ieeexplore.ieee.org/abstract/document/8241845/ |
| DOI | 10.1109/TII.2017.2788019 |
CERCIRAS: Connecting Education and Research Communities for an Innovative Resource Aware Society
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Parallel computing platforms have revolutionised the hardware landscape by providing high-performance, low-energy, and specialised (viz. heterogeneous) processing capabilities to a variety of application domains, including mobile, embedded, data-centre and high-performance computing. However, to leverage their potential, system designers must strike a difficult balance in the apportionment of resources to the application components, striving to avoid under- or over-provisions against worst-case utilisation profiles. The entanglement of hardware components in the emerging platforms and the complex behaviour of parallel applications raise conflicting resource requirements, more so in smart, (self-)adaptive and autonomous systems. This scenario presents the hard challenge of understanding and controlling, statically and dynamically, the trade-offs in the usage of system resources (time, space, energy, and data), also from the perspective of the development and maintenance efforts.
Making resource-usage trade-offs at specification, design, implementation, and run time requires profound awareness of the local and global impact caused by parallel threads of applications on individual resources. Such awareness is crucial for academic researchers and industrial practitioners across all European and COST member countries and, therefore, a strategic priority. Reaching this goal requires acting at two levels: (1) networking otherwise fragmented research efforts towards more holistic views of the problem and the solution; (2) leveraging appropriate educational and technology assets to improve the understanding and management of resources by the academia and industry of underperforming economies, in order to promote cooperation inside Europe and achieve economical and societal benefits.
Funding Source
- COST Action, EU
Digital Twin-Enabled Operation Time Analyses
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
This project focuses on developing novel methods to build digital twins of cyber-physical systems to support advanced analyses (e.g., anomaly detection) at the operation time. To this end, we do research in learning digital twins automatically from historical and live data streams with advanced machine learning techniques. Moreover, to develop analyses, we employ various machine learning techniques such as Generative adversarial networks, transfer learning, and curriculum learning.
Funding source
- Internal strategic project. Funding is approximately 1 Million NOK per year.
Publications for Digital Twin-Enabled Operation Time Analyses
Technical reports
Digital Twin-based Anomaly Detection with Curriculum Learning in Cyber-physical Systems
Simula Research Laboratory, 2022.Status: Published
Digital Twin-based Anomaly Detection with Curriculum Learning in Cyber-physical Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Digital Twin-Enabled Operation Time Analyses |
| Publication Type | Technical reports |
| Year of Publication | 2022 |
| Publisher | Simula Research Laboratory |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Gemini Center on Quantum Computing
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Founded in 2020 by NTNU, University of Oslo, and SINTEF, our goal is to make Norway “quantum ready”. By 2021 also Simula is an associate partner. The background for this centre is the emergence of quantum computers, which many experts regard as one of the most important disruptive technologies in the near future. There is a huge investment in this type of technology from governments, research programs, companies and venture capitalists. The total spending on quantum initiatives worldwide in 2020 is estimated to be 22 billion USD. It is vital for Norway to start to build the necessary expertise in order to become “quantum ready”. This centre gathers a strong interdisciplinary, cross-institutional team of researchers from the University of Oslo, NTNU and SINTEF to promote quantum technology, establish contact with internationally leading expert groups, and work together with industrial partners. Highlighting potential use cases and joining forces to build the necessary expertise.
The centre gathers expertise in the emerging technology of quantum computing and focuses on algorithms/software and application areas (rather than hardware). Quantum computers work in a fundamentally different way than classical computers. To utilize them requires that one understands the underlying quantum-physical principles and can use these to rethink how algorithms are designed (quantum software). A multidisciplinary approach is indispensable.
Nordic-Estonian Quantum Computing e-Infrastructure Quest
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
NordiQuEst is a new collaborative effort between four Nordic countries (Norway, Sweden, Finland, and Denmark) and Estonia to build a dedicated Nordic-Estonian quantum computing (QC) ecosystem that integrates various quantum computers and emulators make them accessible to the Nordic-Estonian region to accelerate QC research, development, and education.
Partners
- Chalmers tekniska högskola AB, Sweden
- CSC - IT Center for Science Ltd. Finland
- DTU – Technical University of Denmark, Denmark
- ETAIS - Estonian Scientific Computing Infrastructure - University of Tartu, Estonia
- SINTEF, Norway
- SRL - Simula Research Laboratory AS, Norway
- VTT - VTT Technical Research Centre of Finland Ltd, Finland
Funding Source
- NeIC, NordForsk. Total funding of the Project: 19.2 Million NOK
Co-tester: Collective-Adaptive Testing of Coevolving Autonomous Cyber-Physical Systems of Systems under Uncertainty
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Cyber-Physical Systems of Systems (CPSoS) exhibit unpredictable behaviours due to their coevolution, employed machine learning techniques, collective behaviours, and operation in uncertain environments and unreliable communication networks. Consequently, testing these systems is highly challenging. The Co-tester project aims to address this challenge by developing novel collective-adaptive testing strategies. These strategies will be empowered with advanced artificial intelligence technologies (e.g., machine learning and evolutionary computation techniques), supported with novel strategies for discovering uncertain and unknown behaviours of CPSoS, its constituted Cyber-Physical Systems, operating environment, and networks for extensive testing of CPSoS.
Funding Source
-
Norwegian Research Council's FRIPRO scheme. Total funding 12 Million NOK.
Publications for Co-tester: Collective-Adaptive Testing of Coevolving Autonomous Cyber-Physical Systems of Systems under Uncertainty
Proceedings, refereed
Uncertainty-Aware Transfer Learning to Evolve Digital Twins for Industrial Elevators
In The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2022.Status: Accepted
Uncertainty-Aware Transfer Learning to Evolve Digital Twins for Industrial Elevators
| Afilliation | Software Engineering |
| Project(s) | Co-tester: Collective-Adaptive Testing of Coevolving Autonomous Cyber-Physical Systems of Systems under Uncertainty, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) |
Talks, contributed
Learning Digital Twin Models
In Model-Driven Engineering of Digital Twins Seminar at Dagstuhl. Schloss Dagstuhl, Germany, 2022.Status: Published
Learning Digital Twin Models
Given that operational cyber-physical systems (CPS) produce continuous data, a complementary approach to model-based engineering is to learn digital twins models with machine learning techniques and providing functionalities such as predictions and anomaly detection.
This talk will start with presenting an opinion on the next generation of digital twins (Quantum Digital Twins), where some aspects of digital twins will be implemented as quantum software and executed on quantum computers, e.g., for simulating the physical environment that can be realistically simulated with quantum-mechanical principles.
Followed by this opinion, the talk will present some recent works on learning digital twins from historical data and continuous updates of digital twins with continuous data from operational CPS. Various machine learning techniques were applied, such as generative adversarial networks, curriculum learning, and transfer learning to learn digital twins. The digital twins were built for use cases from the transportation domain and water distribution/treatment plants. These digital twins were focused on anomaly detection and waiting time predictions.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Co-tester: Collective-Adaptive Testing of Coevolving Autonomous Cyber-Physical Systems of Systems under Uncertainty |
| Publication Type | Talks, contributed |
| Year of Publication | 2022 |
| Location of Talk | Model-Driven Engineering of Digital Twins Seminar at Dagstuhl |
| Publisher | Schloss Dagstuhl, Germany |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Quantum Software Engineering Project
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Quantum software engineering (QSE) enables the cost-effective and scalable development of quantum software-enabled applications. However, QSE is a relatively new area of research, yet very critical for building the promised revolutionary quantum computing applications. Towards this end, this project focuses on developing novel quantum software requirements engineering, modelling, development, testing, and debugging approaches.
Funding Source
- This is a strategic initiative at Simula supporting eight person-years to build solid foundations of the QSE area. The overall funding is approx. 10 Million NOK.
Publications for Quantum Software Engineering Project
Proceedings, refereed
Investigating Cause-Effect Graphs
In Q-SE 2022 (WS). Q-SE 2022, 2022.Status: Published
Investigating Cause-Effect Graphs
Cause-effect graphs have shown promising results in identifying relations among causes and effects of classical software systems, followed by designing effective test cases from them. Towards this end, we investigate the use of cause-effect graphs for quantum programs. Classical cause-effect graphs apply classical logic (e.g., AND, OR) to express these relations, which might not be practical for describing similar relations in quantum programs due to quantum superposition and entanglement. Thus, we propose an extension of cause-effect graphs, where quantum logic inspired functions (e.g., Hadamard) and their generalizations are defined and applied. Moreover, we present a metamodel describing various forms of cause-effect graphs. Finally, we demonstrate a possible method for generating test cases from a quantum cause-effect graph applied to a Bell state quantum program. Lastly, the design and utility of the resulting testing method is discussed, along with future prospects for general quantum cause-effect graphs.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Quantum Software Engineering Project |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | Q-SE 2022 (WS) |
| Publisher | Q-SE 2022 |
Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
The primary objective of the Co-evolver project is to explore and exploit the coevolution design of a self-adaptive Cyber-physical Systems (CPSs) to a given level of maturity before deployment and enable the self-evolution of its coevolution strategy during operation, by drawing on theories and technologies from model-based engineering, evolutionary computation, and machine learning. The key scientific outcomes are 1) a multi-paradigm modeling framework for developing executable coevolution design models, 2) novel (co-)evolutionary algorithms and advanced applied studies on uncertainty-related theories and machine learning techniques to enable the continuous exploration and exploitation of coevolution designs, and 3) a comprehensive platform for evolving coevolution design models. The secondary objective is to apply the outcomes to at least one self-CPS application domain, opening a new stream of research in the domain of uncertainty-aware coevolution designs of self-CPSs.
Funding source:
Research Council of Norway
Project leaders:
Simula Research Laboratory
Publications for Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems
Journal Article
Learning Configurations of Operating Environment of Autonomous Vehicles to Maximize their Collisions
IEEE Transactions on Software Engineering (2022).Status: Published
Learning Configurations of Operating Environment of Autonomous Vehicles to Maximize their Collisions
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | IEEE Transactions on Software Engineering |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
On the Preferences of Quality Indicators for Multi-Objective Search Algorithms in Search-Based Software Engineering
The Empirical Software Engineering Journal (EMSE) (2022).Status: Published
On the Preferences of Quality Indicators for Multi-Objective Search Algorithms in Search-Based Software Engineering
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | The Empirical Software Engineering Journal (EMSE) |
| Publisher | Springer |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty-Aware Prediction Validator in Deep Learning Models for Cyber-Physical System Data
ACM Transactions on Software Engineering and Methodology (TOSEM) (2022).Status: Published
Uncertainty-Aware Prediction Validator in Deep Learning Models for Cyber-Physical System Data
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | ACM Transactions on Software Engineering and Methodology (TOSEM) |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Proceedings, refereed
Are Elevator Software Robust Against Uncertainties? Results and Experiences from an Industrial Case Study
In The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2022.Status: Accepted
Are Elevator Software Robust Against Uncertainties? Results and Experiences from an Industrial Case Study
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) |
Talks, contributed
Learning Digital Twin Models
In Model-Driven Engineering of Digital Twins Seminar at Dagstuhl. Schloss Dagstuhl, Germany, 2022.Status: Published
Learning Digital Twin Models
Given that operational cyber-physical systems (CPS) produce continuous data, a complementary approach to model-based engineering is to learn digital twins models with machine learning techniques and providing functionalities such as predictions and anomaly detection.
This talk will start with presenting an opinion on the next generation of digital twins (Quantum Digital Twins), where some aspects of digital twins will be implemented as quantum software and executed on quantum computers, e.g., for simulating the physical environment that can be realistically simulated with quantum-mechanical principles.
Followed by this opinion, the talk will present some recent works on learning digital twins from historical data and continuous updates of digital twins with continuous data from operational CPS. Various machine learning techniques were applied, such as generative adversarial networks, curriculum learning, and transfer learning to learn digital twins. The digital twins were built for use cases from the transportation domain and water distribution/treatment plants. These digital twins were focused on anomaly detection and waiting time predictions.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Co-tester: Collective-Adaptive Testing of Coevolving Autonomous Cyber-Physical Systems of Systems under Uncertainty |
| Publication Type | Talks, contributed |
| Year of Publication | 2022 |
| Location of Talk | Model-Driven Engineering of Digital Twins Seminar at Dagstuhl |
| Publisher | Schloss Dagstuhl, Germany |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Master's thesis
Revealing Dangerous Behaviors in Self-Driving Vehicles Due to Switching between Different Scenarios
In University of Oslo, 2021.Status: Published
Revealing Dangerous Behaviors in Self-Driving Vehicles Due to Switching between Different Scenarios
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Master's thesis |
| Year of Publication | 2021 |
| Degree awarding institution | University of Oslo |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Prototyping Connection Between Digital Twin and Physical Twin for Autonomous Driving to Support Experimentation
In University of Oslo, 2021.Status: Published
Prototyping Connection Between Digital Twin and Physical Twin for Autonomous Driving to Support Experimentation
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Master's thesis |
| Year of Publication | 2021 |
| Degree awarding institution | University of Oslo |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Proceedings, refereed
Restricted Natural Language and Model-based Adaptive Test Generation for Autonomous Driving
In ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS). ACM/IEEE, 2021.Status: Published
Restricted Natural Language and Model-based Adaptive Test Generation for Autonomous Driving
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS) |
| Publisher | ACM/IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Search-based Selection and Prioritization of Test Scenarios for Autonomous Driving Systems
In 13th Symposium on Search-Based Software Engineering. LNCS Springer, 2021.Status: Published
Search-based Selection and Prioritization of Test Scenarios for Autonomous Driving Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | 13th Symposium on Search-Based Software Engineering |
| Publisher | LNCS Springer |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Prediction Surface Uncertainty Quantification in Object Detection Models for Autonomous Driving
In 2021 IEEE International Conference on Artificial Intelligence Testing (AITest). Oxford, England: IEEE, 2021.Status: Published
Prediction Surface Uncertainty Quantification in Object Detection Models for Autonomous Driving
Object detection in autonomous cars is commonly based on camera images and Lidar inputs, which are often used to train prediction models such as deep artificial neural networks for decision making for object recognition, adjusting speed, etc. A mistake in such decision making can be damaging; thus, it is vital to measure the reliability of decisions made by such prediction models via uncertainty measurement. Uncertainty, in deep learning models, is often measured for classification problems. However, deep learning models in autonomous driving are often multi-output regression models. Hence, we propose a novel method called PURE (Prediction sURface uncErtainty) for measuring prediction uncertainty of such regression models. We formulate the object recognition problem as a regression model with more than one outputs for finding object locations in a 2-dimensional camera view. For evaluation, we modified three widely-applied object recognition models (i.e., YoLo, SSD300 and SSD512) and used the KITTI, Stanford Cars, Berkeley DeepDrive, and NEXET datasets. Results showed the statistically significant negative correlation between prediction surface uncertainty and prediction accuracy suggesting that uncertainty significantly impacts the decisions made by autonomous driving.
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | 2021 IEEE International Conference on Artificial Intelligence Testing (AITest) |
| Pagination | 93-100 |
| Publisher | IEEE |
| Place Published | Oxford, England |
| DOI | 10.1109/AITEST52744.2021.00027 |
| Reprint Edition | https://arxiv.org/abs/2107.04991 |
Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
It is well-known that Quantum Computing (QC) has the potential to solve complex problems in various domains and bring breakthroughs in science and technology. Nowadays, quantum applications span over algorithms addressing optimization problems such as radiotherapy optimization, machine learning techniques, e.g., for detecting objects from images, as well as modeling and simulations, e.g., for handling uncertainties when predicting the future. The development of QC is also driven by the urgent need to solve ever-complex and large-scale problems, which current (super)computers cannot solve. QC comes right on time to bring revolutionary computation power to handle such complexity. Testing such applications, however, is a big challenge due to their radically different characteristics from their classical counterparts. This includes superposition, entanglement, and probabilistic nature of qubits. The overall ambition is to develop fundamentally new methods for automated and systematic testing of quantum programs, based on a rigorous theoretical foundation with the ultimate goal of supporting future ubiquitous services and data related to QC applications to guarantee their dependability. Also, to allow for testing complex quantum programs with a minimal amount of QC resources, we will develop novel quantum optimization algorithms. The cost-effectiveness of our methods will be demonstrated by testing quantum programs written in quantum high-level programming languages (e.g., Q#).
Funding source:
IKTPLUSS
Research Council of Norway
Partners:
The University of Malaga, Spain
The University of Maryland, USA
Durham University, UK
Publications for Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs
Journal Article
Quantum Software Testing: Challenges, Early Achievements, and Opportunities
the European Research Consortium for Informatics and Mathematics News (2022).Status: Published
Quantum Software Testing: Challenges, Early Achievements, and Opportunities
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | the European Research Consortium for Informatics and Mathematics News |
| Publisher | ERCIM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
When Software Engineering Meets Quantum Computing
Communications of ACM (2022).Status: Published
When Software Engineering Meets Quantum Computing
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | Communications of ACM |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Proceedings, refereed
QuSBT: Search-Based Testing of Quantum Programs
In 44th International Conference on Software Engineering (ICSE 2022). IEEE/ACM, 2022.Status: Published
QuSBT: Search-Based Testing of Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | 44th International Conference on Software Engineering (ICSE 2022) |
| Publisher | IEEE/ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Edited books
2021 IEEE/ACM 2nd International Workshop on Quantum Software Engineering (Q-SE)
Second ed. IEEE/ACM, 2021.Status: Published
2021 IEEE/ACM 2nd International Workshop on Quantum Software Engineering (Q-SE)
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Edited books |
| Year of Publication | 2021 |
| Edition | Second |
| Publisher | IEEE/ACM |
| ISBN Number | 978-1-6654-4462-0 |
Journal Article
Quantum Software: Model-driven or Search-driven? A Q-SE 2021 Workshop Report
ACM SIGSOFT Software Engineering Notes 46, no. 4 (2021): 23-25.Status: Published
Quantum Software: Model-driven or Search-driven? A Q-SE 2021 Workshop Report
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | ACM SIGSOFT Software Engineering Notes |
| Volume | 46 |
| Issue | 4 |
| Pagination | 23-25 |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
First International Workshop on Quantum Software Engineering (Q-SE 2020)
ACM SIGSOFT Software Engineering Notes 46, no. 2 (2021): 30-32.Status: Published
First International Workshop on Quantum Software Engineering (Q-SE 2020)
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | ACM SIGSOFT Software Engineering Notes |
| Volume | 46 |
| Issue | 2 |
| Pagination | 30-32 |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Proceedings, refereed
Application of Combinatorial Testing to Quantum Programs
In IEEE International Conference on Software Security and Reliability. IEEE, 2021.Status: Published
Application of Combinatorial Testing to Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | IEEE International Conference on Software Security and Reliability |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Quito: a Coverage-Guided Test Generator for Quantum Programs
In The 36th IEEE/ACM International Conference on Automated Software Engineering. IEEE/ACM, 2021.Status: Published
Quito: a Coverage-Guided Test Generator for Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | The 36th IEEE/ACM International Conference on Automated Software Engineering |
| Publisher | IEEE/ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Muskit: A Mutation Analysis Tool for Quantum Software Testing
In The 36th IEEE/ACM International Conference on Automated Software Engineering. IEEE/ACM, 2021.Status: Published
Muskit: A Mutation Analysis Tool for Quantum Software Testing
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | The 36th IEEE/ACM International Conference on Automated Software Engineering |
| Publisher | IEEE/ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Generating Failing Test Suites for Quantum Programs with Search
In 13th Symposium on Search-Based Software Engineering. LNCS, 2021.Status: Published
Generating Failing Test Suites for Quantum Programs with Search
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | 13th Symposium on Search-Based Software Engineering |
| Publisher | LNCS |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
The ADEPTNESS project seeks to implement and investigate a streamlined and automatic workflow that makes methods and tools to be seamlessly used during design phases as well as in operation. We will explore the generation and reuse of test cases and oracles from initial phases of the development to the system in operation and back to the laboratory for reproduction. Integrated into this workflow, unforeseen situations will also be detected in operation to enhance development models for increasing resilience. We will consider several aspects of uncertainties (such as uncertainties in the environment, uncertainty produced due to timing aspects of CPSoS, uncertainty in networks, etc.). Additionally, automatic and synchronized deployment techniques will be investigated to improve the agility of the whole workflow that covers the design-operation continuum.
Coordinator:
Mondragon Goi Eskola Politeknikoa Jose Maria Arizmendiarrieta S Coop
Funding:
Horizon 2020
Publications for Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems
Proceedings, refereed
Automating Test Oracle Generation in DevOps for Industrial Elevators
In 29th IEEE International Conference onSoftware Analysis, Evolution and Reengineering. IEEE, 2022.Status: Accepted
Automating Test Oracle Generation in DevOps for Industrial Elevators
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | 29th IEEE International Conference onSoftware Analysis, Evolution and Reengineering |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Technical reports
Uncertainty-aware Robustness Assessment of Industrial Elevator Systems
Simula Research Laboratory, 2022.Status: Published
Uncertainty-aware Robustness Assessment of Industrial Elevator Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Technical reports |
| Year of Publication | 2022 |
| Publisher | Simula Research Laboratory |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Classifying Uncertainties in an Industrial Elevator with the Cynefin Framework
Simula Research Laboratory, 2022.Status: Published
Classifying Uncertainties in an Industrial Elevator with the Cynefin Framework
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Technical reports |
| Year of Publication | 2022 |
| Publisher | Simula Research Laboratory |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Master's thesis
Simuloop - Testing Framework for an Industrial Elevator System
In University of Oslo, 2021.Status: Published
Simuloop - Testing Framework for an Industrial Elevator System
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Master's thesis |
| Year of Publication | 2021 |
| Degree awarding institution | University of Oslo |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Proceedings, refereed
Genetic Algorithm-based Testing of Industrial Elevators under Passenger Uncertainty
In The 32nd International Symposium on Software Reliability Engineering (ISSRE 2021). IEEE, 2021.Status: Published
Genetic Algorithm-based Testing of Industrial Elevators under Passenger Uncertainty
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | The 32nd International Symposium on Software Reliability Engineering (ISSRE 2021) |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Microservices for Continuous Deployment, Monitoring and Validation in Cyber-Physical Systems: an Industrial Case Study for Elevators Systems
In ICSA Software Architecture in Practice (SAIP) . IEEE, 2021.Status: Published
Microservices for Continuous Deployment, Monitoring and Validation in Cyber-Physical Systems: an Industrial Case Study for Elevators Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | ICSA Software Architecture in Practice (SAIP) |
| Pagination | 46-53 |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
In International Symposium On Leveraging Applications of Formal Methods, Verification and Validation. Vol. 12479. LNCS, Springer, Cham, 2021.Status: Published
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | International Symposium On Leveraging Applications of Formal Methods, Verification and Validation |
| Volume | 12479 |
| Pagination | 54-71 |
| Publisher | LNCS, Springer, Cham |
| DOI | 10.1007/978-3-030-83723-5_5 |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Talks, contributed
Microservices for Continuous Deployment, Monitoring and Validation in Cyber-Physical Systems: an Industrial Case Study for Elevators Systems
In ICSA Software Architecture in Practice (SAIP). IEEE, 2021.Status: Published
Microservices for Continuous Deployment, Monitoring and Validation in Cyber-Physical Systems: an Industrial Case Study for Elevators Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | ICSA Software Architecture in Practice (SAIP) |
| Publisher | IEEE |
| Type of Talk | Conference Paper |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Prediction Surface Uncertainty Quantification in Object Detection Models for Autonomous Driving
In The Third IEEE International Conference On Artificial Intelligence Testing. IEEE, 2021.Status: Published
Prediction Surface Uncertainty Quantification in Object Detection Models for Autonomous Driving
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | The Third IEEE International Conference On Artificial Intelligence Testing |
| Publisher | IEEE |
| Type of Talk | Conference Paper |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
In International Symposium On Leveraging Applications of Formal Methods, Verification and Validation. LNCS, 2021.Status: Published
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | International Symposium On Leveraging Applications of Formal Methods, Verification and Validation |
| Publisher | LNCS |
| Type of Talk | Conference Paper |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
AIT4CR: AI-Powered Testing Infrastructure for Cancer Registry System
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
I am a Postdoctoral Fellow working on the innovation project "AIT4CR: AI-Powered Testing Infrastructure for Cancer Registry System", in collaboration with the Cancer Registry of Norway.
Prior to Simula, I was a PhD student at University of Zurich, where I received my doctoral degree in April 2021, defending my dissertation "Deliberate Microbenchmarking of Software Systems".
Originally, I am from Vienna, Austria.
The innovation planned is a state-of-the-art test infrastructure including new testing techniques utilizing algorithms from machine learning and evolutionary computation (i.e., two sub-areas of AI) to support cost-effective and systematic testing of the Cancer Registration Support System (CaReSS) to significantly improve its quality, and quality of data and statistics it produces, by dealing with the continuous evolution and unpredictable behavior of machine learning algorithms. This will positively affect all its end users, including researchers, patients, doctors, and government officials. The deployment of the new testing infrastructure at the Cancer Registry of Norway (CRN) will contribute to the further digitalization at the CRN, leading to significant improvements in the current testing practice at CRN. With minimal effort, the testing infrastructure has the potential to be deployed to other health registries in Norway and cancer registries in the world.
Funding source
-
The Research Council of Norway
Partners
- Simula Research Laboratory
- Cancer Registry of Norway
Project leaders
- Simula Research Laboratory: Shaukat Ali and Tao Yue
- Cancer Registry of Norway: Jan F. Nygård
Publications for AIT4CR: AI-Powered Testing Infrastructure for Cancer Registry System
Talks, invited
Variability of Microbenchmark Results and How to Deal with It
In Chalmers, Software Engineering Division, University of Gothenburg, Sweden, 2021.Status: Published
Variability of Microbenchmark Results and How to Deal with It
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, AIT4CR: AI-Powered Testing Infrastructure for Cancer Registry System |
| Publication Type | Talks, invited |
| Year of Publication | 2021 |
| Location of Talk | Chalmers, Software Engineering Division, University of Gothenburg, Sweden |
I am a Postdoctoral Fellow working on the innovation project "AIT4CR: AI-Powered Testing Infrastructure for Cancer Registry System", in collaboration with the Cancer Registry of Norway.
Prior to Simula, I was a PhD student at University of Zurich, where I received my doctoral degree in April 2021, defending my dissertation "Deliberate Microbenchmarking of Software Systems".
Originally, I am from Vienna, Austria.
WTT4Oslo: Improving Quality of IoT-based Welfare Technology Solutions in the City of Oslo
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Welfare Technology is an emerging concept that refers to the use of new technology solutions within the fields of nursing, health, and social care. Preliminary studies have shown that Welfare Technology Solutions (WTSs) provide significant gains both in terms of increased quality of life and reduced levels of health service consumption by citizens.
The objective of this innovation is to improve the quality of the current and future WTSs of the City of Oslo (CoO), enhance their safety, privacy and security, and improve the testing practice at CoO, via automated, systematic, and cost-effective testing, for enhancing the overall quality of welfare services provided by the WTSs to end-users. The proposed testing solution is expected to reduce testing time and improve the overall efficiency of developing WTSs, while it will become accessible to other municipalities and counties in Norway in the future.
Funding
- The Research Council of Norway
Partners
- Simula Research Laboratory
- City of Oslo
Department of Engineering Complex Software Systems
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
There is an increased reliance on our daily life on Complex Software Systems such as smart Cyber-Physical Systems and Internet of Things. Such systems are used in many safety/mission-critical such as healthcare, autonomous cars, boats and underwater vehicles, maritime, energy, and smart roads, grids, buildings, and cities. Such systems are progressively turning into complex, heterogeneous, open, networked and “smart” systems that are composed of agents, sensors, actuators, information networks, and dedicated middleware and infrastructures. Given these complexities and high interaction with the physical environment, new, smart and efficient Software Engineering (SE) paradigms are needed to design, develop, test, and maintain such systems to ensure their dependability (e.g., safety) as well as other relevant quality attributes such as security and privacy.
Such systems are typically built by integrating various physical units, with known and uncertain assumptions on their physical operating environments, future deployments, middleware, and infrastructures. However, as the first dimension of the complexity and challenges, during a real operation of such a system, there is no guarantee that these (often flawed) assumptions will hold, which makes such a system vulnerable to unforeseen loopholes concerning one or more of the quality attributes. Therefore, a novel SE paradigm is expected to manage and handle uncertainty information (e.g., uncertain assumptions) throughout complex system development lifecycles in an intelligent manner. The second dimension is due to the increased deployment of Artificial Intelligence (AI) components for all kinds of reasoning and recognition tasks in modern complex systems. Current system development tools and techniques are not sufficient to cope with this complexity and challenge, and therefore a radical shift moving toward a novel SE paradigm is urgently expected. The third dimension concerns with the multi-disciplinary nature of developing, maintaining, and operating complex software systems, which concerns not only technical aspects but also ethical principles and address societal concerns especially when developing (AI-enabled) smart systems.
The aim of the Department of Engineering Complex Software Systems (ComplexSE) is to develop novel SE paradigms to address the challenges mentioned above to facilitate engineering of modern complex software systems. The ComplexSE department establishes itself with several well-established SE sub-disciplines including Model-Based Engineering, Model-Based Testing, Search-based Software Engineering, and Empirical Software Engineering.
Publications for Department of Engineering Complex Software Systems
Book Chapter
Human-in-the-Loop Enhanced COVID-19 Detection in Transfer Learning-Based CNN Models
In Computational Intelligence for COVID-19 and Future Pandemics . Springer, 2022.Status: Published
Human-in-the-Loop Enhanced COVID-19 Detection in Transfer Learning-Based CNN Models
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Book Chapter |
| Year of Publication | 2022 |
| Book Title | Computational Intelligence for COVID-19 and Future Pandemics |
| Publisher | Springer |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Journal Article
Learning Configurations of Operating Environment of Autonomous Vehicles to Maximize their Collisions
IEEE Transactions on Software Engineering (2022).Status: Published
Learning Configurations of Operating Environment of Autonomous Vehicles to Maximize their Collisions
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | IEEE Transactions on Software Engineering |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
On the Preferences of Quality Indicators for Multi-Objective Search Algorithms in Search-Based Software Engineering
The Empirical Software Engineering Journal (EMSE) (2022).Status: Published
On the Preferences of Quality Indicators for Multi-Objective Search Algorithms in Search-Based Software Engineering
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | The Empirical Software Engineering Journal (EMSE) |
| Publisher | Springer |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty-Aware Prediction Validator in Deep Learning Models for Cyber-Physical System Data
ACM Transactions on Software Engineering and Methodology (TOSEM) (2022).Status: Published
Uncertainty-Aware Prediction Validator in Deep Learning Models for Cyber-Physical System Data
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | ACM Transactions on Software Engineering and Methodology (TOSEM) |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Quantum Software Testing: Challenges, Early Achievements, and Opportunities
the European Research Consortium for Informatics and Mathematics News (2022).Status: Published
Quantum Software Testing: Challenges, Early Achievements, and Opportunities
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | the European Research Consortium for Informatics and Mathematics News |
| Publisher | ERCIM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
When Software Engineering Meets Quantum Computing
Communications of ACM (2022).Status: Published
When Software Engineering Meets Quantum Computing
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | Communications of ACM |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Proceedings, refereed
Are Elevator Software Robust Against Uncertainties? Results and Experiences from an Industrial Case Study
In The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2022.Status: Accepted
Are Elevator Software Robust Against Uncertainties? Results and Experiences from an Industrial Case Study
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) |
Uncertainty-Aware Transfer Learning to Evolve Digital Twins for Industrial Elevators
In The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2022.Status: Accepted
Uncertainty-Aware Transfer Learning to Evolve Digital Twins for Industrial Elevators
| Afilliation | Software Engineering |
| Project(s) | Co-tester: Collective-Adaptive Testing of Coevolving Autonomous Cyber-Physical Systems of Systems under Uncertainty, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) |
Investigating Cause-Effect Graphs
In Q-SE 2022 (WS). Q-SE 2022, 2022.Status: Published
Investigating Cause-Effect Graphs
Cause-effect graphs have shown promising results in identifying relations among causes and effects of classical software systems, followed by designing effective test cases from them. Towards this end, we investigate the use of cause-effect graphs for quantum programs. Classical cause-effect graphs apply classical logic (e.g., AND, OR) to express these relations, which might not be practical for describing similar relations in quantum programs due to quantum superposition and entanglement. Thus, we propose an extension of cause-effect graphs, where quantum logic inspired functions (e.g., Hadamard) and their generalizations are defined and applied. Moreover, we present a metamodel describing various forms of cause-effect graphs. Finally, we demonstrate a possible method for generating test cases from a quantum cause-effect graph applied to a Bell state quantum program. Lastly, the design and utility of the resulting testing method is discussed, along with future prospects for general quantum cause-effect graphs.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Quantum Software Engineering Project |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | Q-SE 2022 (WS) |
| Publisher | Q-SE 2022 |
Automating Test Oracle Generation in DevOps for Industrial Elevators
In 29th IEEE International Conference onSoftware Analysis, Evolution and Reengineering. IEEE, 2022.Status: Accepted
Automating Test Oracle Generation in DevOps for Industrial Elevators
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | 29th IEEE International Conference onSoftware Analysis, Evolution and Reengineering |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Publications
Proceedings, refereed
Are Elevator Software Robust Against Uncertainties? Results and Experiences from an Industrial Case Study
In The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2022.Status: Accepted
Are Elevator Software Robust Against Uncertainties? Results and Experiences from an Industrial Case Study
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) |
Automating Test Oracle Generation in DevOps for Industrial Elevators
In 29th IEEE International Conference onSoftware Analysis, Evolution and Reengineering. IEEE, 2022.Status: Accepted
Automating Test Oracle Generation in DevOps for Industrial Elevators
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | 29th IEEE International Conference onSoftware Analysis, Evolution and Reengineering |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Investigating Cause-Effect Graphs
In Q-SE 2022 (WS). Q-SE 2022, 2022.Status: Published
Investigating Cause-Effect Graphs
Cause-effect graphs have shown promising results in identifying relations among causes and effects of classical software systems, followed by designing effective test cases from them. Towards this end, we investigate the use of cause-effect graphs for quantum programs. Classical cause-effect graphs apply classical logic (e.g., AND, OR) to express these relations, which might not be practical for describing similar relations in quantum programs due to quantum superposition and entanglement. Thus, we propose an extension of cause-effect graphs, where quantum logic inspired functions (e.g., Hadamard) and their generalizations are defined and applied. Moreover, we present a metamodel describing various forms of cause-effect graphs. Finally, we demonstrate a possible method for generating test cases from a quantum cause-effect graph applied to a Bell state quantum program. Lastly, the design and utility of the resulting testing method is discussed, along with future prospects for general quantum cause-effect graphs.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Quantum Software Engineering Project |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | Q-SE 2022 (WS) |
| Publisher | Q-SE 2022 |
QuSBT: Search-Based Testing of Quantum Programs
In 44th International Conference on Software Engineering (ICSE 2022). IEEE/ACM, 2022.Status: Published
QuSBT: Search-Based Testing of Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | 44th International Conference on Software Engineering (ICSE 2022) |
| Publisher | IEEE/ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty-Aware Transfer Learning to Evolve Digital Twins for Industrial Elevators
In The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2022.Status: Accepted
Uncertainty-Aware Transfer Learning to Evolve Digital Twins for Industrial Elevators
| Afilliation | Software Engineering |
| Project(s) | Co-tester: Collective-Adaptive Testing of Coevolving Autonomous Cyber-Physical Systems of Systems under Uncertainty, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2022 |
| Conference Name | The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) |
Technical reports
Classifying Uncertainties in an Industrial Elevator with the Cynefin Framework
Simula Research Laboratory, 2022.Status: Published
Classifying Uncertainties in an Industrial Elevator with the Cynefin Framework
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Technical reports |
| Year of Publication | 2022 |
| Publisher | Simula Research Laboratory |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Digital Twin-based Anomaly Detection with Curriculum Learning in Cyber-physical Systems
Simula Research Laboratory, 2022.Status: Published
Digital Twin-based Anomaly Detection with Curriculum Learning in Cyber-physical Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Digital Twin-Enabled Operation Time Analyses |
| Publication Type | Technical reports |
| Year of Publication | 2022 |
| Publisher | Simula Research Laboratory |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty-aware Robustness Assessment of Industrial Elevator Systems
Simula Research Laboratory, 2022.Status: Published
Uncertainty-aware Robustness Assessment of Industrial Elevator Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Technical reports |
| Year of Publication | 2022 |
| Publisher | Simula Research Laboratory |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Journal Article
Learning Configurations of Operating Environment of Autonomous Vehicles to Maximize their Collisions
IEEE Transactions on Software Engineering (2022).Status: Published
Learning Configurations of Operating Environment of Autonomous Vehicles to Maximize their Collisions
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | IEEE Transactions on Software Engineering |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
On the Preferences of Quality Indicators for Multi-Objective Search Algorithms in Search-Based Software Engineering
The Empirical Software Engineering Journal (EMSE) (2022).Status: Published
On the Preferences of Quality Indicators for Multi-Objective Search Algorithms in Search-Based Software Engineering
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | The Empirical Software Engineering Journal (EMSE) |
| Publisher | Springer |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Quantum Software Testing: Challenges, Early Achievements, and Opportunities
the European Research Consortium for Informatics and Mathematics News (2022).Status: Published
Quantum Software Testing: Challenges, Early Achievements, and Opportunities
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | the European Research Consortium for Informatics and Mathematics News |
| Publisher | ERCIM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty-Aware Prediction Validator in Deep Learning Models for Cyber-Physical System Data
ACM Transactions on Software Engineering and Methodology (TOSEM) (2022).Status: Published
Uncertainty-Aware Prediction Validator in Deep Learning Models for Cyber-Physical System Data
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | ACM Transactions on Software Engineering and Methodology (TOSEM) |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
When Software Engineering Meets Quantum Computing
Communications of ACM (2022).Status: Published
When Software Engineering Meets Quantum Computing
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Journal Article |
| Year of Publication | 2022 |
| Journal | Communications of ACM |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Talks, contributed
Learning Digital Twin Models
In Model-Driven Engineering of Digital Twins Seminar at Dagstuhl. Schloss Dagstuhl, Germany, 2022.Status: Published
Learning Digital Twin Models
Given that operational cyber-physical systems (CPS) produce continuous data, a complementary approach to model-based engineering is to learn digital twins models with machine learning techniques and providing functionalities such as predictions and anomaly detection.
This talk will start with presenting an opinion on the next generation of digital twins (Quantum Digital Twins), where some aspects of digital twins will be implemented as quantum software and executed on quantum computers, e.g., for simulating the physical environment that can be realistically simulated with quantum-mechanical principles.
Followed by this opinion, the talk will present some recent works on learning digital twins from historical data and continuous updates of digital twins with continuous data from operational CPS. Various machine learning techniques were applied, such as generative adversarial networks, curriculum learning, and transfer learning to learn digital twins. The digital twins were built for use cases from the transportation domain and water distribution/treatment plants. These digital twins were focused on anomaly detection and waiting time predictions.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Co-tester: Collective-Adaptive Testing of Coevolving Autonomous Cyber-Physical Systems of Systems under Uncertainty |
| Publication Type | Talks, contributed |
| Year of Publication | 2022 |
| Location of Talk | Model-Driven Engineering of Digital Twins Seminar at Dagstuhl |
| Publisher | Schloss Dagstuhl, Germany |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Edited books
2021 IEEE/ACM 2nd International Workshop on Quantum Software Engineering (Q-SE)
Second ed. IEEE/ACM, 2021.Status: Published
2021 IEEE/ACM 2nd International Workshop on Quantum Software Engineering (Q-SE)
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Edited books |
| Year of Publication | 2021 |
| Edition | Second |
| Publisher | IEEE/ACM |
| ISBN Number | 978-1-6654-4462-0 |
Proceedings, refereed
Analyzing the Impact of Product Configuration Variations on Advanced Driver Assistance Systems with Search
In The Genetic and Evolutionary Computation Conference. ACM, 2021.Status: Published
Analyzing the Impact of Product Configuration Variations on Advanced Driver Assistance Systems with Search
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | The Genetic and Evolutionary Computation Conference |
| Pagination | 1106–1114 |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Anomaly Detection with Digital Twin in Cyber-Physical Systems
In 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST). IEEE, 2021.Status: Published
Anomaly Detection with Digital Twin in Cyber-Physical Systems
Cyber-Physical Systems (CPSs) are susceptible to various anomalies during their operations. Thus, it is important to detect such anomalies. Detecting such anomalies is challenging since it is uncertain when and where anomalies can happen. To this end, we present a novel approach called Anomaly deTection with digiTAl twIN (ATTAIN), which continuously and automatically builds a digital twin with live data obtained from a CPS for anomaly detection. ATTAIN builds a Timed Automaton Machine (TAM) as the digital representation of the CPS, and implements a Generative Adversarial Network (GAN) to detect anomalies. GAN uses a GCN-LSTM-based module as a generator, which can capture temporal and spatial characteristics of the input data and learn to produce realistic unlabeled fake samples. TAM labels these fake samples, which are then fed into a discriminator along with real labeled samples. After training, the discriminator is capable of distinguishing anomalous data from normal data with a high F1 score. To evaluate our approach, we used three publicly available datasets collected from three CPS testbeds. Evaluation results show that ATTAIN improved the performance of two state-of-art anomaly detection methods by 2.413\%, 8.487\% and 5.438\% on average on the three datasets, respectively. Moreover, ATTAIN achieved on average 8.39\% increase in the anomaly detection capability with digital twins as compared with an approach of not using digital twins.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST) |
| Publisher | IEEE |
| DOI | 10.1109/ICST49551.2021.00031 |
Application of Combinatorial Testing to Quantum Programs
In IEEE International Conference on Software Security and Reliability. IEEE, 2021.Status: Published
Application of Combinatorial Testing to Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | IEEE International Conference on Software Security and Reliability |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
In IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE, 2021.Status: Published
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | IEEE International Conference on Software Testing, Verification and Validation (ICST) |
| Pagination | 13-23 |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Generating Failing Test Suites for Quantum Programs with Search
In 13th Symposium on Search-Based Software Engineering. LNCS, 2021.Status: Published
Generating Failing Test Suites for Quantum Programs with Search
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | 13th Symposium on Search-Based Software Engineering |
| Publisher | LNCS |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Genetic Algorithm-based Testing of Industrial Elevators under Passenger Uncertainty
In The 32nd International Symposium on Software Reliability Engineering (ISSRE 2021). IEEE, 2021.Status: Published
Genetic Algorithm-based Testing of Industrial Elevators under Passenger Uncertainty
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | The 32nd International Symposium on Software Reliability Engineering (ISSRE 2021) |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Microservices for Continuous Deployment, Monitoring and Validation in Cyber-Physical Systems: an Industrial Case Study for Elevators Systems
In ICSA Software Architecture in Practice (SAIP) . IEEE, 2021.Status: Published
Microservices for Continuous Deployment, Monitoring and Validation in Cyber-Physical Systems: an Industrial Case Study for Elevators Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | ICSA Software Architecture in Practice (SAIP) |
| Pagination | 46-53 |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Muskit: A Mutation Analysis Tool for Quantum Software Testing
In The 36th IEEE/ACM International Conference on Automated Software Engineering. IEEE/ACM, 2021.Status: Published
Muskit: A Mutation Analysis Tool for Quantum Software Testing
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | The 36th IEEE/ACM International Conference on Automated Software Engineering |
| Publisher | IEEE/ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Parameter-Based Testing and Debugging of Autonomous Driving Systems
In 2021 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2021.Status: Published
Parameter-Based Testing and Debugging of Autonomous Driving Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | 2021 IEEE Intelligent Vehicles Symposium (IV) |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Prediction Surface Uncertainty Quantification in Object Detection Models for Autonomous Driving
In 2021 IEEE International Conference on Artificial Intelligence Testing (AITest). Oxford, England: IEEE, 2021.Status: Published
Prediction Surface Uncertainty Quantification in Object Detection Models for Autonomous Driving
Object detection in autonomous cars is commonly based on camera images and Lidar inputs, which are often used to train prediction models such as deep artificial neural networks for decision making for object recognition, adjusting speed, etc. A mistake in such decision making can be damaging; thus, it is vital to measure the reliability of decisions made by such prediction models via uncertainty measurement. Uncertainty, in deep learning models, is often measured for classification problems. However, deep learning models in autonomous driving are often multi-output regression models. Hence, we propose a novel method called PURE (Prediction sURface uncErtainty) for measuring prediction uncertainty of such regression models. We formulate the object recognition problem as a regression model with more than one outputs for finding object locations in a 2-dimensional camera view. For evaluation, we modified three widely-applied object recognition models (i.e., YoLo, SSD300 and SSD512) and used the KITTI, Stanford Cars, Berkeley DeepDrive, and NEXET datasets. Results showed the statistically significant negative correlation between prediction surface uncertainty and prediction accuracy suggesting that uncertainty significantly impacts the decisions made by autonomous driving.
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | 2021 IEEE International Conference on Artificial Intelligence Testing (AITest) |
| Pagination | 93-100 |
| Publisher | IEEE |
| Place Published | Oxford, England |
| DOI | 10.1109/AITEST52744.2021.00027 |
| Reprint Edition | https://arxiv.org/abs/2107.04991 |
Quito: a Coverage-Guided Test Generator for Quantum Programs
In The 36th IEEE/ACM International Conference on Automated Software Engineering. IEEE/ACM, 2021.Status: Published
Quito: a Coverage-Guided Test Generator for Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | The 36th IEEE/ACM International Conference on Automated Software Engineering |
| Publisher | IEEE/ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Restricted Natural Language and Model-based Adaptive Test Generation for Autonomous Driving
In ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS). ACM/IEEE, 2021.Status: Published
Restricted Natural Language and Model-based Adaptive Test Generation for Autonomous Driving
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS) |
| Publisher | ACM/IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Search-based Selection and Prioritization of Test Scenarios for Autonomous Driving Systems
In 13th Symposium on Search-Based Software Engineering. LNCS Springer, 2021.Status: Published
Search-based Selection and Prioritization of Test Scenarios for Autonomous Driving Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | 13th Symposium on Search-Based Software Engineering |
| Publisher | LNCS Springer |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
In International Symposium On Leveraging Applications of Formal Methods, Verification and Validation. Vol. 12479. LNCS, Springer, Cham, 2021.Status: Published
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2021 |
| Conference Name | International Symposium On Leveraging Applications of Formal Methods, Verification and Validation |
| Volume | 12479 |
| Pagination | 54-71 |
| Publisher | LNCS, Springer, Cham |
| DOI | 10.1007/978-3-030-83723-5_5 |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Talks, contributed
Anomaly Detection with Digital Twin in Cyber-Physical Systems
In IEEE International Conference on Software Testing, Verification and Validation (ICST), 2021.Status: Published
Anomaly Detection with Digital Twin in Cyber-Physical Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | IEEE International Conference on Software Testing, Verification and Validation (ICST) |
| Type of Talk | Paper Presentation |
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
In IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE, 2021.Status: Published
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | IEEE International Conference on Software Testing, Verification and Validation (ICST) |
| Publisher | IEEE |
| Type of Talk | Conference Paper |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Do Quality Indicators Prefer Particular Multi-Objective Search Algorithms in Search-Based Software Engineering? (Hot Off the Press)
In The Genetic and Evolutionary Computation Conference, 2021.Status: Published
Do Quality Indicators Prefer Particular Multi-Objective Search Algorithms in Search-Based Software Engineering? (Hot Off the Press)
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | The Genetic and Evolutionary Computation Conference |
| Type of Talk | Hot Off the Press |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Genetic Algorithm-based Testing of Industrial Elevators under Passenger Uncertainty
In The 32nd International Symposium on Software Reliability Engineering (ISSRE 2021). IEEE, 2021.Status: Published
Genetic Algorithm-based Testing of Industrial Elevators under Passenger Uncertainty
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | The 32nd International Symposium on Software Reliability Engineering (ISSRE 2021) |
| Publisher | IEEE |
| Type of Talk | Conference Paper |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Quality Indicators in Search-Based Software Engineering: An Empirical Evaluation
In 43rd International Conference on Software Engineering. IEEE, 2021.Status: Published
Quality Indicators in Search-Based Software Engineering: An Empirical Evaluation
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | 43rd International Conference on Software Engineering |
| Publisher | IEEE |
| Type of Talk | Journal First Presentation |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
In International Symposium On Leveraging Applications of Formal Methods, Verification and Validation. LNCS, 2021.Status: Published
Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2021 |
| Location of Talk | International Symposium On Leveraging Applications of Formal Methods, Verification and Validation |
| Publisher | LNCS |
| Type of Talk | Conference Paper |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Talk, keynote
Application of Artificial Intelligence Techniques for Requirements Optimization, Learning, and Evolution for Cyber-Physical Systems
In An RE'21 Workshop on Environment-Driven Requirements Engineering, 2021.Status: Published
Application of Artificial Intelligence Techniques for Requirements Optimization, Learning, and Evolution for Cyber-Physical Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talk, keynote |
| Year of Publication | 2021 |
| Location of Talk | An RE'21 Workshop on Environment-Driven Requirements Engineering |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Talks, invited
Automated, Systematic, and Optimized Testing of Quantum Programs
In Gemini Center, 2021.Status: Published
Automated, Systematic, and Optimized Testing of Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Talks, invited |
| Year of Publication | 2021 |
| Location of Talk | Gemini Center |
| Type of Talk | Invited Talk |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Black Box Testing of Quantum Programs
In QC Talks, University of Porto, Portugal, 2021.Status: Published
Black Box Testing of Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Talks, invited |
| Year of Publication | 2021 |
| Location of Talk | QC Talks, University of Porto, Portugal |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
The Role of Quantum Software Engineering in the Second Quantum Revolution
In Communications of the ACM Europe Region Special Section Virtual Workshop, 2021.Status: Published
The Role of Quantum Software Engineering in the Second Quantum Revolution
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Talks, invited |
| Year of Publication | 2021 |
| Location of Talk | Communications of the ACM Europe Region Special Section Virtual Workshop |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Journal Article
First International Workshop on Quantum Software Engineering (Q-SE 2020)
ACM SIGSOFT Software Engineering Notes 46, no. 2 (2021): 30-32.Status: Published
First International Workshop on Quantum Software Engineering (Q-SE 2020)
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | ACM SIGSOFT Software Engineering Notes |
| Volume | 46 |
| Issue | 2 |
| Pagination | 30-32 |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Quantum Software: Model-driven or Search-driven? A Q-SE 2021 Workshop Report
ACM SIGSOFT Software Engineering Notes 46, no. 4 (2021): 23-25.Status: Published
Quantum Software: Model-driven or Search-driven? A Q-SE 2021 Workshop Report
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | ACM SIGSOFT Software Engineering Notes |
| Volume | 46 |
| Issue | 4 |
| Pagination | 23-25 |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Recommending Faulty Configurations for Interacting Systems Under Test Using Multi-Objective Search
ACM Transactions on Software Engineering and Methodology 30, no. 4 (2021): 1-36.Status: Published
Recommending Faulty Configurations for Interacting Systems Under Test Using Multi-Objective Search
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | ACM Transactions on Software Engineering and Methodology |
| Volume | 30 |
| Issue | 4 |
| Pagination | 1-36 |
| Publisher | ACM |
| DOI | 10.1145/3464939 |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Testing Self-Healing Cyber-Physical Systems under Uncertainty with Reinforcement Learning: An Empirical Study
Empirical Software Engineering 26, no. 3 (2021): 52.Status: Published
Testing Self-Healing Cyber-Physical Systems under Uncertainty with Reinforcement Learning: An Empirical Study
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2021 |
| Journal | Empirical Software Engineering |
| Volume | 26 |
| Issue | 3 |
| Pagination | 52 |
| Publisher | Springer |
| DOI | 10.1007/s10664-021-09941-z |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Technical reports
Muskit: A Mutation Analysis Tool for Quantum Software Testing
Technical Report, Simula Research Laboratory, 2021.Status: Published
Muskit: A Mutation Analysis Tool for Quantum Software Testing
Given that quantum software testing is a new area of research, there is a lack of benchmark programs and bug repositories to assess the effectiveness of testing techniques. To this end, quantum mutation analysis focuses on systematically generating faulty versions of Quantum Programs (QPs), called mutants, using mutation operators. Such mutants can be used as benchmarks to assess the quality of test cases in a test suite. Thus, we present muskit -- a quantum mutation analysis tool for QPs coded in IBM's Qiskit language. muskit defines mutation operators on gates of QPs and selection criteria to reduce the number of mutants to generate. Moreover, it allows for the execution of test cases on mutants and the generation of results for test analyses. muskit is provided as a command-line interface, GUI, and web application. We validated muskit by using it to generate and execute mutants for four QPs.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Technical reports |
| Year of Publication | 2021 |
| Publisher | Technical Report, Simula Research Laboratory |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Journal Article
A framework for automated multi‑stage and multi‑step product confguration of cyber‑physical systems
Software and Systems Modeling (SoSym) 19, no. 4 (2020): 1-55.Status: Published
A framework for automated multi‑stage and multi‑step product confguration of cyber‑physical systems
Product line engineering (PLE) has been employed to large-scale cyber-physical systems (CPSs) to provide customization based on users’ needs. A PLE methodology can be characterized by its support for capturing and managing the abstractions as commonalities and variabilities and the automation of the confguration process for efective selection and customization of reusable artifacts. The automation of a confguration process heavily relies on the captured abstractions and formally specifed constraints using a well-defned modeling methodology. Based on the results of our previous work and a thorough literature review, in this paper, we propose a conceptual framework to support multi-stage and multi-step automated product confguration of CPSs, including a comprehensive classifcation of constraints and a list of automated functionalities of a CPS confguration solution. Such a framework can serve as a guide for researchers and practitioners to evaluate an existing CPS PLE solution or devise a novel CPS PLE solution. To validate the framework, we conducted three real-world case studies. Results show that the framework fulflls all the requirements of the case studies in terms of capturing and managing variabilities and constraints. Results of the literature review indicate that the framework covers all the functionalities concerned by the literature, suggesting that the framework is complete for enabling the maximum automation of confguration in CPS PLE.
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | Software and Systems Modeling (SoSym) |
| Volume | 19 |
| Issue | 4 |
| Pagination | 1-55 |
| Date Published | 06/2020 |
| Publisher | Springer |
| Keywords | Automated configuration, Constraint classification, Cyber-Physical Systems, Multi-stage and multi-step configuration process, Product Line Engineering, Real-world case studies, Variability Modeling |
| DOI | 10.1007/s10270-020-00803-8 |
Pattern-based Interactive Configuration Derivation for Cyber-Physical System Product Lines
ACM Transactions on Cyber-Physical Systems 4, no. 4 (2020): 24.Status: Published
Pattern-based Interactive Configuration Derivation for Cyber-Physical System Product Lines
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Cyber-Physical Systems |
| Volume | 4 |
| Issue | 4 |
| Pagination | 24 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 2378-962X |
| URL | https://doi.org/10.1145/3389397 |
| DOI | 10.1145/3389397 |
Quality indicators in search-based software engineering: an empirical evaluation
ACM Transactions on Software Engineering and Methodology 29 (2020).Status: Published
Quality indicators in search-based software engineering: an empirical evaluation
Search-Based Software Engineering (SBSE) researchers who apply multi-objective search algorithms (MOSAs) often assess the quality of solutions produced by MOSAs with one or more quality indicators (QIs). However, SBSE lacks evidence providing insights on commonly used QIs, especially about agreements among them and their relations with SBSE problems and applied MOSAs. Such evidence about QIs agreements is essential to understand relationships among QIs, identify redundant QIs, and consequently devise guidelines for SBSE researchers to select appropriate QIs for their specific contexts. To this end, we conducted an extensive empirical evaluation to provide insights on commonly used QIs in the context of SBSE, by studying agreements among QIs with and without considering differences of SBSE problems and MOSAs. In addition, by defining a systematic process based on three common ways of comparing MOSAs in SBSE, we present additional observations that were automatically produced based on the results of our empirical evaluation. These observations can be used by SBSE researchers to gain a better understanding of the commonly used QIs in SBSE, in particular, regarding their agreements. Finally, based on the results, we also provide a set of guidelines for SBSE researchers to select appropriate QIs for their particular context.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Software Engineering and Methodology |
| Volume | 29 |
| Number | 2 |
| Date Published | 04/2020 |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty-wise Requirements Prioritization with Search
ACM Transactions on Software Engineering and Methodology 30, no. 1 (2020): 1-54.Status: Published
Uncertainty-wise Requirements Prioritization with Search
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2020 |
| Journal | ACM Transactions on Software Engineering and Methodology |
| Volume | 30 |
| Issue | 1 |
| Pagination | 1-54 |
| Date Published | December 2020 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| ISSN | 1049-331X |
| URL | https://doi.org/10.1145/3408301 |
| DOI | 10.1145/3408301 |
Proceedings, refereed
Do Quality Indicators Prefer Particular Multi-Objective Search Algorithms in Search-Based Software Engineering?
In 12th Symposium on Search-Based Software Engineering. LNCS, 2020.Status: Published
Do Quality Indicators Prefer Particular Multi-Objective Search Algorithms in Search-Based Software Engineering?
In Search-Based Software Engineering (SBSE), users typically select a set of Multi-Objective Search Algorithms (MOSAs) for their experiments without any justification, or they simply choose an MOSA because of its popularity (e.g., NSGA-II). On the other hand, users know certain characteristics of solutions they are interested in. Such characteristics are typically measured with Quality Indicators (QIs) that are commonly used to evaluate the quality of solutions produced by an MOSA. Consequently, these QIs are often employed to empirically evaluate a set of MOSAs for a particular search problem to find the best MOSA. Thus, to guide SBSE users in choosing an MOSA that represents the solutions measured by a specific QI they are interested in, we present an empirical evaluation with a set of SBSE problems to study the relationships among commonly used QIs and MOSAs in SBSE. Our aim, by studying such relationships, is to identify whether there are certain characteristics of a QI because of which it prefers a certain MOSA. Such preferences are then used to provide insights and suggestions to SBSE users in selecting an MOSA, given that they know which quality aspects of solutions they are looking for.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | 12th Symposium on Search-Based Software Engineering |
| Publisher | LNCS |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems
In IEEE International Conference on Software Testing, Verification and Validation (ICST) 2020. IEEE, 2020.Status: Published
Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | IEEE International Conference on Software Testing, Verification and Validation (ICST) 2020 |
| Pagination | 375-386 |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Modeling Quantum Programs: Challenges, Initial Results, and Research Directions
In 1st International Workshop on Architectures and Paradigms for Engineering Quantum Software (APEQS). ACM, 2020.Status: Published
Modeling Quantum Programs: Challenges, Initial Results, and Research Directions
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | 1st International Workshop on Architectures and Paradigms for Engineering Quantum Software (APEQS) |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Simultaneously Searching and Solving Multiple Avoidable Collisions for Testing Autonomous Driving Systems
In The Genetic and Evolutionary Computation Conference (GECCO). ACM, 2020.Status: Published
Simultaneously Searching and Solving Multiple Avoidable Collisions for Testing Autonomous Driving Systems
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2020 |
| Conference Name | The Genetic and Evolutionary Computation Conference (GECCO) |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Talks, contributed
Empowering Model-based Engineering with the CynefinFramework for Systematic Uncertainty Thinking
In 1st Uncertainty in Modeling Workshop 2020, Co-located with the ACM/IEEE 213rd International Conference on Model Driven Engineering Languages and Systems, 2020.Status: Published
Empowering Model-based Engineering with the CynefinFramework for Systematic Uncertainty Thinking
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | 1st Uncertainty in Modeling Workshop 2020, Co-located with the ACM/IEEE 213rd International Conference on Model Driven Engineering Languages and Systems |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
In 5th Workshop on Monitoring and Testing of Cyber-Physical Systems (MT-CPS), 2020.Status: Published
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | 5th Workshop on Monitoring and Testing of Cyber-Physical Systems (MT-CPS) |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
In IEEE International Conference on Software Testing, Verification and Validation (ICST 2020), 2020.Status: Published
Modeling Foundations for Executable Model-Based Testing of Self-Healing Cyber-Physical Systems
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | IEEE International Conference on Software Testing, Verification and Validation (ICST 2020) |
| Type of Talk | Journal First Presentation |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Specifying Uncertainty in Use Case Models
In International Conference on Model-Driven Engineering and Software Development (MODELSWARD 2020). INSTICC , 2020.Status: Published
Specifying Uncertainty in Use Case Models
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, Department of Engineering Complex Software Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2020 |
| Location of Talk | International Conference on Model-Driven Engineering and Software Development (MODELSWARD 2020) |
| Publisher | INSTICC |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Technical reports
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
Simula Research Laboratory, 2020.Status: Published
Handling Uncertainties in Cyber-Physical Systems during TheirOperations with Digital Twins
It is a well-recognized fact that a Cyber-Physical System (CPS) experiences uncertain (including unknown) situations during their operations. Some of such uncertainties could potentially lead to failures of CPS operations. Factors contribute to such uncertainties include 1) the intrinsically unpredictable physical environment of a CPS, 2) the use of communication networks continuously experiencing problems (e.g., slower connection than expected), and 3) the increasing use of machine learning algorithms in CPSs which introduce inherent uncertainties to these CPSs. No matter how meticulously a CPS is designed and developed, it is impossible to predict all possible uncertain situations it will experience during its operation. Thus, there is a need for new methods for discovering and handling uncertain situations during the CPS operation to prevent it from failure. In this paper, we present our ideas on how digital twins, i.e., "live models" of CPSs can help in discovering and handling potentially unsafe situations during its operation. We present the research challenges and potential solutions to develop, deploy, and operate such digital twins. The presented work is planned to be performed in a recently accepted European research-based innovation project, ADEPTNESS.
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Design-Operation Continuum Methods for Testing and Deployment under Unforeseen Conditions for Cyber-Physical Systems of Systems |
| Publication Type | Technical reports |
| Year of Publication | 2020 |
| Publisher | Simula Research Laboratory |
| Other Numbers | 2020-01 |
Talks, invited
Testing Quantum Programs
In Oslo Metropolitan University, Oslo, Norway, 2020.Status: Published
Testing Quantum Programs
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Talks, invited |
| Year of Publication | 2020 |
| Location of Talk | Oslo Metropolitan University, Oslo, Norway |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Proceedings, refereed
Big data from the cloud to the edge: the aggregate computing solution
In Proceedings of the 13th European Conference on Software Architecture. Vol. 2. New York, NY, USA: ACM, 2019.Status: Published
Big data from the cloud to the edge: the aggregate computing solution
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | Proceedings of the 13th European Conference on Software Architecture |
| Volume | 2 |
| Pagination | 177-180 |
| Publisher | ACM |
| Place Published | New York, NY, USA |
| URL | https://dl.acm.org/citation.cfm?id=3344988&dl=ACM&coll=DL |
| DOI | 10.1145/3344948.3344988 |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Experiences of studying Attention through EEG in the Context of Review Tasks
In EASE '19: Proceedings of the Evaluation and Assessment on Software Engineering. New York, NY, USA: ACM Press, 2019.Status: Published
Experiences of studying Attention through EEG in the Context of Review Tasks
Context: Electroencephalograms (EEG) have been used in a few cases in the context of software engineering (SE). EEGs allow capturing emotions and cognitive functioning. Such human factors have already shown to be important to understand software engineering tasks. Therefore, it is essential to gain experience in the community to utilize EEG as a research tool. Objective: To report experiences of using EEG in the context of a software engineering education (review of master theses proposals). We provide our reflections and lessons learned of (1) how to plan an EEG study, (2) how to conduct and execute (e.g., tools), (3) how to analyze. Method: We carried out an experiment using an EEG headset to measure the participants' attention rate. The experiment task includes reviewing three master thesis project plans. Results: We describe how we evolved our understanding of experimentation practices to collect and analyze psychological and cognitive data. We also provide a set of lessons learned regarding the application of EEG technology for research. Conclusions: We believe that that EEG could benefit software engineering research to collect cognitive information under certain conditions. The lessons learned reported here should be used as inputs for future experiments in software engineering, where human aspects are of interest.
| Afilliation | Software Engineering |
| Project(s) | Department of IT Management, Simula Metropolitan Center for Digital Engineering |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | EASE '19: Proceedings of the Evaluation and Assessment on Software Engineering |
| Pagination | 313-318 |
| Publisher | ACM Press |
| Place Published | New York, NY, USA |
| ISBN Number | 9781450371452 |
| Keywords | attention, experiment, human subjects, lectroencephalogram |
| URL | http://dl.acm.org/citation.cfm?doid=3319008http://dl.acm.org/citation.cf... |
| DOI | 10.1145/331900810.1145/3319008.3319357 |
Stability Analysis for Safety of Automotive Multi-Product Lines: A Search-Based Approach
In The Genetic and Evolutionary Computation Conference (GECCO). ACM, 2019.Status: Published
Stability Analysis for Safety of Automotive Multi-Product Lines: A Search-Based Approach
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | The Genetic and Evolutionary Computation Conference (GECCO) |
| Pagination | 1241-1249 |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Towards a Framework for the Analysis of Multi-Product Lines in the Automotive Domain
In Proceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems. New York, NY, USA: ACM, 2019.Status: Published
Towards a Framework for the Analysis of Multi-Product Lines in the Automotive Domain
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Proceedings, refereed |
| Year of Publication | 2019 |
| Conference Name | Proceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems |
| Publisher | ACM |
| Place Published | New York, NY, USA |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Talks, invited
Digital Twins for Cyber-Physical Systems, and Quantum Software Engineering: Research Agenda
In National Institute of Informatics, Tokyo, Japan, 2019.Status: Published
Digital Twins for Cyber-Physical Systems, and Quantum Software Engineering: Research Agenda
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems, Enabling Future Dependable Ubiquitous Services and Data with Novel Testing Methods for Quantum Programs |
| Publication Type | Talks, invited |
| Year of Publication | 2019 |
| Location of Talk | National Institute of Informatics, Tokyo, Japan |
| Type of Talk | Colloquium |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Practical Cyber-Physical Systems Testing with Artificial Intelligence Techniques
In Artificial Intelligence Lab, Oslo Metropolitan University, Norway, 2019.Status: Published
Practical Cyber-Physical Systems Testing with Artificial Intelligence Techniques
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Talks, invited |
| Year of Publication | 2019 |
| Location of Talk | Artificial Intelligence Lab, Oslo Metropolitan University, Norway |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty in Requirements
In Nanjing University of Aeronautics and Astronautics, Nanjing, China, 2019.Status: Published
Uncertainty in Requirements
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Talks, invited |
| Year of Publication | 2019 |
| Location of Talk | Nanjing University of Aeronautics and Astronautics, Nanjing, China |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Journal Article
Employing Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
Journal of Systems and Software 153 (2019): 86-104.Status: Published
Employing Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
Test case prioritization (TP) is widely used in regression testing for optimal reordering of test cases to achieve specific criteria (e.g., higher fault detection capability) as early as possible. In our earlier work, we proposed an approach for black-box dynamic TP using rule mining and multi-objective search (named as REMAP) by defining two objectives (fault detection capability and test case reliance score) and considering test case execution results at runtime. In this paper, we conduct an extensive empirical evaluation of REMAP by employing three different rule mining algorithms and three different multi-objective search algorithms, and we also evaluate REMAP with one additional objective (estimated execution time) for a total of 18 different configurations (i.e., 3 rule mining algorithms × 3 search algorithms × 2 different set of objectives) of REMAP. Specifically, we empirically evaluated the 18 variants of REMAP with 1) two variants of random search while using two objectives and three objectives, 2) three variants of greedy algorithm based on one objective, two objectives, and three objectives, 3) 18 variants of static search-based prioritization approaches, and 4) six variants of rule-based prioritization approaches using two industrial and three open source case studies. Results showed that the two best variants of REMAP with two objectives and three objectives significantly outperformed the best variants of competing approaches by 84.4% and 88.9%, and managed to achieve on average 14.2% and 18.8% higher Average Percentage of Faults Detected per Cost (APFDc) scores.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Journal of Systems and Software |
| Volume | 153 |
| Pagination | 86-104 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
Search-Based Test Case Implantation for Testing Untested Configurations
Information and Software Technology 111 (2019): 22-36.Status: Published
Search-Based Test Case Implantation for Testing Untested Configurations
Context: Modern large-scale software systems are highly configurable, and thus require a large number of test cases to be implemented and revised for testing a variety of system configurations. This makes testing highly configurable systems very expensive and time-consuming.
Objective: Driven by our industrial collaboration with a video conferencing company, we aim to automatically analyze and implant existing test cases (i.e., an original test suite) to test the untested configurations.
Method: We propose a search-based test case implantation approach (named as SBI) consisting of two key components: 1) Test case analyzer that statically analyzes each test case in the original test suite to obtain the program dependence graph for test case statements and 2) Test case implanter that uses multi-objective search to select suitable test cases for implantation using three operators, i.e., selection, crossover, and mutation (at the test suite level) and implants the selected test cases using a mutation operator at the test case level including three operations (i.e., addition, modification, and deletion).
Conclusion: SBI can be applied to automatically implant a test suite with the aim of testing untested configurations and thus achieving higher configuration coverage.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Information and Software Technology |
| Volume | 111 |
| Pagination | 22-36 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
| Keywords | genetic algorithms, Multi-objective optimization, Search, test case implantation |
Testing Self-Healing Cyber-Physical Systems under Uncertainty: A Fragility-Oriented Approach
Software Quality Journal 27, no. 2 (2019): 615-649.Status: Published
Testing Self-Healing Cyber-Physical Systems under Uncertainty: A Fragility-Oriented Approach
| Afilliation | Software Engineering |
| Project(s) | MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Department of Engineering Complex Software Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Software Quality Journal |
| Volume | 27 |
| Issue | 2 |
| Pagination | 615–649 |
| Date Published | 03/2019 |
| Publisher | Springer |
| URL | https://link.springer.com/article/10.1007/s11219-018-9437-3 |
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
Journal of Systems and Software 153 (2019).Status: Published
Uncertainty-wise Test Case Generation and Minimization for CyberPhysical Systems
| Afilliation | Software Engineering |
| Project(s) | U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, MBT4CPS: Model-Based Testing For Cyber-Physical Systems , Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Journal of Systems and Software |
| Volume | 153 |
| Date Published | 07/2019 |
| Publisher | Elsevier |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Using multi-objective search and machine learning to infer rules constraining product configurations
Automated Software Engineering (2019): 1-62.Status: Published
Using multi-objective search and machine learning to infer rules constraining product configurations
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems, MBT4CPS: Model-Based Testing For Cyber-Physical Systems |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Journal | Automated Software Engineering |
| Pagination | 1-62 |
| Publisher | Springer |
| DOI | 10.1007/s10515-019-00266-2 |
Talks, contributed
Enabling automated requirements reuse and configuration
In 23rd International Systems and Software Product Line Conference, Paris, France. ACM, 2019.Status: Published
Enabling automated requirements reuse and configuration
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2019 |
| Location of Talk | 23rd International Systems and Software Product Line Conference, Paris, France |
| Publisher | ACM |
| Type of Talk | Paris, France |
| DOI | 10.1145/3336294.3342370 |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Search-based test case implantation for testing untested configurations
In 34th IEEE/ACM International Conference on Automated Software Engineering (ASE 2019), San Diego (CA), USA. ACM, 2019.Status: Published
Search-based test case implantation for testing untested configurations
| Afilliation | Software Engineering |
| Project(s) | No Simula project |
| Publication Type | Talks, contributed |
| Year of Publication | 2019 |
| Location of Talk | 34th IEEE/ACM International Conference on Automated Software Engineering (ASE 2019), San Diego (CA), USA |
| Publisher | ACM |
| Type of Talk | Journal First Presentation |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Uncertainty-wise Test Case Generation and Minimization for Cyber-Physical Systems
In 34th IEEE/ACM International Conference on Automated Software Engineering (ASE 2019), San Diego (CA), USA. San Diego (CA), USA: ACM, 2019.Status: Published
Uncertainty-wise Test Case Generation and Minimization for Cyber-Physical Systems
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Talks, contributed |
| Year of Publication | 2019 |
| Location of Talk | 34th IEEE/ACM International Conference on Automated Software Engineering (ASE 2019), San Diego (CA), USA |
| Publisher | ACM |
| Place Published | San Diego (CA), USA |
| Type of Talk | Journal First Presentation |
Edited books
Evaluation and Assessment in Software Engineering 2019
ACM, 2019.Status: Published
Evaluation and Assessment in Software Engineering 2019
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Edited books |
| Year of Publication | 2019 |
| Publisher | ACM |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Second International Workshop on Verification and Validation of Internet of Things
IEEE, 2019.Status: Published
Second International Workshop on Verification and Validation of Internet of Things
| Afilliation | Software Engineering |
| Project(s) | Department of Engineering Complex Software Systems |
| Publication Type | Edited books |
| Year of Publication | 2019 |
| Publisher | IEEE |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
The 15th European Conference on Modelling Foundations and Applications (ECMFA)
2nd ed. Vol. 18. The Journal of Object Technology, 2019.Status: Published
The 15th European Conference on Modelling Foundations and Applications (ECMFA)
| Afilliation | Software Engineering |
| Project(s) | Co-Evolver: Uncertainty-Aware Coevolution Design of Self-Adaptive Cyber-Physical Systems |
| Publication Type | Edited books |
| Year of Publication | 2019 |
| Volume | 18 |
| Edition | 2 |
| Date Published | 09/2019 |
| Publisher | The Journal of Object Technology |
| URL | http://www.jot.fm/contents/issue_2019_02.html |
| DOI | 10.5381/jot.2019.18.2.e1 |
Consider publishing in the Empirical Software Engineering journal:
https://www.springer.com/journal/10664
Technical reports
Automated Test Case Implantation to Test Untested Configurations: A Cost-Effective Search-Based Approach
Simula Research Laboratory, 2018.Status: Published
Automated Test Case Implantation to Test Untested Configurations: A Cost-Effective Search-Based Approach
Modern large-scale software systems are highly configurable, and thus require a large number of test cases to be implemented and revised for testing a variety of system configurations. This makes testing highly configurable systems very expensive and time-consuming. Driven by our industrial collaboration with a video conferencing company, we aim to automatically analyze and implant existing test cases (i.e., an original test suite) to test the untested configurations. We propose a search-based test case implantation approach (named as SBI) consisting of two key components: 1) Test case analyzer that statically analyzes each test case in the original test suite to obtain the program dependence graph for test case statements and 2) Test case implanter that uses multi-objective search to select suitable test cases for implantation using three operators, i.e., selection, crossover, and mutation (at the test suite level) and implants the selected test cases using a mutation operator at the test case level including three operations (i.e., addition, modification, and deletion). We empirically evaluated SBI with an industrial case study and an open source case study by comparing the implanted test suites produced by SBI with the original test suite using evaluation metrics such as statement coverage (SC), branch coverage (BC), mutation score (MS). Results show that for both the case studies, the implanted test suites performed significantly better than the original test suites with on average 21.9% higher coverage of configuration variable values. For the open source case study, SBI managed to improve SC, BC, and MS with 4.8%, 7.5%, and 2.6%, respectively. SBI can be applied to automatically implant an existing test suite with the aim of testing untested configurations and thus achieving higher configuration coverage.
| Afilliation | Software Engineering |
| Project(s) | The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Technical reports |
| Year of Publication | 2018 |
| Publisher | Simula Research Laboratory |
| Keywords | Genetic Algorithm, Multi-objective optimization, Search, test case implantation |
| Notes | This research was supported by the Research Council of Norway (RCN) funded Certus SFI. Shuai Wang is also supported by RFF Hovedstaden funded MBE-CR project and COST Action CA15140 (ImAppNIO). Tao Yue and Shaukat Ali are supported by RCN funded Zen-Configurator project, RCN funded MBTCPS project, and COST Action IC1404. |
| URL | https://www.simula.no/sites/default/files/publications/files/automated_t... |
Employing Multi-Objective Search and Machine Learning to Mine Cross Product Line Rules: A Technical Report
Oslo, Norway: Simula Research Laboratory, 2018.Status: Published
Employing Multi-Objective Search and Machine Learning to Mine Cross Product Line Rules: A Technical Report
Modern systems are being developed by integrating multiple products within/across product lines that communicate with each other through information networks. Runtime behaviors of such systems are related to product configurations and information networks. Cost-effectively supporting Product Line Engineering (PLE) of such systems is challenging mainly because of lacking the support of automation of the configuration process. Capturing rules is the key for automating the configuration process in PLE. However, there does not exist explicitly-specified rules constraining configurable parameter values of such products and product lines. Manually specifying such rules is tedious and time-consuming. To address this challenge, in this paper, we present an improved version (named as SBRM+) of our previously proposed Search-based Rule Mining (SBRM) approach. SBRM+ incorporates two machine learning algorithms (i.e., C4.5 and PART) and two multi-objective search algorithms (i.e., NSGA-II and NSGA-III), employs a clustering algorithm (i.e., k-means) for classifying rules as high or low confidence rules, which are used for defining three objectives to guide the search. To evaluate SBRM+ (i.e., SBRM+NSGA-II-C45, SBRM+NSGA-III-C45, SBRM+NSGA-II-PART, and SBRM+NSGA-III-PART), we performed two case studies (Cisco and Jitsi) and conducted three types of analyses of results: difference analysis, correlation analysis, and trend analysis. Results of the analyses show that all the SBRM+ approaches performed significantly better than two Random Search-based approaches (RBRM+-C45 and RBRM+-PART) in terms of fitness values, six quality indicators, and 17 machine learning quality measurements (MLQMs). As compared to RBRM+ approaches, SBRM+ approaches have improved the quality of rules based on MLQMs up to 27% for the Cisco case study and 28% for the Jitsi case study.
| Afilliation | Software Engineering |
| Project(s) | Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines |
| Publication Type | Technical reports |
| Year of Publication | 2018 |
| Pagination | 1-59 |
| Date Published | 05/2018 |
| Publisher | Simula Research Laboratory |
| Place Published | Oslo, Norway |
| Keywords | Configuration, Interacting Products, Machine learning, Multi-Objective Search, Product Line, Rule Mining |
| Notes | This work was supported by the Zen-Configurator project funded by the Research Council of Norway (Grant No. 240024/F20) under the category of Young Research Talents of the FRIPO funding scheme. Tao Yue and Shaukat Ali are also supported by the Co-evolver project funded by the Research Council of Norway (grant no. 286898/LIS) under the category of Researcher Projects of the FRIPO funding scheme. |
| URL | https://www.simula.no/sites/default/files/publications/files/employing_m... |
Employing Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
Simula Research Laboratory, 2018.Status: Published
Employing Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization
Test case prioritization (TP) is widely used in regression testing for optimal reordering of test cases to achieve specific criteria (e.g., higher fault detection capability) as early as possible. In our earlier work, we proposed an approach for black-box dynamic TP using rule mining and multi-objective search (named as REMAP) by defining two objectives (fault detection capability and test case reliance score) by considering test case execution results at runtime. In this paper, we conduct an extensive empirical evaluation of REMAP by employing three different rule mining algorithms and three different multi-objective search algorithms, and we also evaluate REMAP with one additional objective (estimated execution time) for a total of 18 different configurations of REMAP. Specifically, we empirically evaluated the 18 variants of REMAP with 1) two variants of random search while using two objectives and three objectives, 2) three variants of greedy algorithm based on one objective, two objectives, and three objectives, 3) 18 variants of static search-based prioritization approaches, and 4) six variants of rule-based prioritization approaches using two industrial and three open source case studies. Results showed that the two b