Simulation games are designed to cultivate expertise and rehearse particular skill sets. To yield longitudinal effects, sequences of events must be crafted to yield intended learning outcomes, sometimes by focusing on particularly difficult situations and replaying variants. The present paper develops a logic-based approach for encoding the interrelation between action, events, and objects in a manner that allows the resulting scenario description to immediately be executed in a game development environment. This has the dual effect of decoupling the description of a scenario from the simulation platform itself, as well as supporting iterative and flexible development of learning content. To this end, we provide three interrelated components: First, we develop a scenario description language based on Answer Set Programming. The language is designed to allow an automated reasoner to deduce a schedule of the future events that are caused by an action taken in a given simulation environment. Second, we define a protocol for exchanging actions and computed futures between, respectively, the simulation environment and the external automated reasoner. Finally, as a proof of concept, we develop an Application Programming Interface (API) for the Unity Real-Time Development Platform that implements the protocol and offers a software framework for connecting the computed future events to concrete game objects. This allows the game to evolve coherently from the specification. We argue that the resulting system inherits capabilities for artificial commonsense reasoning from its declarative basis which are useful for reasoning about an evolving emergency incident or training scenario.

We develop a logic-based approach for designing simulation-based training scenarios. Our methodology embodies a concise definition of the scenario concept and integrates the notions of training goals, acceptable versus unacceptable actions and performance scoring. The approach applies classical artificial intelligence (AI) planning to extract coherent plays from a causal description of the training domain. The domain- and task-specific parts are defined in a high-level action description language AL. Generic causal and temporal logic is added when the causal theory is compiled into the underlying Answer Set Programming (ASP) language. The ASP representation is used to derive a scoring function that reflects the quality of a play or training session, based on a distinction of states and actions into green (acceptable) and red (unacceptable) ones. To that end, we add to the casual theory a set of norms that specify an initial assignment of colors. The ASP engine uses these norms as axioms and propagates colors by consulting the causal theory. We prove that any set of such norms constitutes a conservative extension of the underlying causal theory. With this work, we hope to lay the foundation for the development of design and analysis tools for exercise managers. We envision a software system that lets an exercise manager view all plays of a tentative scenario design, with expediency information and scores for each possible play. Our approach is applicable to any domain in which means-ends reasoning is pertinent. We illustrate the approach in the domain of crisis response and management.

This open access book presents a set of basic techniques for estimating the benefit of IT development projects and portfolios. It also offers methods for monitoring how much of that estimated benefit is being achieved during projects.

Readers can then use these benefit estimates together with cost estimates to create a benefit/cost index to help them decide which functionalities to send into construction and in what order. This allows them to focus on constructing the functionality that offers the best value for money at an early stage.

Although benefits management involves a wide range of activities in addition to estimation and monitoring, the techniques in this book provides a clear guide to achieving what has always been the goal of project and portfolio stakeholders: developing systems that produce as much usefulness and value as possible for the money invested. The techniques can also help deal with vicarious motives and obstacles that prevent this happening. The book equips readers to recognize when a project budget should not be spent in full and resources be allocated elsewhere in a portfolio instead. It also provides development managers and upper management with common ground as a basis for making informed decisions.

The vision of network-centric operations is to increase operational capabilities through networked collaboration. NATO and its member nations state this vision in strategic documents at a very high level of abstraction. While suitable for giving an overall feel, current documentation renders the steps toward implementing those visions largely unsupported. We outline a method, based on agile requirements engineering, for converting high-level strategic visions into capabilities, whose form lend themselves to incremental implementation. We illustrate the use of this method in two cases that concern both operational capabilities and technical capabilities. We also  show how  the method enables one to prioritize  which capabilities to develop first. We conclude that some form of explicit methodology to span the gap between strategic visions and implementing those visions is necessary.

There is an immediate need for a greater number of highly skilled cybersecurity personnel to meet intensified cyber attacks. We propose a cyber range exercise management architecture that employs machine reasoning to structure the design, execution and analysis of cyber range training scenarios. The scenarios are then used in simulation-based training in an emulated IT infrastructure environment. The machine reasoning is obtained by combining four AI methods: attack-defence trees, formal argumentation theory, answer set programming and multiagent systems. We argue that this type of advanced functionality that supports exercise managers in their design and analysis of scenarios is strictly necessary to improve current exercise management systems and build the required cybersecurity expertise.

Modeling and Simulation as a Service (MSaaS) embodies the idea that simulations should be composed quickly for the task at hand from loosely coupled shared components, simulation services, in a cloud-based environment. These simulations are then offered, as composed simulation services, to human and technical consumers. Instrumental to this, is functionality that lets a simulation operator discover and compose simulation services and execute the composition. We describe this functionality in terms of what we call MSaaS infrastructure capabilities. Following the idea of stepwise refinement, the discovery and composition of simulation services can be done at design time using implementation-independent information about simulation services and at implementation time using implementation-specific information about simulation services. The execution environment can also be set up at design time and at implementation time. We therefore describe the MSaaS infrastructure capabilities in terms of how they are used on both implementation-independent and implementation-specific service information. By doing these elaborations, we intend to gain greater insight into how to perform simulation service discovery, composition, and execution. We conclude that although much of the required functionality for a MSaaS infrastructure is available through existing platforms and frameworks, it is necessary to offer this functionality as services, alongside (composed) simulation services, to fulfill the MSaaS vision.

We investigate an approach that allows exercise managers to design simulations with an explicit focus on building skills, rather than having to focus on all the objects and interactions that a simulation must have. Exercise managers may design exercises at various levels of abstraction and always independently of how those sessions are implemented in simulations, while simulation components that implement the design are assembled and to some extent, automatically, behind the scenes. We outline (1) how Answer Set Programming can assist exercise managers in exercise planning and (2) how automated stage and content generation may be used to invoke appropriate simulation components to realize the design. For deliberate and recurrent training of decision-making skills, stages and content must vary to avoid familiarity (testing effects). We conclude by distilling a main research hypothesis that stipulates how (1) and (2) represent two modes of automated reasoning (so-called deductive versus abductive) and how that distinction clarifies the planning task.

Estimates are inherently uncertain, and it’s fair to expect that business value estimates are even more so. By making this uncertainty explicit, you can manage and monitor your agile project according to upper and lower margins of uncertainty. The trick is to instantiate benefit points and story points with values reflecting various scenarios.

We argue that current technology for crisis training does not explicitly cater well enough for managing training objectives and skill building metrics throughout the lifespan of training. We suggest how successful crisis training may be enabled by interoperating next-generation exercise management tools with mixed-reality simulations. We propose an architecture consisting of (1) a front-end in which training objectives, essential skills, corresponding events and metrics can be declared, (2) a back-end consisting of simulations that implement the events and metrics and (3) a middleware which transfers information between the front-end and back-end to enable semi-automatic composition of the simulations and performance analysis. The purpose of this architecture is to facilitate learning through the principles of deliberate practice. We indicate where emerging technologies are necessary to achieve this.

The Allied Framework for Modelling and Simulation as a Service (MSaaS) proposes that simulations should be composed of services and applications, which may be stored in repositories, found by registry services, and composed by modelling services. Docker containerization may implement several of these aspects of the Allied Framework, and has emerged as a promising technology for MSaaS. We provide a case study of using Docker containerization to close in on the vision of the Allied Framework. Containerizing a simulation system is not a trivial task, and we describe a set of issues and challenges that must be considered. Docker has no mechanisms for describing which services are offered by an image, so this must be handled by an external registry service. As future work, we outline how service descriptions might be written using semantic  technology, so that containerized applications implementing these descriptions might be composed on the fly.

The Allied Framework for Modeling and Simulation (M&S) as a Service (MSaaS) is proposed by NATO MSG-136 as a permanent service- and cloud-based M&S ecosystem for use by NATO and partner nations. The framework is designed to aid stakeholders to utilize state-of-the-art service-oriented and cloud-based methodology and technology to achieve interoperability between participating simulation systems and ensuring credibility of results. This paper presents the reference architecture that is currently developed as part of the technical concept for the Allied Framework for MSaaS. The reference architecture is structured on The Open Group SOA Reference Architecture, while its contents is supplied via the NATO C3 Taxonomy in the form of architecture building blocks and architecture patterns. The MSaaS Reference Architecture is not a final product. It will change over time as new architecture building blocks and patterns are identified and added, and existing ones modified and improved.

Context
Research into software engineering teams focuses on human and social team factors. Social psychology deals with the study of team formation and has found that personality factors and group processes such as team climate are related to team effectiveness. However, there are only a handful of empirical studies dealing with personality and team climate and their relationship to software development team effectiveness.

Objective
We present aggregate results of a twice replicated quasi-experiment that evaluates the relationships between personality, team climate, product quality and satisfaction in software development teams.

Method
Our experimental study measures the personalities of team members based on the Big Five personality traits (openness, conscientiousness, extraversion, agreeableness, neuroticism) and team climate factors (participative safety, support for innovation, team vision and task orientation) preferences and perceptions. We aggregate the results of the three studies through a meta-analysis of correlations. The study was conducted with students.

Results
The aggregation of results from the baseline experiment and two replications corroborates the following findings. There is a positive relationship between all four climate factors and satisfaction in software development teams. Teams whose members score highest for the agreeableness personality factor have the highest satisfaction levels. The results unveil a significant positive correlation between the extraversion personality factor and software product quality. High participative safety and task orientation climate perceptions are significantly related to quality.

Conclusions
First, more efficient software development teams can be formed heeding personality factors like agreeableness and extraversion. Second, the team climate generated in software development teams should be monitored for team member satisfaction. Finally, aspects like people feeling safe giving their opinions or encouraging team members to work hard at their job can have an impact on software quality. Software project managers can take advantage of these factors to promote developer satisfaction and improve the resulting product.

Augmenting and extending military training by combining Live, Virtual and Constructive (LVC) simulation is thought to yield a range of benefits. A Concept Development and Experimentation (CD&E) activity was initiated to investigate the feasibility of complementing the Live training range at The Norwegian Army Combat & Manoeuvre Training Centre (NACMTC) with Virtual and Constructive simulations of BLUEFOR and OPFOR vehicles, UAV and artillery. The focus was on leadership training, and the military situation was augmented by activating contextual forces from the scenario and introducing more resources to administrate. The LVC simulation system prototype was run as a trial during an actual military exercise over four days. We found that the prototype did enable enhanced training, and that it is feasible and desirable to establish the LVC capability in full. Further, it is desirable to extend the LVC capability with augmented reality for Live forces and Joint Terminal Attack Controller (JTAC)/Close Air Support (CAS) simulation. We found it premature to evaluate the effect of LVC simulation training; because this is a longer-term activity, and because of a lack of validated instruments for measuring skill acquisition; in particular for decision making and judgement tasks. We conclude that there is a clear desire among operational personnel to acquire the LVC capabilities, and that several aspects of the capabilities can be established quickly. However, the use of LVC simulation for training and education must be mandated at all levels and incorporated explicitly in training plans and curricula, with a sufficient business case, so that political decisions for acquiring LVC capabilities can be made.

Context: To select the essential, non-negotiable product features is a key skill for stakeholders in software projects. Such selection relies on human judgment, possibly supported by structured prioritization techniques and tools. Goal: Our goal was to investigate whether certain attributes of prioritization techniques affect stakeholders' threshold for judging product features as essential. The four investigated techniques represent four combinations of granularity (low, high) and cognitive support (low, high). Method: To control for robustness and masking effects when investigating in the field, we conducted both an artificial experiment and a field experiment using the same prioritization techniques. In the artificial experiment, 94 subjects in four treatment groups indicated the features (from a list of 16) essential when buying a new cell phone. In the field experiment, 44 domain experts indicated the software product features that were essential for the fulfillment of the project's vision. The effects of granularity and cognitive support on the number of essential ratings were analyzed and compared between the experiments. Result: With lower granularity, significantly more features were rated as essential. The effect was large in the general experiment and extreme in the field experiment. Added cognitive support had medium effect, but worked in opposite directions in the two experiments, and was not statistically significant in the field experiment. Implications: Software projects should avoid taking stakeholders' judgments of essentiality at face value. Practices and tools should be designed to counteract biases and to support the conscious knowledge-based elements of prioritizing.

Estimating the effort of software development is fraught with difficulties, and it is clear that effort should be invested in improving the accuracy and the reliability (consistency) of effort estimates, as well as the assessment of estimate uncertainty. However, it is less clear where to target such improvement efforts. We discuss the degree to which it is feasible to improve the expertise of the person(s) who estimate(s), and the environment in which the estimation is performed. The former hinges on what there is to say about the development of estimation expertise and the task characteristics of effort estimation. The latter hinges on what contextual support may be developed in terms of environment control and tools and methodology. We integrate several theories to make a framework for discussing software effort estimation and planning. On the basis of that discussion, we conclude that present guidelines almost exclusively concern the environment and its influence on broad psychological factors, that task-specific estimation expertise is too weak a signal in the noise of biases, and that strengthening this expertise requires new efforts in understanding the task-specific elements in software effort estimation and planning, as well as environmental measures (tools and methodologies) that support expert behavior and expert learning.

The development of scientific software has similarities with processes that follow the software engineering Agile Manifesto: responsiveness to change and collaboration are of utmost importance. But how well do current scientific software development processes match the practices found in agile development methods, and what are the effects of using agile practices in such processes? To find out, the authors conducted a literature review of past projects and a multiple case study of three ongoing scientific software development projects.

The nature of scientific research and the development of scientific software have similarities with processes that follow the agile manifesto: responsiveness to change and collaboration are of the utmost importance. But how well do current scientific software development processes match the practices found in agile development methods? As a representative for a modern scientific software project, we briefly outline the FEniCS project whose objective is to automate solution of differential equations. Based on initial investigations, we propose a casestudy where we investigate how the development process in FEniCS can be mapped to agile development methods, and whether SCRUM can be suitable to administer such a project. The case study will use another project, the more traditional Dalton project, as a contrast.

Empirical studies have become an integral element of software engineering research and practice. This unique text/reference includes chapters from some of the top international empirical software engineering researchers and focuses on the practical knowledge necessary for conducting, reporting and using empirical methods in software engineering.

Increased realism in software engineering experiments is often promoted as an important means to increase generalizability and industrial relevance. In this context, artificiality, e.g., the use of constructed tasks in place of realistic tasks, is seen as a threat. In this article, we examine the opposite view, that deliberately introduced artificial design elements may increase knowledge gain and enhance both generalizability and relevance. In the first part of the article, we identify and evaluate arguments and examples in favor of, and against, deliberately introducing artificiality into software engineering experiments. In the second part of the article, we summarize a content analysis of articles reporting software engineering experiments published over the ten-year period 1993-2002. The analysis reveals a striving for realism and external validity, but little awareness of for what and when, various degrees of artificiality and realism are appropriate. We conclude that an increased awareness and deliberation in these respects is essential. However, arguments in favor of artificial design elements should not be used to justify studies that are badly designed or that have research questions of low relevance.

The classical method for identifying cause-effect relationships is to conduct controlled experiments. This paper reports upon the present state of how controlled experiments in software engineering are conducted and the extent to which relevant information is reported. Among the 5,453 scientific articles published in 12 leading software engineering journals and conferences in the decade from 1993 to 2002, 103 articles (1.9 percent) reported controlled experiments in which individuals or teams performed one or more software engineering tasks. This survey quantitatively characterizes the topics of the experiments and their subjects (number of subjects, students versus professionals, recruitment, and rewards for participation), tasks (type of task, duration, and type and size of application) and environments (location, development tools). Furthermore, the survey reports on how internal and external validity is addressed and the extent to which experiments are replicated. The gathered data reflects the relevance of software engineering experiments to industrial practice and the scientific maturity of software engineering research.

The use of safety-critical software relies crucially on the certified verification relative to an unambiguous specification that the software will perform correctly, even on the first occasion it is put to use. One way of achieving unambiguity and assurance is to use mathematical or formal methods in the development process of such software, so that the result is software that is mathematically proven correct relative to a formal specification. The concept of algebraic specification refinement embodies a framework in which to develop software formally. In this thesis, this concept is expressed in a different, and in many respects, richer mathematical framework, namely that of type theory, polymorphic lambda-calculus, and an intuitionistic logic extended to assert relational parametricity, as well as key features of specification refinement. This translation into a different framework enables important extensions to the concept of algebraic specification refinement that enhance the proving power, the applicability, and hence the usefulness of the concept.

The thesis deals with topics in the field of formal abstract specification and verification of programs, particularly within the framework of algebraic methods restricted to equational logic. The discussion views equational specification in conjunctiom with the closely related topic of simple term-rewriting as more concrete and nearer to implementation than specification in general. Therefore initial and final models are considered, rather than complete model-classes. Also mostly constructive, ``executable'' equational specifications are considered giving an inclination towards viewing specifications as abstract programs. The thesis is structured around the following two topics:

––

1. The method of equational algebraic specification is generalized in a manner allowing a certain mode of modular specification. The generalization has two forms accommodating initial and final algebra semantics respectively. The generalization is modular in the sense that complex specifications may be constructed stepwise from simpler kernel specifications. Each construction step provides the choice of initial or final generalized form. The resulting complex equational specifications differ from general case predicate logic hierarchical specifications, in the sense that the polarization into initial and final semantics gives inhomogeneous specifications; a complex specification cannot in principle be viewed as a non-hierarchical simple specification.

A concept of consistency relative to kernel specifications is developed and it is possible to reason about such relative consistency without specific knowledge about the kernel specification.

The possibility of analogously constructing complex formal-mechanical proof methods from simpler methods is briefly discussed.

2. A variant of (initial) algebraic specification called indirect specification is developed. Motivation and a foundation for indirect specification is given by syntactical specifier functions; i.e. terms are to be understood identical iff the their values under some given function (with syntactical codomain) are identical. A special and interesting case of syntactical specifier functions are functions giving canonical representatives in some well-defined sense.

Besides providing the specification language with additional means of specifying equality, indirect specification expands the class of congruences (over ground terms) decidable by simple term-rewriting. In addition, although the class of initial congruences equationally specifiable is identical to the class specifiable by syntactical specifier functions with canonical representatives as codomain, there exist syntactical specifier functions which do not give canonical representatives. (Such a function may for instance give a representative in some other class than in which the argument belongs.) It may therefore be the fact that the class of congruences indirectly specifiable is greater than the class equationally specifiable. Characteristically, indirect specification represents a more operational mode of specification than does usual algebraic specification.

Indirect specification introduces further inhomogeneousity to equational algebraic specification. The modularity of the generalized specification strategy discussed in part 1 is used to encapsulate indirect specification in the hierarchical framework.

It is then shown that a trivial augmentation of indirect specification can be reduced to standard non-generalized initial or final semantics specification; under certain interesting circumstances. It is also shown that Knuth-Bendix completion of an augmentation of indirect specification, if successful and under a certain congruency condition, will give an equivalent standard algebraic specification. Viewing equational specifications as abstract programs, this might be seen as a program transformation. (A more applicable strategy for automatic transformation in this sense is sketched just as an idea without any sort of further proof.)

––

The overall structure sees part 1 primarily as giving a framework for the discussion in part 2.

The subject of consistency permeates the entire discussion. Consistency is viewed as related to basic ideas of some semantic domain which are expressed as untamperable presuppositions in the act of specifying. For example, the basic idea that a mathematical proposition cannot simultaneously be true and false is presupposed in predicate calculus by predefined interpretations of the symbols true and false, and by referring to a set of axioms as inconsistent iff the predicate true = false is deducible from the axioms. The concept of (in)consistency seen always relatively to such presuppositions allows a generalized notion of consistency in hierarchical specification relative to kernel specifications as discussed in part 1. (This also generalizes Guttag's notion of consistency.) Some simple methods for detecting (generalized) inconsistency and for establishing consistency are briefly presented.

The notion of artificial inconsistency is introduced, as inconsistency due to auxiliary functions; i.e. functions helpful or necessary during constructive definition or implementation, but otherwise really not belonging to the semantical objects under implementation. Formalisms and results are developed showing that auxiliary functions and hence artificial inconsistency can be hidden from the formal reasoning, still allowing for the full implementatory benefits. This goes beyond the model-theoretical notions of hidden sorts and symbols. The theory developed can probably be used and implemented by modifying existing proof methods (based on the concepts of proof by consistency and inductive completion).

Finally an extension of Knuth-Bendix completion is presented as a step towards mechanical generation of constructive proofs in the input theory given to the process. Constructive proofs may give deeper insight into a theory. It also turns out that the extension of Knuth-Bendix completion under certain circumstances can be used in establishing consistency in conjunction with indirect specification (part 2). The extension also turns out to be instrumental in proving that hiding of auxiliary functions and artificial inconsistency may be implemented in proof methods based on inductive completion.