-
How Deduction Systems Can Help You To Verify Stability Properties
Authors:
Mario Gleirscher,
Rehab Massoud,
Dieter Hutter,
Christoph Lüth
Abstract:
Mathematical proofs are a cornerstone of control theory, and it is important to get them right. Deduction systems can help with this by mechanically checking the proofs. However, the structure and level of detail at which a proof is represented in a deduction system differ significantly from a proof read and written by mathematicians and engineers, hampering understanding and adoption of these sys…
▽ More
Mathematical proofs are a cornerstone of control theory, and it is important to get them right. Deduction systems can help with this by mechanically checking the proofs. However, the structure and level of detail at which a proof is represented in a deduction system differ significantly from a proof read and written by mathematicians and engineers, hampering understanding and adoption of these systems.
This paper aims at helping to bridge the gap between machine-checked proofs and proofs in engineering and mathematics by presenting a machine-checked proof for stability using Lyapunov's theorem in a human-readable way. The structure of the proof is analyzed in detail, and potential benefits of such a proof are discussed, such as generalizability, reusability and increased trust in correctness.
△ Less
Submitted 16 April, 2024;
originally announced April 2024.
-
Scalable Automated Verification for Cyber-Physical Systems in Isabelle/HOL
Authors:
Jonathan Julián Huerta y Munive,
Simon Foster,
Mario Gleirscher,
Georg Struth,
Christian Pardillo Laursen,
Thomas Hickman
Abstract:
We formally introduce IsaVODEs (Isabelle verification with Ordinary Differential Equations), a framework for the verification of cyber-physical systems. We describe the semantic foundations of the framework's formalisation in the Isabelle/HOL proof assistant. A user-friendly language specification based on a robust state model makes our framework flexible and adaptable to various engineering workf…
▽ More
We formally introduce IsaVODEs (Isabelle verification with Ordinary Differential Equations), a framework for the verification of cyber-physical systems. We describe the semantic foundations of the framework's formalisation in the Isabelle/HOL proof assistant. A user-friendly language specification based on a robust state model makes our framework flexible and adaptable to various engineering workflows. New additions to the framework increase both its expressivity and proof automation. Specifically, formalisations related to forward diamond correctness specifications, certification of unique solutions to ordinary differential equations (ODEs) as flows, and invariant reasoning for systems of ODEs contribute to the framework's scalability and usability. Various examples and an evaluation validate the effectiveness of our framework.
△ Less
Submitted 22 January, 2024;
originally announced January 2024.
-
A Stochastic Approach to Classification Error Estimates in Convolutional Neural Networks
Authors:
Jan Peleska,
Felix Brüning,
Mario Gleirscher,
Wen-ling Huang
Abstract:
This technical report presents research results achieved in the field of verification of trained Convolutional Neural Network (CNN) used for image classification in safety-critical applications. As running example, we use the obstacle detection function needed in future autonomous freight trains with Grade of Automation (GoA) 4. It is shown that systems like GoA 4 freight trains are indeed certifi…
▽ More
This technical report presents research results achieved in the field of verification of trained Convolutional Neural Network (CNN) used for image classification in safety-critical applications. As running example, we use the obstacle detection function needed in future autonomous freight trains with Grade of Automation (GoA) 4. It is shown that systems like GoA 4 freight trains are indeed certifiable today with new standards like ANSI/UL 4600 and ISO 21448 used in addition to the long-existing standards EN 50128 and EN 50129. Moreover, we present a quantitative analysis of the system-level hazard rate to be expected from an obstacle detection function. It is shown that using sensor/perceptor fusion, the fused detection system can meet the tolerable hazard rate deemed to be acceptable for the safety integrity level to be applied (SIL-3). A mathematical analysis of CNN models is performed which results in the identification of classification clusters and equivalence classes partitioning the image input space of the CNN. These clusters and classes are used to introduce a novel statistical testing method for determining the residual error probability of a trained CNN and an associated upper confidence limit. We argue that this greybox approach to CNN verification, taking into account the CNN model's internal structure, is essential for justifying that the statistical tests have covered the trained CNN with its neurons and inter-layer mappings in a comprehensive way.
△ Less
Submitted 21 December, 2023;
originally announced January 2024.
-
Proceedings Fifth International Workshop on Formal Methods for Autonomous Systems
Authors:
Marie Farrell,
Matt Luckcuck,
Mario Gleirscher,
Maike Schwammberger
Abstract:
This EPTCS volume contains the proceedings for the Fifth International Workshop on Formal Methods for Autonomous Systems (FMAS 2023), which was held on the 15th and 16th of November 2023. FMAS 2023 was co-located with 18th International Conference on integrated Formal Methods (iFM) (iFM'22), organised by Leiden Institute of Advanced Computer Science of Leiden University. The workshop itself was he…
▽ More
This EPTCS volume contains the proceedings for the Fifth International Workshop on Formal Methods for Autonomous Systems (FMAS 2023), which was held on the 15th and 16th of November 2023. FMAS 2023 was co-located with 18th International Conference on integrated Formal Methods (iFM) (iFM'22), organised by Leiden Institute of Advanced Computer Science of Leiden University. The workshop itself was held at Scheltema Leiden, a renovated 19th Century blanket factory alongside the canal.
FMAS 2023 received 25 submissions. We received 11 regular papers, 3 experience reports, 6 research previews, and 5 vision papers. The researchers who submitted papers to FMAS 2023 were from institutions in: Australia, Canada, Colombia, France, Germany, Ireland, Italy, the Netherlands, Sweden, the United Kingdom, and the United States of America. Increasing our number of submissions for the third year in a row is an encouraging sign that FMAS has established itself as a reputable publication venue for research on the formal modelling and verification of autonomous systems. After each paper was reviewed by three members of our Programme Committee we accepted a total of 15 papers: 8 long papers and 7 short papers.
△ Less
Submitted 15 November, 2023;
originally announced November 2023.
-
Probabilistic Risk Assessment of an Obstacle Detection System for GoA 4 Freight Trains
Authors:
Mario Gleirscher,
Anne E. Haxthausen,
Jan Peleska
Abstract:
In this paper, a quantitative risk assessment approach is discussed for the design of an obstacle detection function for low-speed freight trains with grade of automation (GoA)~4. In this 5-step approach, starting with single detection channels and ending with a three-out-of-three (3oo3) model constructed of three independent dual-channel modules and a voter, a probabilistic assessment is exemplif…
▽ More
In this paper, a quantitative risk assessment approach is discussed for the design of an obstacle detection function for low-speed freight trains with grade of automation (GoA)~4. In this 5-step approach, starting with single detection channels and ending with a three-out-of-three (3oo3) model constructed of three independent dual-channel modules and a voter, a probabilistic assessment is exemplified, using a combination of statistical methods and parametric stochastic model checking. It is illustrated that, under certain not unreasonable assumptions, the resulting hazard rate becomes acceptable for specific application settings. The statistical approach for assessing the residual risk of misclassifications in convolutional neural networks and conventional image processing software suggests that high confidence can be placed into the safety-critical obstacle detection function, even though its implementation involves realistic machine learning uncertainties.
△ Less
Submitted 26 June, 2023;
originally announced June 2023.
-
Qualification of Proof Assistants, Checkers, and Generators: Where Are We and What Next?
Authors:
Mario Gleirscher,
Robert Sachtleben,
Jan Peleska
Abstract:
Cyber-physical systems, such as learning robots and other autonomous systems, employ high-integrity software in their safety-critical control. This software is developed using a range of tools some of which need to be qualified for this purpose according to international standards. In this article, we first evaluate the state of the art of tool qualification for proof assistants, checkers (e.g., m…
▽ More
Cyber-physical systems, such as learning robots and other autonomous systems, employ high-integrity software in their safety-critical control. This software is developed using a range of tools some of which need to be qualified for this purpose according to international standards. In this article, we first evaluate the state of the art of tool qualification for proof assistants, checkers (e.g., model checkers), and generators (e.g., code generators, compilers) by means of a SWOT (Strengths, Weaknesses, Opportunities, Threats) analysis. Our focus is on the qualification of tools in the three mentioned categories. Our objective is to assess under which conditions these tools are already fit or could be made fit for use in the practical engineering and assurance of high-integrity control software. In a second step, we derive a viewpoint and a corresponding range of suggestions for improved tool qualification from the results of our SWOT analysis.
△ Less
Submitted 19 February, 2023;
originally announced February 2023.
-
Sound Development of Safety Supervisors
Authors:
Mario Gleirscher,
Lukas Plecher,
Jan Peleska
Abstract:
Safety supervisors are controllers enforcing safety properties by keeping a system in (or returning it to) a safe state. The development of such high-integrity components can benefit from a rigorous workflow integrating formal design and verification. In this paper, we present a workflow for the sound development of safety supervisors combining the best of two worlds, verified synthesis and comple…
▽ More
Safety supervisors are controllers enforcing safety properties by keeping a system in (or returning it to) a safe state. The development of such high-integrity components can benefit from a rigorous workflow integrating formal design and verification. In this paper, we present a workflow for the sound development of safety supervisors combining the best of two worlds, verified synthesis and complete testing. Synthesis allows one to focus on problem specification and model validation. Testing compensates for the crossing of abstraction, formalism, and tool boundaries and is a key element to obtain certification credit before entry into service. We establish soundness of our workflow through a rigorous argument. Our approach is tool-supported, aims at modern autonomous systems, and is illustrated with a collaborative robotics example.
△ Less
Submitted 16 March, 2022;
originally announced March 2022.
-
A Manifesto for Applicable Formal Methods
Authors:
Mario Gleirscher,
Jaco van de Pol,
Jim Woodcock
Abstract:
Formal methods were frequently shown to be effective and, perhaps because of that, practitioners are interested in using them more often. Still, these methods are far less applied than expected, particularly, in critical domains where they are strongly recommended and where they have the greatest potential. Our hypothesis is that formal methods still seem not to be applicable enough or ready for t…
▽ More
Formal methods were frequently shown to be effective and, perhaps because of that, practitioners are interested in using them more often. Still, these methods are far less applied than expected, particularly, in critical domains where they are strongly recommended and where they have the greatest potential. Our hypothesis is that formal methods still seem not to be applicable enough or ready for their intended use. In critical software engineering, what do we mean when we speak of a formal method? And what does it mean for such a method to be applicable both from a scientific and practical viewpoint? Based on what the literature tells about the first question, with this manifesto, we lay out a set of principles that when followed by a formal method give rise to its mature applicability in a given scope. Rather than exercising criticism of past developments, this manifesto strives to foster an increased use of formal methods to the maximum benefit.
△ Less
Submitted 22 August, 2023; v1 submitted 23 December, 2021;
originally announced December 2021.
-
Proceedings First Workshop on Applicable Formal Methods
Authors:
Mario Gleirscher,
Jaco van de Pol,
Jim Woodcock
Abstract:
This volume contains the proceedings of the 1st International Workshop on Applicable Formal Methods (AppFM 2021), 23 November 2021, held online as part of the 24th International Symposium on Formal Methods (FM). The aim of the AppFM workshop is to bring together researchers who improve and evaluate existing formal approaches and new variants in practical contexts and support the transfer of these…
▽ More
This volume contains the proceedings of the 1st International Workshop on Applicable Formal Methods (AppFM 2021), 23 November 2021, held online as part of the 24th International Symposium on Formal Methods (FM). The aim of the AppFM workshop is to bring together researchers who improve and evaluate existing formal approaches and new variants in practical contexts and support the transfer of these approaches to software engineering practice.
△ Less
Submitted 15 November, 2021;
originally announced November 2021.
-
Complete Test of Synthesised Safety Supervisors for Robots and Autonomous Systems
Authors:
Mario Gleirscher,
Jan Peleska
Abstract:
Verified controller synthesis uses world models that comprise all potential behaviours of humans, robots, further equipment, and the controller to be synthesised. A world model enables quantitative risk assessment, for example, by stochastic model checking. Such a model describes a range of controller behaviours some of which -- when implemented correctly -- guarantee that the overall risk in the…
▽ More
Verified controller synthesis uses world models that comprise all potential behaviours of humans, robots, further equipment, and the controller to be synthesised. A world model enables quantitative risk assessment, for example, by stochastic model checking. Such a model describes a range of controller behaviours some of which -- when implemented correctly -- guarantee that the overall risk in the actual world is acceptable, provided that the stochastic assumptions have been made to the safe side. Synthesis then selects an acceptable-risk controller behaviour. However, because of crossing abstraction, formalism, and tool boundaries, verified synthesis for robots and autonomous systems has to be accompanied by rigorous testing. In general, standards and regulations for safety-critical systems require testing as a key element to obtain certification credit before entry into service. This work-in-progress paper presents an approach to the complete testing of synthesised supervisory controllers that enforce safety properties in domains such as human-robot collaboration and autonomous driving. Controller code is generated from the selected controller behaviour. The code generator, however, is hard, if not infeasible, to verify in a formal and comprehensive way. Instead, utilising testing, an abstract test reference is generated, a symbolic finite state machine with simpler semantics than code semantics. From this reference, a complete test suite is derived and applied to demonstrate the observational equivalence between the synthesised abstract test reference and the generated concrete controller code running on a control system platform.
△ Less
Submitted 24 October, 2021;
originally announced October 2021.
-
Verified Synthesis of Optimal Safety Controllers for Human-Robot Collaboration
Authors:
Mario Gleirscher,
Radu Calinescu,
James Douthwaite,
Benjamin Lesage,
Colin Paterson,
Jonathan Aitken,
Rob Alexander,
James Law
Abstract:
We present a tool-supported approach for the synthesis, verification and validation of the control software responsible for the safety of the human-robot interaction in manufacturing processes that use collaborative robots. In human-robot collaboration, software-based safety controllers are used to improve operational safety, e.g., by triggering shutdown mechanisms or emergency stops to avoid acci…
▽ More
We present a tool-supported approach for the synthesis, verification and validation of the control software responsible for the safety of the human-robot interaction in manufacturing processes that use collaborative robots. In human-robot collaboration, software-based safety controllers are used to improve operational safety, e.g., by triggering shutdown mechanisms or emergency stops to avoid accidents. Complex robotic tasks and increasingly close human-robot interaction pose new challenges to controller developers and certification authorities. Key among these challenges is the need to assure the correctness of safety controllers under explicit (and preferably weak) assumptions. Our controller synthesis, verification and validation approach is informed by the process, risk analysis, and relevant safety regulations for the target application. Controllers are selected from a design space of feasible controllers according to a set of optimality criteria, are formally verified against correctness criteria, and are translated into executable code and validated in a digital twin. The resulting controller can detect the occurrence of hazards, move the process into a safe state, and, in certain circumstances, return the process to an operational state from which it can resume its original task. We show the effectiveness of our software engineering approach through a case study involving the development of a safety controller for a manufacturing work cell equipped with a collaborative robot.
△ Less
Submitted 11 June, 2021;
originally announced June 2021.
-
Hybrid Systems Verification with Isabelle/HOL: Simpler Syntax, Better Models, Faster Proofs
Authors:
Simon Foster,
Jonathan Julián Huerta y Munive,
Mario Gleirscher,
Georg Struth
Abstract:
We extend a semantic verification framework for hybrid systems with the Isabelle/HOL proof assistant by an algebraic model for hybrid program stores, a shallow expression model for hybrid programs and their correctness specifications, and domain-specific deductive and calculational support. The new store model yields clean separations and dynamic local views of variables, e.g. discrete/continuous,…
▽ More
We extend a semantic verification framework for hybrid systems with the Isabelle/HOL proof assistant by an algebraic model for hybrid program stores, a shallow expression model for hybrid programs and their correctness specifications, and domain-specific deductive and calculational support. The new store model yields clean separations and dynamic local views of variables, e.g. discrete/continuous, mutable/immutable, program/logical, and enhanced ways of manipulating them using combinators, projections and framing. This leads to more local inference rules, procedures and tactics for reasoning with invariant sets, certifying solutions of hybrid specifications or calculating derivatives with increased proof automation and scalability. The new expression model provides more user-friendly syntax, better control of name spaces and interfaces connecting the framework with real-world modelling languages.
△ Less
Submitted 10 June, 2021;
originally announced June 2021.
-
Maintaining driver attentiveness in shared-control autonomous driving
Authors:
Radu Calinescu,
Naif Alasmari,
Mario Gleirscher
Abstract:
We present a work-in-progress approach to improving driver attentiveness in cars provided with automated driving systems. The approach is based on a control loop that monitors the driver's biometrics (eye movement, heart rate, etc.) and the state of the car; analyses the driver's attentiveness level using a deep neural network; plans driver alerts and changes in the speed of the car using a formal…
▽ More
We present a work-in-progress approach to improving driver attentiveness in cars provided with automated driving systems. The approach is based on a control loop that monitors the driver's biometrics (eye movement, heart rate, etc.) and the state of the car; analyses the driver's attentiveness level using a deep neural network; plans driver alerts and changes in the speed of the car using a formally verified controller; and executes this plan using actuators ranging from acoustic and visual to haptic devices. The paper presents (i) the self-adaptive system formed by this monitor-analyse-plan-execute (MAPE) control loop, the car and the monitored driver, and (ii) the use of probabilistic model checking to synthesise the controller for the planning step of the MAPE loop.
△ Less
Submitted 5 February, 2021;
originally announced February 2021.
-
YAP: Tool Support for Deriving Safety Controllers from Hazard Analysis and Risk Assessments
Authors:
Mario Gleirscher
Abstract:
Safety controllers are system or software components responsible for handling risk in many machine applications. This tool paper describes a use case and a workflow for YAP, a research tool for risk modelling and discrete-event safety controller design. The goal of this use case is to derive a safety controller from hazard analysis and risk assessment, to define a design space for this controller,…
▽ More
Safety controllers are system or software components responsible for handling risk in many machine applications. This tool paper describes a use case and a workflow for YAP, a research tool for risk modelling and discrete-event safety controller design. The goal of this use case is to derive a safety controller from hazard analysis and risk assessment, to define a design space for this controller, and to select a verified optimal controller instance from this design space. We represent this design space as a stochastic model and use YAP for risk modelling and generation of parts of this stochastic model. For the controller verification and selection step, we use a stochastic model checker. The approach is illustrated by an example of a collaborative robot operated in a manufacturing work cell.
△ Less
Submitted 2 December, 2020;
originally announced December 2020.
-
Integration of Formal Proof into Unified Assurance Cases with Isabelle/SACM
Authors:
Simon Foster,
Yakoub Nemouchi,
Mario Gleirscher,
Ran Wei,
Tim Kelly
Abstract:
Assurance cases are often required to certify critical systems. The use of formal methods in assurance can improve automation, increase confidence, and overcome errant reasoning. However, assurance cases can never be fully formalised, as the use of formal methods is contingent on models that are validated by informal processes. Consequently, assurance techniques should support both formal and info…
▽ More
Assurance cases are often required to certify critical systems. The use of formal methods in assurance can improve automation, increase confidence, and overcome errant reasoning. However, assurance cases can never be fully formalised, as the use of formal methods is contingent on models that are validated by informal processes. Consequently, assurance techniques should support both formal and informal artifacts, with explicated inferential links between them. In this paper, we contribute a formal machine-checked interactive language, called Isabelle/SACM, supporting the computer-assisted construction of assurance cases compliant with the OMG Structured Assurance Case Meta-Model. The use of Isabelle/SACM guarantees well-formedness, consistency, and traceability of assurance cases, and allows a tight integration of formal and informal evidence of various provenance. In particular, Isabelle brings a diverse range of automated verification techniques that can provide evidence. To validate our approach, we present a substantial case study based on the Tokeneer secure entry system benchmark. We embed its functional specification into Isabelle, verify its security requirements, and form a modular security case in Isabelle/SACM that combines the heterogeneous artifacts. We thus show that Isabelle is a suitable platform for critical systems assurance.
△ Less
Submitted 25 September, 2020;
originally announced September 2020.
-
Challenges in the Safety-Security Co-Assurance of Collaborative Industrial Robots
Authors:
Mario Gleirscher,
Nikita Johnson,
Panayiotis Karachristou,
Radu Calinescu,
James Law,
John Clark
Abstract:
The coordinated assurance of interrelated critical properties, such as system safety and cyber-security, is one of the toughest challenges in critical systems engineering. In this chapter, we summarise approaches to the coordinated assurance of safety and security. Then, we highlight the state of the art and recent challenges in human-robot collaboration in manufacturing both from a safety and sec…
▽ More
The coordinated assurance of interrelated critical properties, such as system safety and cyber-security, is one of the toughest challenges in critical systems engineering. In this chapter, we summarise approaches to the coordinated assurance of safety and security. Then, we highlight the state of the art and recent challenges in human-robot collaboration in manufacturing both from a safety and security perspective. We conclude with a list of procedural and technological issues to be tackled in the coordinated assurance of collaborative industrial robots.
△ Less
Submitted 17 July, 2020;
originally announced July 2020.
-
Safety Controller Synthesis for Collaborative Robots
Authors:
Mario Gleirscher,
Radu Calinescu
Abstract:
In human-robot collaboration (HRC), software-based automatic safety controllers (ASCs) are used in various forms (e.g. shutdown mechanisms, emergency brakes, interlocks) to improve operational safety. Complex robotic tasks and increasingly close human-robot interaction pose new challenges to ASC developers and certification authorities. Key among these challenges is the need to assure the correctn…
▽ More
In human-robot collaboration (HRC), software-based automatic safety controllers (ASCs) are used in various forms (e.g. shutdown mechanisms, emergency brakes, interlocks) to improve operational safety. Complex robotic tasks and increasingly close human-robot interaction pose new challenges to ASC developers and certification authorities. Key among these challenges is the need to assure the correctness of ASCs under reasonably weak assumptions. To address this need, we introduce and evaluate a tool-supported ASC synthesis method for HRC in manufacturing. Our ASC synthesis is: (i) informed by the manufacturing process, risk analysis, and regulations; (ii) formally verified against correctness criteria; and (iii) selected from a design space of feasible controllers according to a set of optimality criteria. The synthesised ASC can detect the occurrence of hazards, move the process into a safe state, and, in certain circumstances, return the process to an operational state from which it can resume its original task.
△ Less
Submitted 7 July, 2020;
originally announced July 2020.
-
Towards Deductive Verification of Control Algorithms for Autonomous Marine Vehicles
Authors:
Simon Foster,
Mario Gleirscher,
Radu Calinescu
Abstract:
The use of autonomous vehicles in real-world applications is often precluded by the difficulty of providing safety guarantees for their complex controllers. The simulation-based testing of these controllers cannot deliver sufficient safety guarantees, and the use of formal verification is very challenging due to the hybrid nature of the autonomous vehicles. Our work-in-progress paper introduces a…
▽ More
The use of autonomous vehicles in real-world applications is often precluded by the difficulty of providing safety guarantees for their complex controllers. The simulation-based testing of these controllers cannot deliver sufficient safety guarantees, and the use of formal verification is very challenging due to the hybrid nature of the autonomous vehicles. Our work-in-progress paper introduces a formal verification approach that addresses this challenge by integrating the numerical computation of such a system (in GNU/Octave) with its hybrid system verification by means of a proof assistant (Isabelle). To show the effectiveness of our approach, we use it to verify differential invariants of an Autonomous Marine Vehicle with a controller switching between multiple modes.
△ Less
Submitted 16 June, 2020;
originally announced June 2020.
-
Mechanised Assurance Cases with Integrated Formal Methods in Isabelle
Authors:
Yakoub Nemouchi,
Simon Foster,
Mario Gleirscher,
Tim Kelly
Abstract:
Assurance cases are often required as a means to certify a critical system. Use of formal methods in assurance can improve automation, and overcome problems with ambiguity, faulty reasoning, and inadequate evidentiary support. However, assurance cases can rarely be fully formalised, as the use of formal methods is contingent on models validated by informal processes. Consequently, we need assuranc…
▽ More
Assurance cases are often required as a means to certify a critical system. Use of formal methods in assurance can improve automation, and overcome problems with ambiguity, faulty reasoning, and inadequate evidentiary support. However, assurance cases can rarely be fully formalised, as the use of formal methods is contingent on models validated by informal processes. Consequently, we need assurance techniques that support both formal and informal artifacts, with explicated inferential links and assumptions that can be checked by evaluation. Our contribution is a mechanical framework for developing assurance cases with integrated formal methods based in the Isabelle system. We demonstrate an embedding of the Structured Assurance Case Meta-model (SACM) using Isabelle/DOF, and show how this can be linked to formal analysis techniques originating from our verification framework, Isabelle/UTP. We validate our approach by mechanising a fragment of the Tokeneer security case, with evidence supplied by formal verification.
△ Less
Submitted 15 May, 2019;
originally announced May 2019.
-
Risk Structures: Towards Engineering Risk-aware Autonomous Systems
Authors:
Mario Gleirscher
Abstract:
Inspired by widely-used techniques of causal modelling in risk, failure, and accident analysis, this work discusses a compositional framework for risk modelling. Risk models capture fragments of the space of risky events likely to occur when operating a machine in a given environment. Moreover, one can build such models into machines such as autonomous robots, to equip them with the ability of ris…
▽ More
Inspired by widely-used techniques of causal modelling in risk, failure, and accident analysis, this work discusses a compositional framework for risk modelling. Risk models capture fragments of the space of risky events likely to occur when operating a machine in a given environment. Moreover, one can build such models into machines such as autonomous robots, to equip them with the ability of risk-aware perception, monitoring, decision making, and control. With the notion of a risk factor as the modelling primitive, the framework provides several means to construct and shape risk models. Relational and algebraic properties are investigated and proofs support the validity and consistency of these properties over the corresponding models. Several examples throughout the discussion illustrate the applicability of the concepts. Overall, this work focuses on the qualitative treatment of risk with the outlook of transferring these results to probabilistic refinements of the discussed framework.
△ Less
Submitted 23 April, 2019;
originally announced April 2019.
-
Assurance of System Safety: A Survey of Design and Argument Patterns
Authors:
Mario Gleirscher,
Stefan Kugele
Abstract:
The specification, design, and assurance of safety encompasses various concepts and best practices, subject of reuse in form of patterns. This work summarizes applied research on such concepts and practices with a focus on the last two decades and on the state-of-the-art of patterns in safety-critical system design and assurance argumentation. We investigate several aspects of such patterns, for e…
▽ More
The specification, design, and assurance of safety encompasses various concepts and best practices, subject of reuse in form of patterns. This work summarizes applied research on such concepts and practices with a focus on the last two decades and on the state-of-the-art of patterns in safety-critical system design and assurance argumentation. We investigate several aspects of such patterns, for example, where and when they are applied, their characteristics and purposes, and how they are related. For each aspect, we provide an overview of relevant studies and synthesize a taxonomy of first principles underlying these patterns. Furthermore, we comment on how these studies address known challenges and we discuss suggestions for further research. Our findings disclose a lack of research on how patterns improve system safety claims and, vice versa, on the decomposition of system safety into separated local concerns, and on the impact of security on safety.
△ Less
Submitted 14 February, 2019;
originally announced February 2019.
-
New Opportunities for Integrated Formal Methods
Authors:
Mario Gleirscher,
Simon Foster,
Jim Woodcock
Abstract:
Formal methods have provided approaches for investigating software engineering fundamentals and also have high potential to improve current practices in dependability assurance. In this article, we summarise known strengths and weaknesses of formal methods. From the perspective of the assurance of robots and autonomous systems (RAS), we highlight new opportunities for integrated formal methods and…
▽ More
Formal methods have provided approaches for investigating software engineering fundamentals and also have high potential to improve current practices in dependability assurance. In this article, we summarise known strengths and weaknesses of formal methods. From the perspective of the assurance of robots and autonomous systems (RAS), we highlight new opportunities for integrated formal methods and identify threats to the adoption of such methods. Based on these opportunities and threats, we develop an agenda for fundamental and empirical research on integrated formal methods and for successful transfer of validated research to RAS assurance. Furthermore, we outline our expectations on useful outcomes of such an agenda.
△ Less
Submitted 4 November, 2019; v1 submitted 25 December, 2018;
originally announced December 2018.
-
Formal Methods in Dependable Systems Engineering: A Survey of Professionals from Europe and North America
Authors:
Mario Gleirscher,
Diego Marmsoler
Abstract:
Context: Formal methods (FMs) have been around for a while, still being unclear how to leverage their benefits, overcome their challenges, and set new directions for their improvement towards a more successful transfer into practice. Objective: We study the use of formal methods in mission-critical software domains, examining industrial and academic views. Method: We perform a cross-sectional on-l…
▽ More
Context: Formal methods (FMs) have been around for a while, still being unclear how to leverage their benefits, overcome their challenges, and set new directions for their improvement towards a more successful transfer into practice. Objective: We study the use of formal methods in mission-critical software domains, examining industrial and academic views. Method: We perform a cross-sectional on-line survey. Results: Our results indicate an increased intent to apply FMs in industry, suggesting a positively perceived usefulness. But the results also indicate a negatively perceived ease of use. Scalability, skills, and education seem to be among the key challenges to support this intent. Conclusions: We present the largest study of this kind so far (N = 216), and our observations provide valuable insights, highlighting directions for future theoretical and empirical research of formal methods. Our findings are strongly coherent with earlier observations by Austin and Parkin (1993).
△ Less
Submitted 22 September, 2020; v1 submitted 20 December, 2018;
originally announced December 2018.
-
Safety Practice and its Practitioners: Exploring a Diverse Profession
Authors:
Mario Gleirscher,
Anne Nyokabi
Abstract:
System safety refers to a diverse engineering discipline assessing and improving various aspects of safety in socio-technical systems and their software-intensive sub-systems. While system safety has been a vital area of applied research for many decades, its practice and practitioners seem empirically still not well studied. Beyond mainly anecdotal evidence (interviews, on-line discussions), inci…
▽ More
System safety refers to a diverse engineering discipline assessing and improving various aspects of safety in socio-technical systems and their software-intensive sub-systems. While system safety has been a vital area of applied research for many decades, its practice and practitioners seem empirically still not well studied. Beyond mainly anecdotal evidence (interviews, on-line discussions), incident reports, and surveys, we are missing open, large-scale, and long-term investigations that promote knowledge transfer and research validation. We explore means for work safety practitioners rely on, factors influencing their performance, and their perception of their role in the system life cycle. Along with that we examine observations from previous research. We build a construct of safety practice, collect data for this construct using an on-line survey, summarise and interpret the collected data, and investigate several hypotheses based on the previous observations. We analyse and present the responses of 124 practitioners in safety-critical system and software projects. Aside from other findings, our data suggests that safety decision making mainly depends on expert opinion and project memory, lacks evidence that safety is typically a cost-benefit question, does not exhibit the prejudice that formal methods are not beneficial, leaves it unclear as to whether or not standards and methods have become inadequate, and indicates that safety is not typically confused with reliability. Additionally, we contribute a research design directing towards explanatory empirical studies of safety practice. Empirical research of safety practice is still in an early stage, bearing the risk of undesirable mismatches of the state of the art and the state of practice. However, this situation offers great opportunities for research.
△ Less
Submitted 20 December, 2018;
originally announced December 2018.
-
SCAV'18: Report of the 2nd International Workshop on Safe Control of Autonomous Vehicles
Authors:
Mario Gleirscher,
Sven Linker,
Stefan Kugele
Abstract:
This report summarizes the discussions, open issues, take-away messages, and conclusions of the 2nd SCAV workshop.
This report summarizes the discussions, open issues, take-away messages, and conclusions of the 2nd SCAV workshop.
△ Less
Submitted 5 November, 2018;
originally announced November 2018.
-
Proceedings 2nd International Workshop on Safe Control of Autonomous Vehicles
Authors:
Mario Gleirscher,
Stefan Kugele,
Sven Linker
Abstract:
These are the proceedings of the Second International Workshop on Safe Control of Autonomous Vehicles, which took place on the 10th of April 2018 in Porto, Portugal as an affiliated workshop of CSPWeek. The task of this workshop is to identify open research problems, discuss recent achievements, bring together researchers in, e.g., control theory, adaptive systems, machine self-organization and au…
▽ More
These are the proceedings of the Second International Workshop on Safe Control of Autonomous Vehicles, which took place on the 10th of April 2018 in Porto, Portugal as an affiliated workshop of CSPWeek. The task of this workshop is to identify open research problems, discuss recent achievements, bring together researchers in, e.g., control theory, adaptive systems, machine self-organization and autonomy, mobile intelligent robotics, transportation, traffic control, machine learning, software verification, and dependability and security engineering.
△ Less
Submitted 10 April, 2018;
originally announced April 2018.
-
From Hazard Analysis to Hazard Mitigation Planning: The Automated Driving Case
Authors:
Mario Gleirscher,
Stefan Kugele
Abstract:
Vehicle safety depends on (a) the range of identified hazards and (b) the operational situations for which mitigations of these hazards are acceptably decreasing risk. Moreover, with an increasing degree of autonomy, risk ownership is likely to increase for vendors towards regulatory certification. Hence, highly automated vehicles have to be equipped with verified controllers capable of reliably i…
▽ More
Vehicle safety depends on (a) the range of identified hazards and (b) the operational situations for which mitigations of these hazards are acceptably decreasing risk. Moreover, with an increasing degree of autonomy, risk ownership is likely to increase for vendors towards regulatory certification. Hence, highly automated vehicles have to be equipped with verified controllers capable of reliably identifying and mitigating hazards in all possible operational situations. To this end, available methods for the design and verification of automated vehicle controllers have to be supported by models for hazard analysis and mitigation. In this paper, we describe (1) a framework for the analysis and design of planners (i.e., high-level controllers) capable of run-time hazard identification and mitigation, (2) an incremental algorithm for constructing planning models from hazard analysis, and (3) an exemplary application to the design of a fail-operational controller based on a given control system architecture. Our approach equips the safety engineer with concepts and steps to (2a) elaborate scenarios of endangerment and (2b) design operational strategies for mitigating such scenarios.
△ Less
Submitted 22 February, 2018;
originally announced February 2018.
-
Run-Time Risk Mitigation in Automated Vehicles: A Model for Studying Preparatory Steps
Authors:
Mario Gleirscher
Abstract:
We assume that autonomous or highly automated driving (AD) will be accompanied by tough assurance obligations exceeding the requirements of even recent revisions of ISO 26262 or SOTIF. Hence, automotive control and safety engineers have to (i) comprehensively analyze the driving process and its control loop, (ii) identify relevant hazards stemming from this loop, (iii) establish feasible automated…
▽ More
We assume that autonomous or highly automated driving (AD) will be accompanied by tough assurance obligations exceeding the requirements of even recent revisions of ISO 26262 or SOTIF. Hence, automotive control and safety engineers have to (i) comprehensively analyze the driving process and its control loop, (ii) identify relevant hazards stemming from this loop, (iii) establish feasible automated measures for the effective mitigation of these hazards or the alleviation of their consequences.
By studying an example, this article investigates some achievements in the modeling for the steps (i), (ii), and (iii), amenable to formal verification of desired properties derived from potential assurance obligations such as the global existence of an effective mitigation strategy. In addition, the proposed approach is meant for step-wise refinement towards the automated synthesis of AD safety controllers implementing such properties.
△ Less
Submitted 8 September, 2017;
originally announced September 2017.
-
Arguing from Hazard Analysis in Safety Cases: A Modular Argument Pattern
Authors:
Mario Gleirscher,
Carmen Carlan
Abstract:
We observed that safety arguments are prone to stay too abstract, e.g. solutions refer to large packages, argument strategies to complex reasoning steps, contexts and assumptions lack traceability. These issues can reduce the confidence we require of such arguments. In this paper, we investigate the construction of confident arguments from (i) hazard analysis (HA) results and (ii) the design of sa…
▽ More
We observed that safety arguments are prone to stay too abstract, e.g. solutions refer to large packages, argument strategies to complex reasoning steps, contexts and assumptions lack traceability. These issues can reduce the confidence we require of such arguments. In this paper, we investigate the construction of confident arguments from (i) hazard analysis (HA) results and (ii) the design of safety measures, i.e., both used for confidence evaluation. We present an argument pattern integrating three HA techniques, i.e., FTA, FMEA, and STPA, as well as the reactions on the results of these analyses, i.e., safety requirements and design increments. We provide an example of how our pattern can help in argument construction and discuss steps towards using our pattern in formal analysis and computer-assisted construction of safety cases.
△ Less
Submitted 20 February, 2018; v1 submitted 12 April, 2017;
originally announced April 2017.
-
Introduction of Static Quality Analysis in Small and Medium-Sized Software Enterprises: Experiences from Technology Transfer
Authors:
Mario Gleirscher,
Dmitriy Golubitskiy,
Maximilian Irlbeck,
Stefan Wagner
Abstract:
Today, small and medium-sized enterprises (SMEs) in the software industry face major challenges. Their resource constraints require high efficiency in development. Furthermore, quality assurance (QA) measures need to be taken to mitigate the risk of additional, expensive effort for bug fixes or compensations. Automated static analysis (ASA) can reduce this risk because it promises low application…
▽ More
Today, small and medium-sized enterprises (SMEs) in the software industry face major challenges. Their resource constraints require high efficiency in development. Furthermore, quality assurance (QA) measures need to be taken to mitigate the risk of additional, expensive effort for bug fixes or compensations. Automated static analysis (ASA) can reduce this risk because it promises low application effort. SMEs seem to take little advantage of this opportunity. Instead, they still mainly rely on the dynamic analysis approach of software testing. In this article, we report on our experiences from a technology transfer project. Our aim was to evaluate the results static analysis can provide for SMEs as well as the problems that occur when introducing and using static analysis in SMEs. We analysed five software projects from five collaborating SMEs using three different ASA techniques: code clone detection, bug pattern detection and architecture conformance analysis. Following the analysis, we applied a quality model to aggregate and evaluate the results. Our study shows that the effort required to introduce ASA techniques in SMEs is small (mostly below one person-hour each). Furthermore, we encountered only few technical problems. By means of the analyses, we could detect multiple defects in production code. The participating companies perceived the analysis results to be a helpful addition to their current QA and will include the analyses in their QA process. With the help of the Quamoco quality model, we could efficiently aggregate and rate static analysis results. However, we also encountered a partial mismatch with the opinions of the SMEs. We conclude, that ASA and quality models can be a valuable and affordable addition to the QA process of SMEs.
△ Less
Submitted 22 November, 2016;
originally announced November 2016.
-
On the Benefit of Automated Static Analysis for Small and Medium-Sized Software Enterprises
Authors:
Mario Gleirscher,
Dmitriy Golubitskiy,
Maximilian Irlbeck,
Stefan Wagner
Abstract:
Today's small and medium-sized enterprises (SMEs) in the software industry are faced with major challenges. While having to work efficiently using limited resources they have to perform quality assurance on their code to avoid the risk of further effort for bug fixes or compensations. Automated static analysis can reduce this risk because it promises little effort for running an analysis. We repor…
▽ More
Today's small and medium-sized enterprises (SMEs) in the software industry are faced with major challenges. While having to work efficiently using limited resources they have to perform quality assurance on their code to avoid the risk of further effort for bug fixes or compensations. Automated static analysis can reduce this risk because it promises little effort for running an analysis. We report on our experience in analysing five projects from and with SMEs by three different static analysis techniques: code clone detection, bug pattern detection and architecture conformance analysis. We found that the effort that was needed to introduce those techniques was small (mostly below one person-hour), that we can detect diverse defects in production code and that the participating companies perceived the usefulness of the presented techniques as well as our analysis results high enough to include the techniques in their quality assurance.
△ Less
Submitted 22 November, 2016;
originally announced November 2016.
-
Model-based Hazard and Impact Analysis
Authors:
Sonila Dobi,
Mario Gleirscher,
Maria Spichkova,
Peter Struss
Abstract:
Hazard and impact analysis is an indispensable task during the specification and development of safety-critical technical systems, and particularly of their software-intensive control parts. There is a lack of methods supporting an effective (reusable, automated) and integrated (cross-disciplinary) way to carry out such analyses.
This report was motivated by an industrial project whose goal was…
▽ More
Hazard and impact analysis is an indispensable task during the specification and development of safety-critical technical systems, and particularly of their software-intensive control parts. There is a lack of methods supporting an effective (reusable, automated) and integrated (cross-disciplinary) way to carry out such analyses.
This report was motivated by an industrial project whose goal was to survey and propose methods and models for documentation and analysis of a system and its environment to support hazard and impact analysis as an important task of safety engineering and system development. We present and investigate three perspectives of how to properly encode safety-relevant domain knowledge for better reuse and automation, identify and assess all relevant hazards, as well as pre-process this information and make it easily accessible for reuse in other safety and systems engineering activities and, moreover, in similar engineering projects.
△ Less
Submitted 9 December, 2015;
originally announced December 2015.
