Making critical autonomous AI-based systems safe

Making critical autonomous AI-based systems safe


To improve the explainability and traceability of DL components

To provide clear safety patterns for the incremental adoption of DL software in Critical Autonomous AI-based Systems (CAIS)

To integrate the SAFEXPLAIN libraries with an industrial system-testing toolset

To create architectures of DL components with quantifiable and controllable confidence, and that have the ability to identify when predictions should not be released based on applicability’s scope or security concerns

To design, implement, or update selected representative DL software libraries according to safety patterns and safety lifecycle considerations, meeting specific performance requirements on  relevant platforms

Deep Learning (DL) techniques are key for most future advanced
software functions in Critical Autonomous AI-based Systems (CAIS) in
cars, trains and satellites. Hence, those CAIS industries depend on their
ability to design, implement, qualify, and certify DL-based software
products under bounded effort/cost

Case studies

Railway: Based on Automatic Train Operation (ATO), this case study seeks to check the viability of a safety architectural pattern composed of: DL artificial vision software elements that serve as “sensors” to provide information to safety-related software elements

Space: This case study envisions the use of state-of-the-art mission autonomy and artificial intelligence technologies to enable fully autonomous operations during space missions

Automotive: This case study will consider Apollo deployed on a variety of prototype vehicles. It supports state-of the-art hardware such as latest LIDARs and cameras as well as GPU acceleration

Safexplain starts its activities

Safexplain starts its activities

The first face-to-face meeting of the project takes place on 26-27 October 2022 at the Barcelona Supercomputing Center (BSC) in order to discuss the first research project activities and next steps. The meeting is hosted by BSC at its premises.