Written by: IKERLAN
To enhance safety in AI-based systems in the railway domain, the SAFEXPLAIN project has been working to integrate cutting-edge simulation technologies with robust communication frameworks. Doing so has unlocked new possibilities for system validation and performance benchmarking, bringing validation closer to real-world scenarios.
For example, integrating Unreal Engine (UE) 5 with Robot Operating System 2 (ROS 2) generates dynamic, interactive simulations that accurately replicate real-world railway scenarios. This integration facilitates the transmission of critical ground truth data—including train speeds, stereo camera imagery, and obstacle distances—from the simulated environment to the real systems. This facilitates the comprehensive testing and benchmarking of complex task such as object detection, and safety-related train functionalities, such as braking control.
Figure 1 illustrates this workflow:
- Images and data from simulated environments in UE5 are transmitted through the UE5-to-ROS2 bridge and processed by the Obstacle detection and suggestion system.
- This system evaluates the data to determine the presence of critical obstacles, and responds accordingly by performing actions such as braking or issuing warnings.
- The recommendations are sent to the Train’s Operational Control, where a human driver supervises the suggested actions. If the driver notices that the suggestion system is not working as expected, they can deactivate it, and ignore all the suggestions.
- Depending on the action required action, the system will calculate next train speed, which is communicated back to the simulation via the bridge.
Figure 1. Closed loop simulated system.
This closed-loop process allows engineers to precisely measure outcomes, such as the train’s position upon braking. This approach evaluates the Obstacle detection and suggestion system. More information about this can be found in deliverable D5.1.
UE5 & ROS2 integration for closed-loop simulation and advantages for the railway case
Unreal Engine 5: Crafting Railway Scenarios
UE 5, renowned for its high-fidelity graphics and real-time rendering capabilities, serves as an ideal platform for constructing immersive railway simulations. Developers can design intricate scenarios involving various obstacles on or near the tracks, ranging from stationary objects like static pedestrians to dynamic entities such as vehicles crossing at level crossings, as it is depicted in Figure 2.
Figure 2. Simple example of UE 5 scenario
These virtual environments provide a safe, cost-effective approach to generating tests for validating railway-domain systems under diverse conditions, including varying weather patterns and lighting situations. Many of these complex scenarios are either difficult or prohibitively expensive and time-consuming to replicate in real life. UE 5 addresses this challenge effectively, ensuring comprehensive system evaluation before deployment into real-world operations..
ROS 2: Facilitating Seamless Communication
By leveraging ROS 2’s publish-subscribe architecture, data generated within UE 5—such as train speed metrics, stereo camera feeds, and obstacle distance measurements—can be transmitted in real-time to the control algorithms. This setup mirrors the data flow in actual train operations, allowing engineers to assess how the control systems interpret and respond to sensory inputs. The integration of ROS 2 ensures that the simulation data is handled in a standardized manner, promoting compatibility with the middleware developed within the project.
Closed-Loop Simulation: Enhancing System Validation
The integration of UE5 and ROS 2 facilitates the creation of closed-loop simulations, where the simulation environment and the train’s control systems interact, allowing for real-time feedback and adaptation. This strategy enables engineers to observe and analyze the dynamic interplay between the train’s control mechanisms and its environment, leading to more thorough and accurate validation of safety functions.
Advantages of Closed-Loop Simulation with UE5 and ROS 2
- Realistic Scenario Testing: UE5’s advanced rendering capabilities enable the creation of lifelike environments, ensuring that control systems are tested under conditions that closely resemble real-world operations.
- Real-Time Feedback: The closed-loop nature of the simulation allows for immediate observation of control system responses, facilitating rapid identification and correction of potential issues.
- Cost-Effectiveness: Conducting extensive testing in a virtual environment reduces the need for costly physical prototypes and on-track testing, accelerating the development process.
- Enhanced Safety: Simulating hazardous scenarios in a controlled virtual setting eliminates risks to personnel and equipment, ensuring that safety protocols can be thoroughly evaluated without real-world consequences.
Case Study: AI-Based Obstacle Detection Integration
A pertinent example of this integration is the development of a formal model of train control incorporating AI-based obstacle detection. In this approach, an AI system processes visual data to identify potential obstacles, and the control system determines appropriate responses, such as initiating braking procedures. The closed-loop simulation allows for the evaluation of the AI’s accuracy and the control system’s effectiveness in real-time, providing valuable insights into the system’s overall safety and performance.
