Impulse Project 6: MIRASOFT - Mixed Reality-supported safeguarding of safety-critical automated driving functions

The aim of the MIRASOFT project was to use mixed reality technologies to simulate automated driving functions more realistically, make them more transparent and ensure they are safer – both for drivers and for vulnerable road users such as pedestrians.

In collaboration with partners from industry (AUDI AG, e:fs GmbH) and academia, a comprehensive simulation environment (Mixed Reality Environment, MiRE) was developed, building on the groundwork laid during the SAFIR set-up phase. It enables human behaviour, machine decisions and their interaction to be tested and optimised in a realistic manner.

Sub-project 1: Digitising pedestrian behaviour  

A key objective was to replicate pedestrian behaviour as realistically as possible in digital environments, a prerequisite for valid and reliable testing of automated vehicles. Using motion capture recordings of real pedestrian movements and standardised interfaces (OSI), a digital pedestrian model was created that replicates realistic behaviour in complex traffic situations. However, some fundamental challenges remain for future research: How can human behaviour be reliably modelled when it depends heavily on context, emotions and social norms? How can these models be deployed robustly under varying conditions (weather, culture, traffic density)?

Sub-project 2: Drivers and trust: How much information is enough?

Another area of focus was the driver’s perspective in highly automated vehicles. Studies using virtual reality and real user groups investigated how well people understand automated driving functions and how explanations affect trust, acceptance and the subjective sense of control. A tension became apparent: the better the systems explain themselves, the more trust users place in them – in some cases, even too much (known in the literature as ‘overtrust’). A so-called ‘illusion of control’ can arise, which is potentially dangerous if the actual possibilities for intervention are limited.

Sub-project 3: MiRE Architecture

The final part of the project focused on the further development and evaluation of the architecture underpinning the MiRE mixed reality test environment [1–3], which was initially implemented during the set-up phase of SAFIR. This sub-project was successfully completed in 2022.

Challenges & outstanding issues

Despite significant progress, key challenges remain, such as striking a balance between standardisation and flexibility, the transferability of simulations to real-world scenarios, and integration into existing systems. EuroNCAP also plans to adopt a simulation-based approach in future to support the type-approval of automated driving functions.

Results: With the MiRE test environment, the reality-based pedestrian models and the numerous human factors studies, MIRASOFT has laid a strong foundation for explainable, traceable driving functions, the validation of safety-critical situations and more transparent communication between vehicles and people.

The tools, interfaces and findings developed will remain available to the research community and industry even after the project has ended, with the potential for spin-offs and integration into European initiatives such as EuroNCAP, or for further development within the framework of Horizon Europe projects.

MIRASOFT demonstrates how mixed reality can be key to making automated driving safer, more understandable and more widely accepted. However, widespread adoption is still some way off and requires further research to develop robust standards.