iEXODDUS Advancing Human-Aware Perception for Operational Design Domain (ODD) Extension

THI Contribution

As part of iEXODDUS, the Computer Vision for Intelligent Mobility Systems (CVIMS) group at AIMotion Bavaria, THI coordinates the Work Package “Extended Perception & Decision Making”, focusing on advanced visual detection capabilities to enhance safety and efficiency in work zones, incidents, and tunnels. In addition, the CVIMS provides datasets for the work package led by CARISSMA to develop a behavioral model for traffic participants to enable the Connected Automated Vehicles (CAV) through work zones, tunnels, and incident areas.

The CVIMS contributes its core expertise in computer vision and AI, particularly in human-centered perception, contextual understanding, and the development of interpretable models for occupant behavior. This work lays the foundation for adaptive, safe, and user-aware automated driving functionality.

Project Team

The Computer Vision for Intelligent Mobility Systems (CVIMS) research group specializes in computer vision, multimodal generative models, and autonomous driving functionality. Our team has expertise in multimodal perception, as well as in end-to-end autonomous driving functionality.

Within iEXODDUS, CVIMS assesses safety and traffic impacts resulting from automation in the relevant Operational Design Domains (ODDs). We provide validated input for traffic simulations and support deep learning development for environment perception, enabling safe navigation within and beyond defined ODDs.

Prof. Dr. Torsten Schön supervises this workflow. Xujun Xie, a PhD student at CVIMS, is actively involved in implementing this work package.

In addition, CVIMS contributes to a separate work package led by Prof. Dr. Werner Huber at THI’s CARISSMA institute, which focuses on traffic simulation and the behavioral model of traffic participants.

Through these efforts, our work in iEXODDUS builds on CVIMS’s long-standing research in multimodal perception, end-to-end autonomous driving, further advancing human-centered innovation in mobility.
 

Tasks

CVIMS is responsible for the following tasks within iEXODDUS:

• Robust detection, classification, localization, and interpretation of construction site–specific objects and hazards.
• Visual detection and categorization of road work zones.
• Context-based in-vehicle perception for road work zones.
• Detection of vulnerable road users (VRUs) in road work zones to ensure worksite safety.
• Free-space segmentation under challenging conditions within construction sites and incident areas.
• Collection, annotation, simulation, and publication of public data as open-source resources within the project.
• Safe free-space detection for trajectory planning in incident scenarios involving road obstacles.
• Extension of the open-source autonomous driving simulator CARLA to simulate diverse types of work zones.

Perception data generated through these tasks will feed deep learning models that have been trained on a combination of real-world measurements and simulated data. The primary objective is to accurately detect and interpret key features of road work zones, such as lane markings and road objects, that are critical to driving safety.

Key Features

CVIMS develops robust perception methods tailored to dynamic, irregular traffic environments that are critical for extending Operational Design Domains (ODDs). These environments include urban canyons, tunnels, and road work zones.

A central focus is on context-aware scene understanding. CVIMS applies key point-based visual detection to identify critical interaction zones, such as entry and exit points in constrained driving spaces. Traditional object-detection methods are insufficient in such scenarios due to the undefined nature of obstacles and markings. Instead, the models developed by CVIMS anticipate dense motion flows and occupancy patterns throughout the 3D environment.

To enable generalization beyond standard categories, open-vocabulary scene comprehension is employed. This allows the system to adapt to novel elements during inference, particularly relevant for the highly variable layout of temporary work zones.

By integrating perception inputs with environmental context, CVIMS contributes to the safe and scalable deployment of automated vehicles in complex, real-world conditions.

Impact

iEXODDUS bridges the gap between research and deployment by developing automated driving technologies that are viable for real-world application. The project involves close collaboration with industry partners to ensure that the developed systems meet the requirements of manufacturers, infrastructure providers, and regulators.

One key outcome of the project is a unified framework that enables reliable perception and interpretation of complex traffic settings, such as work zones, and real-time communication of this information to relevant vehicle systems and road users.

CVIMS contributes to this impact by developing AI-based perception modules that enhance occupant understanding and contextual awareness in constrained scenarios. These components play a critical role in extending ODDs while maintaining a high level of safety and explainability. Through its contributions, CVIMS ensures that perception systems are both technically robust and practically validated.

Partners

iEXODDUS is a collaborative project supported by 15 partners across Europe.

As part of a larger research consortium, CVIMS cooperates closely with Ford Otosan (Turkey), CEIT (Spain), CEA (France), and AVL (Austria). Together, we develop datasets, perception pipelines, and evaluation strategies that feed into the broader system integration efforts of the project.