The European research project SYNERGIA is developing an adaptive extended reality solution, which is being tested across five use cases in a variety of contexts.

These five use cases focus on industrial training, aerospace assembly, human-robot collaboration, speech rehabilitation, and road safety training for people with cognitive impairments.

Each experiment addresses a specific need and explores the possibilities offered by XR, generative artificial intelligence, Human Digital Twins (HDTs), and multimodal detection.

Here are the five use cases developed as part of the project.

Training and support service engineers

In the industrial sector, technicians’ travel requirements and the limited number of physical systems can slow down training, reduce its effectiveness, and limit its rollout. Current methods such as manuals, videos, or in-person training also lack interactivity, adaptability, and real-time feedback.

To address these challenges, SYNERGIA aims to create immersive virtual environments that simulate operations and enable remote training. Human Digital Support will be able to avoid stress or confusion in the service operation providing appropriate guidance.

A 3D assistant application powered by artificial intelligence also provides run time support during service procedures and adapts actions to the user’s state and their interactions.

Training content can be made accessible across different sectors and platforms through the use of dataspaces.

Targeted Impacts:

• 30% reduction in training time

• 40% improvement in training satisfaction

• 45% savings on training-related costs

• 20% improvement in task efficiency and reduction in human errors

• 60% reduction in capital expenditures related to training equipment

This use case will be driven by the needs of PRIMA Power, the project’s Italian partner.

Assembly line support

Traditional methods of aviation training and assembly (manuals, static videos, or mentor-led instruction) struggle to account for differences among operators, particularly their body type, experience, or physical condition. This can lead to longer onboarding times and human errors. Human Digital Twins will also assess the user’s condition during training and can trigger the 3D assistant to intervene.

SYNERGIA aims to develop dynamic XR tutorials for various drilling stations. These will incorporate official procedures, CAD models, and demonstrations performed by operators.

Step-by-step instructions can be enhanced with context-specific warnings using generative AI. Finally, immersive environments featuring virtual humans will allow procedures to be replicated in realistic situations.

Targeted Impacts:

• 15% reduction in training costs

• 20% improvement in learners’ well-being, mental state, and comfort

• 15% reduction in onboarding time

• 10% reduction in human-error-related mistakes

This use case will be driven by the needs of Airbus Atlantic, the project’s French partner.

Improving interactions between humans and robots

At present, collaborative robots are still underutilized due to their difficulty in adapting to their human colleagues, a lack of training programs, and safety concerns.

SYNERGIA is exploring a more adaptive form of interaction between humans and robots. Multimodal detection including eye tracking and gesture recognition will enable Human Digital Twins to anticipate human actions.

The robot’s behavior can then be adapted to the person’s actions particularly in terms of speed and force without the need for an explicit command.

XR training modules, combined with a 3D assistant based on generative AI and a tool for creating virtual worlds, will enable these interactions to be practiced in a risk-free environment. An XAI (Explainable Artificial Intelligence) module will also ensure the traceability and auditability of decisions.

Targeted Impacts:

• 15% improvement in productivity through increased automation

• 20% improvement in learner satisfaction

• 10% reduction in repetitive tasks traditionally performed by humans

• 15% reduction in onboarding time

This use case will be driven by the needs of HIS, the project’s Swedish partner.

Adapting speech therapy

Speech therapy may be limited by a shortage of therapists, long wait times, and inconsistent access to care. Adherence to treatment may also be low when exercises are not sufficiently tailored.

SYNERGIA proposes to develop an approach that personalizes speech therapy exercises. Hybrid Transformers/LSTM models will classify the characteristics of speech disorders using annotated data from the Hopale Foundation. The extraction of prosodic features will then allow the patient’s speech to be compared to adaptive normative models.

Generative AI will then be able to generate personalized speech exercises and motivational dialogues based on the patient’s mental and cognitive state.

A 3D assistant will simulate a virtual therapist and offer interactions tailored to the user’s context and emotional state. The XAI module, for its part, will ensure traceability and reliability in the feedback provided.

Targeted Impacts:

• 20% reduction in reliance on clinical resources

• 15% reduction in the workload of speech-language pathologists

• 15% improvement in adherence to interventions and scalable care models

This use case will be driven by the needs of the Fondation Hopale, the project’s French partner.

Making safety training more accessible

Workers with cognitive impairments currently have access to traditional tools that are not always accessible, scalable, or capable of adapting in real time. As a result, these tools are less easy to learn.

SYNERGIA aims to use multimodal detection of emotions and cognitive state to identify feelings such as frustration or confusion. The system will then be able to adjust the complexity and pace of tasks.

3D assistants based on generative AI will provide personalized and accessible guidance. A tool for creating immersive worlds will also enable the on-demand generation of realistic scenarios, including, for example, different traffic lights, weather conditions, or sources of distraction.

The system also includes contactless interactions, dynamic scenario adaptation, dashboards for trainers, and offline deployment capabilities. The XAI layer will aim to ensure GDPR compliance, transparency, and data protection.

Targeted Impacts:

• 20 to 30% reduction in training time

• 25% savings on training and operational costs

• 30% improvement in learner satisfaction

• 25% improvement in operational and task efficiency

• 15% reduction in human errors and performance variability

This use case will be driven by the needs of Gureak, the project’s Spanish partner.

Five ways to explore XR

Through these five use cases, SYNERGIA is putting a single ambition to the test: to develop an XR solution capable of adapting to needs, situations, and users.

Experiments will now continue in each of these areas. As the project progresses, we will share updates on these five use cases, the solutions developed, and the results of the experiments.

Stay tuned in the coming months to follow their progress and discover the next steps for SYNERGIA.