AIXTRA

AI-Assisted XR Collaboration and Training with Multilingual Support.

Challenge:

Integration challenge

AIXTRA’s consortium consists of LEFX GmbH and MindPort GmbH, two recognized experts in the field of enterprise XR, with a strong focus on VR training.

MindPort is the company behind VR Builder, the free and open source Unity plug-in for VR app development. With over 10.000 downloads and a consistent 5 star rating for almost 3 years since publishing, it is one of the most successful add-ons for VR in Unity.

LEFX specializes in interactive digital formats. Since 2011, they have delivered high-quality real-time visualizations, product presentations, and VR/AR applications across several sectors. LEFX excels in enterprise-level XR and real-time 3D applications. LEFX has been a strong project contributor to VR Builder since its inception. 

During our shared years of experience, we uncovered two huge and reappearing challenges in enterprise projects, for which VOXReality is the perfect solution. Within AIXTRA, we will solve these actual needs of many enterprise clients with a three component system delivering two new and innovative capabilities, namely:

  • Overcoming the Language Barrier: In VR-based training, language differences can hinder communication. AIXTRA tackles this by integrating AI-based in-VR translation, allowing participants to communicate seamlessly in their native languages. A demo will show participants completing complex tasks together with real-time translations to enhance understanding and collaboration. This will improve communication, making the learning experience more inclusive and accessible.
  • AI-Based Virtual Training Partner: Multi-user training can be very impactful, but requires multiple participants to be available at the same time. If replaced by single-user approaches, the quality of the training suffers because processes requiring cooperation are replaced by predefined interaction steps or entirely omitted. AIXTRA seeks to address this by integrating AI to simulate human-to-human interactions. This AI-based virtual training partner allows trainees to request assistance and communicate about their current status, enabling rich training interactions without the need for a physical training companion.

By focusing on these two objectives, AIXTRA seeks to create a more effective and accessible VR training environment, ultimately contributing to the advancement of VR training methodologies and technologies and the VOXReality project as a whole.

The three systems are:

  • VOXReality System, which unlocks the VOXReality capabilities for our two use cases, as described above.
  • The Authoring Tool based on VR Builder which integrates the new capabilities from the VOXReality System and makes them easily accessible.
  • A Demo Application, showcasing those new capabilities in real-world scenarios taken from actual challenges that occurred when working on enterprise projects.

AIXTRA will transform VR training by integrating VOXReality technologies with the VR Builder toolset, making advanced, AI-enhanced training scenarios easier to develop and implement. This will broaden the reach of VOXReality technologies, connecting global teams and driving the adoption of cutting-edge AI solutions. By enabling simpler yet comprehensive training scenarios, AIXTRA will enhance collaboration, inclusivity, and the overall effectiveness of VR training, contributing significantly to the future of professional development and education.

Use Case Description

The advent of real-time translation, VR/XR training, multinational group VR/XR training and AI-based virtual training partners is transforming various fields by dismantling barriers and enhancing the learning experience. The implementation of real-time translation and AI-based coaches have the potential to facilitate global collaboration by enabling seamless communication between multinational teams. Furthermore, it can improve coach services by allowing agents to assist trainees in their native language and expand educational opportunities by allowing them to access content and participate in courses in different languages. 

However, VR/XR training provides immersive training experiences for remote workers, simulates hazardous environments for safe, hands-on training in industries such as construction and manufacturing, and allows medical professionals to practice procedures in a risk-free virtual environment together. Artificial intelligence-based virtual training coaches facilitate personalized learning experiences by adapting training applications to individual learning styles and paces. They offer continuous training opportunities without the need for human trainers to be present and monitor and analyze performance data to identify areas for improvement and optimize training applications. 

In addition, the possibility to create one training application in a specific language and auto-translate it to various languages improves the scalability of such training applications. The usage of AI-based virtual training partners to provide greater variety would also be an effective method of enhancing the experiential aspect of VR/XR training. Through our authoring tool, companies can create their own training using AI-based virtual training partners, automated real-time translation and existing functionalities of VR Builder to provide better VR/XR training experiences. 

So, multi-User-Training scenario with strong focus on communication and collaboration to showcase and evaluate the realtime translation functionalities with a close-to-real-world process. As well as, single-User-Training on a similar real-world scenario, which usually would require collaboration with another human operator, which will instead be achieved by utilizing the AI-enabled functionalities of the AIXTRA-project.

CONSORTIUM

Mindport logo

MindPort GmbH

Democratizing Immersive Experiences.

LEFX GmbH

XR specialists with strong focus on high quality real-time content.

TEAM

David
Lähner

Mindport

Founder & CEO

Sebastian Pötter​

Mindport

Software Engineer

Markus Wellmann​

Mindport

Senior Developer

Till
Schure

LEFX GmbH

XR-Developer

Tom Micklich

LEFX GmbH

Co-Founder & Co-CEO

Aron Schaub

LEFX GmbH

Unity Developer

MENTOR

Olga Chatzifoti

Gruppo Maggioli

Shopping Basket