In this fifth installment of our Partner Interview series, Leesa Joyce, Head of Research Implementation at Hololight, sits down with Carina Pamminger, Head of Research at Hololight, to explore their organization’s pivotal role in the VOXReality project. As a leader in extended reality (XR) technology, Hololight is pushing the boundaries of augmented reality (AR) solutions, particularly within industrial training applications. Through their work on the Virtual Training Assistant use case, Carina sheds light on how AR is transforming training processes by integrating AI-driven interactions and real-time performance evaluation. The interview delves into the innovative ways AR is being utilized to enhance assembly line training, the incorporation of safety protocols, and the future of immersive learning experiences at Hololight.
Can you provide an overview of your organisation's involvement in the VOXReality project and your specific role within the consortium?
HOLO is an extended reality (XR) technology provider contributing with its augmented reality (AR) solutions to stream and display 3D computer-aided-design (CAD) models and manipulate them in AR environment. HOLO is a task leader where it is leading the development of novel interactive XR applications. HOLO is also the leader of the use case “Virtual Training Assistant”. The Training Assistant use case revolves around the enhancement of an AR industrial assembly training application with the goal of enhancing the training process by incorporating the automated speech recognition (ASR) model and dialogue system of VOXReality. Conventional training techniques frequently exhibit a deficiency in interactivity and adaptability, resulting in less-than-optimal educational results. Through the integration of artificial intelligence within the AR setting, this scenario aims to establish a more captivating and efficient training atmosphere. Noteworthy characteristics of the application encompass the visualization and manipulation of 3D CAD files within the AR environment, an interactive virtual training aide featuring real-time performance evaluation, as well as a dynamic dialogue system driven by natural language processing (NLP) and speech-to-text functionalities.
The prime constituent of the training assistant technology is the application Hololight Space Assembly. Trainees are guided to precisely assemble components within the CAD model, ensuring everything fits perfectly. The system effortlessly integrates with pre-existing asset bundles, providing all the necessary details, such as CAD files, tools, and additional elements like tables or shelves. It also includes intuitive scripts for model interaction, easy-to-navigate menus, and smart algorithms to enhance the assembly experience. In addition, Assembly leverages Hololight Stream to remotely render the application from a high-performance laptop to AR smart glasses, overcoming the device’s rendering limitations. This remote rendering and streaming setup allows the AR training application to be hosted on a powerful laptop (server) and seamlessly streamed to the HoloLens 2 (client).
How is AR seamlessly integrated into training applications, and what specific advantages does it bring to the learning experience?
Integrating AR into training applications allows assembly line workers to train in a highly realistic, digitally replicated environment that mirrors their actual workspace. This immersive experience helps workers develop muscle memory and recognize environmental cues, making the transition to the real assembly line smoother and more intuitive. Since the training environment is digital, it can be accessed from anywhere, at any time, providing flexibility and convenience for both trainees and companies.
Moreover, AR-based training is resource-friendly and cost-effective. Multiple workers can use the same training files repeatedly, allowing for efficient use of resources. The digital nature of the environment also means that training scenarios can be easily modified, redesigned, or personalized to meet specific needs, enhancing the learning experience. By incorporating sensory cues, AR helps reinforce learning, making it a powerful tool for building skills that are critical in a fast-paced, high-precision environment like an assembly line.
How does the AR training application cater to different skill levels among trainees, ensuring a gradual learning curve for beginners and challenging modules for more experienced assembly technicians?
The AR training application is designed to accommodate various skill levels, ensuring that both beginners and experienced assembly technicians can benefit from the training. For those with some assembly knowledge but needing to master a new object / engine / machine, the difficulty modes come in handy. These modes guide trainees through the correct order of assembly, gradually increasing in complexity. This personalized approach allows the training to adapt to the expertise and learning pace of everyone, making it accessible to slow learners while still providing a challenge for those who pick up the process quickly.
By progressing through these difficulty levels, trainees not only learn the assembly process but also reinforce it through repetition, ensuring they internalize each step. As they clear each difficulty mode, they build confidence and gradually commit the entire process to memory. This approach ensures that by the end of the training, regardless of their initial skill level, all trainees will have mastered the assembly process and be fully prepared to apply their knowledge on the actual assembly line.
Considering the critical nature of turbine assembly, how does the AR application incorporate safety protocols and guidelines to ensure that trainees adhere to industry standards during the training process?
The AR application prioritizes safety by guiding trainees through the correct order of turbine assembly, creating a complete awareness about the process and the parts that need to be handled, reducing the likelihood of mistakes that could lead to serious risks in real-life scenarios. By learning and practicing each step in a controlled, digital environment, trainees can focus on mastering the process without the immediate dangers associated with heavy machinery. This approach ensures that they are well-prepared to follow industry standards and protocols when transitioning to the actual assembly line, where adherence to safety guidelines is critical.
However, some safety aspects remain areas for improvement. Currently, ergonomic assessments can only be conducted in real-life settings, requiring external analysis to ensure proper posture and technique. Additionally, the integration of Personal Protective Equipment (PPE) within the AR training is limited due to compatibility issues between safety goggles and AR glasses. While the application effectively reduces risks by teaching the correct assembly sequence, future developments could enhance safety training by incorporating ergonomic evaluations and better PPE integration.
Looking ahead, what plans are in place for future enhancements and expansions of the AR training application for turbine assembly? Are there additional features or modules on the horizon to further enrich the learning experience?
The AR training application for turbine assembly is set to undergo significant enhancements, particularly with the integration of VOXY, an AI-assisted dialogue agent with voice assistance. VOXY is already a game-changing addition, streamlining interactions within the application by eliminating the need for clumsy AR hand gestures. This ensures a smoother, more immersive experience, allowing users to stay fully engaged with the training process. VOXY also introduces AI-driven support, making it easier for trainees to navigate complex assembly tasks while receiving real-time guidance and feedback.
Future expansions include developing a platform to host training files and an analysis mode to evaluate trainee performance more comprehensively. We’re also exploring the incorporation of real, trackable tools in the AR environment, enabling physical interaction with virtual elements to improve ergonomics and weight memory. Additionally, we’re researching ways to integrate safety equipment into the AR training, with ongoing efforts under the SUN project funded by Horizon Europe. These enhancements will not only enrich the learning experience but also ensure that trainees are better prepared for the physical demands and safety requirements of turbine assembly.
Leesa Joyce
Head of Research Implementation at Hololight
&
Carina Pamminger
Head of Research at Hololight
