In this interview, Leesa Joyce, Head of Research at Hololight, reflects on the evolution and impact of the Virtual Training Assistant use case developed as part of the VOXReality project. Drawing from HOLO’s initial efforts in XR-enabled training, she discusses how the team transitioned from simple task-oriented guidance to a more immersive, adaptive, and hands-free learning experience tailored to real industrial environments. Through key innovations such as voice-controlled AI assistance, free-mode assembly training, and high-fidelity workspace replication, HOLO has redefined the potential of XR technologies in industrial skills development and workforce training.
Reflecting on your leadership of the Virtual Training Assistant use case, what were the most significant innovations HOLO brought to the XR training use case?
HOLO initially developed a preliminary version of an intuitive XR training application for virtual assembly tasks. This early design relied heavily on assistive cues—such as color coding, guiding lines, automatic snapping, and audio prompts—which enabled users to complete assembly procedures quickly and with minimal effort.
While this approach demonstrated the potential of XR to simplify task execution, it also revealed critical limitations for industrial training contexts. Specifically, such systems ignore the knowledge gap: assembly line workers, construction personnel, and shopfloor employees might not have previous experience with XR technology and virtual interaction. Additionally, the user may complete tasks in the virtual environment without genuinely learning or internalizing the underlying assembly logic or machinery operation. In other words, while task completion is facilitated, procedural understanding and skill transfer to real-world contexts remain limited.
Furthermore, the reliance on hand-based interactions and hand menus presents practical drawbacks in industrial scenarios. Workers’ hands are often already occupied with physical tasks, making simultaneous interaction with virtual menus cumbersome and disruptive.
To address these challenges, HOLO’s contribution within the VOXReality project, specifically in the Virtual Training Assistant use case, introduced several significant innovations:
Free-Mode Assembly Training:
Unlike the earlier stepwise, rigid training approach, HOLO implemented a free-mode assembly system. This allowed trainees to explore and attempt logical assembly sequences independently, fostering deeper understanding of the assembly process rather than mere task completion.
Voice Interaction through ASR Integration:
HOLO integrated Automatic Speech Recognition (ASR) technology developed by UM, enabling natural speech input from users. Spoken commands were transcribed into text and processed by the XR system and the AI assistant.
AI Assistant Support:
A conversational AI agent was embedded into the training workflow, providing on-demand, context-aware support based on user requests. Importantly, the agent offered assistance without unnecessary interruptions, ensuring that users retained control of the training process.
Hands-Free Interaction:
By shifting the primary mode of interaction from hand-based menus to voice-based dialogue, the usecase freed users’ hands for assembly tasks. This design improvement addressed one of the key practical barriers for XR adoption in industrial training scenarios.
Together, these innovations transformed HOLO’s training application from a task-completion aid into a more effective learning tool, capable of supporting skill acquisition, promoting logical reasoning, and better aligning XR training with real-world industrial needs.
How did you achieve the digital replication of real workspaces in your AR solution, and in what ways has this contributed to improving trainee engagement and skill retention?
We achieved the digital replication of real workspaces by leveraging HOLO’s XR streaming technology, which offloads complex rendering tasks to high-performance servers. This approach enables the visualization of highly detailed, photorealistic holograms on XR headsets without compromising performance. Furthermore, the user interface was designed with a strong emphasis on intuitive and minimalistic visual elements, ensuring that trainees remain focused on the task at hand rather than being distracted by extraneous details. The combination of high-fidelity digital environments and carefully structured interaction design enhances immersion, which in turn supports higher levels of trainee engagement, immersion and contributes to improved skill retention.
What design strategies did you implement to tailor the training experience for users with varying levels of expertise and learning speeds, and how did you develop these strategies?
The core XR training environment was designed to remain consistent across all participants, ensuring a standardized baseline of learning. However, adaptability was introduced through the integration of the AI agent, which personalizes the assistance provided. Rather than altering the training content itself, the system dynamically adjusts the level and depth of support based on user requests and demonstrated needs. For experienced users, this typically translates into fewer interactions, with a focus on advanced or technical queries. In contrast, novice users would ask basic and contextual questions and are offered more contextual explanations. This adaptive approach allows the training to accommodate varying levels of expertise and learning speeds, in contrast to traditional group-based training methods where individual differences often receive limited attention.
What future technological advancements or integrations-such as Al voice assistance or physical tool tracking-are you most excited about for enhancing the training platform?
A significant future advancement would be enhancing the AI agent with scene learning capabilities, enabling it to interpret user intentions within the training environment. Insights from our research on the Training Assistant use case suggest that such context-aware understanding would substantially improve the system’s ability to provide adaptive guidance. Integrating this capability would represent a major step forward in extending the effectiveness and intelligence of the training platform.
Leesa Joyce
Head of Research, Hololight