VOXReality
Voice-driven interaction in XR spaces
VAARHeT
Voice-Activated Augmented Reality Heritage Tours
Challenge:
Integration Challenge
The Voice-Activated Augmented Reality Heritage Tours (VAARHeT) sub-project is an integration challenge implementing voice-activated interactions in XR applications aimed at museum visitors.
Championing a user-centred approach, project partners XR Ireland and Āraiši ezerpils Archeological Parks designed and implemented three pilot applications to enliven the way visitors engage with the museum exhibits, while simultaneously extending the museum’s offerings and addressing challenges such as limited interpretive resources, accessibility and seasonal restrictions that apply particularly in open-air museums.
VAARHeT has implemented three pilot applications whose architecture will later be integrated in a content management system for cultural heritage sites and museums as a user-facing presentation layer.
- Pilot 1: XR Welcome Avatar – a voice-driven interactive mobile phone AR application, displaying an avatar providing basic information about the museum to visitors.
- Pilot 2: VR site augmentation – A headset-based, voice-interactive VR experience. Recognising natural speech by museum visitors, the application displays relevant educational content about building reconstructions at the museum.
- Pilot 3: AR live translation application – Available on wearable AR glasses and mobile phone, the application provides text translation from the speech of a tour guide or craft demonstrator at the museum
Key features include voice-driven conversational agents, multilingual translations delivered via visitors’ own smart devices, and an additional full VR voice-activated experience.
Through user testing and feedback rounds in-situ, VAARHeT documented the value of AI-assisted voice interaction in XR experiences, and the added value of XR experiences to the museum visitor experience. During the validation sprint usability tests were conducted on all three pilots with 39 test participants each in July 2025.
The VAARHeT pilot applications proved to be well received by museum visitors as an innovative approach to museum experiences making heritage content more interactive and accessible year-round. Museum visitors found that voice activation proved particularly useful in the mobile translation application and aided in the learnability of the VR experience.
The novelty of both the interaction modality and the XR design aspects of the applications presented the museum environment as a positive, contemporary, calm and educational space that visitors felt was particularly beneficial for families and young people.
The museum partner’s goal of raising awareness of Āraiši ezerpils Archaeological Parks as a destination with an extensive educational and novel offering was fulfilled, as especially educators and parents reacted positively to the project and the technology implementations.
For museum professionals from the region as well as national institutions, VAARHeT has been an opportunity to experience up-close the applications of novel technology for visitor engagement, leading to additional interest from policy and decision makers.
To conclude the VAARHeT project, the museum team will install the pilot application most popular with visitors – the VR experience – at Āraiši ezerpils Archaeological Parks, so that longer-term feedback on the value of the application to the visitor experience can be collected.
Use Case Description
The VAARHeT project represents a transformative use case for augmented reality (AR) applications in cultural heritage, especially in open-air museums like the Āraiši ezerpils Archeological Parks. Unlike conventional AR tours, which often rely on static content or manual interaction, VAARHeT introduces voice-activated engagement, allowing visitors to interact with a conversational AI that delivers dynamic personalised tours. This makes the experience more interactive, responsive, and tailored to individual preferences, significantly enhancing user engagement.
A standout feature of VAARHeT is its integration of neural machine translation (NMT), which provides real time multilingual support. Visitors can experience live guided tours in their preferred language, breaking down linguistic barriers and making the tours accessible to a wider, more diverse audience. This capability is particularly important to heritage sites that attract international visitors.
By using their own smart devices, visitors can access the AR experience without the need for specialised hardware, making the application scalable, cost-effective, and easy to adopt for museums. VAARHeT’s innovative use of extended reality (XR) technologies enhances the visitor experience while addressing critical challenges faced by cultural heritage sites, such as the resource-intensive nature of physical reconstructions and the limitations of seasonal tourism. Offering a year-round on-demand experience reduces operational strain and expands museums’ capacity to engage audiences. This combination of AI-driven personalisation, multilingual accessibility, and seamless integration with physical environments sets VAARHeT apart as a pioneering application of AR in the cultural heritage sector..
CONSORTIUM
Āraiši Ezerpils Archaeological Park
The Āraišu lake settlement is the only reconstruction of a 9th-10th century fortified settlement in Europe. The park is distinguished by its unique archaeological findings, the reconstruction’s location in its original setting and the well-preserved cultural landscape.
TEAM
Guillaume Auvray
XR Ireland
Founder of the Innovation Hub Nuwa Ltd, XR Ireland and the MSP 1Cluster, and former multilingual AI/ML Engineer at Apple, he leads the organisations as Head of Spatial Cloud Computing & XR Technologies combining 8+ years of technical expertise with hands-on experience in AI, XR and use case applications in the heritage, performing arts and humanitarian sector.
Cordula Hansen PhD
Senior Creative Developer specialised in User Experience Design brings to the sub project 10+ years of experience in immersive design and prototyping, notably in the heritage sector.
XR Ireland
Anastasiia Petrova
XR Ireland
Communications Officer overseeing communications and dissemination, ensuring innovative solutions and key project milestones reach the right audience with clear and impactful content across platforms.
Edu Prado
XR Ireland
Edu Prado is a multi passionate engineer, developer and producer, founder of Edu Prado Sounds, co-director of XR Ireland, and the mind behind Breaking The Inertia; working on a broad range of creative and innovative projects, from software and business development, to composing for media, producing artists and VR operas.
Anastasiia Zhmur
XR Ireland
Finance manager with solid experience working for international NGO’s multi million budgets.
Janis Meinerts
Araisi ezerpils Archaeological Park
Jānis Meinerts is an archaeologist and heritage specialist, his research interests include landscape and settlement archaeology, as well as regional studies and underwater cultural heritage.
Eva Koljera
Araisi ezerpils Archaeological Park
Eva Koljera is the manager of Araisi ezerpils Archaeological Park. With 20+ years of experience, Eva is a tourism and cultural heritage management specialist.
TESTIMONIALS
Guillaume Auvray
XR Ireland
“As technologists, we really appreciated the chance to both implement the latest developments in European research, but as importantly, to learn more about organisations in the cultural heritage sector. Working closely with Āraiši over the course of our VOXReality sub-project, we were able to really understand what drives museums, and what they expect for their visitors.
It was amazing to see how well XR was received by visitors, and how the voice interaction impacted the user experience.
For our company, the project was central in growing our connection with the cultural heritage sector. We are now looking forward to exploring commercial applications of the XR pilots further.”
Eva Koljera
Āraiši Ezerpils Archaeological Parks
“We want to present history in a modern way, especially one that is understandable to the younger generation. Joining XR Ireland as a partner on VAARHeT offered a great opportunity to get to know what is possible in the latest technology today. As cultural heritage professionals, we were happy to have this support from people who know the technology.
The technology testing phase helped us, as a museum, gain a deeper understanding of our visitors—their expectations and their relationship with modern technologies. It also provided valuable insights into what we might implement in the future”