VOXReality template LinkedIn v3 (2).pdf (Instagram Post) (6)

Partner Interview #3 with Visual Computing Lab (VCL)@CERTH/ITI

by Blog post

In this third installment of our Partner Interview series, we had the pleasure of speaking with Petros Drakoulis, Research Associate, Project Manager, and Software Developer at the Visual Computing Lab (VCL)@CERTH/ITI, about their critical role in the VOXReality project. As a founding member of the project team, CERTH brings its deep expertise in Computer Vision to the forefront, working at the intersection of Vision and Language Modeling. Petros shares how their innovative models are adding a “magical” visual context to XR experiences, enabling applications to understand and interact with their surroundings in unprecedented ways. He also provides insights into the future of XR, where these models will transform how users engage with technology through natural, conversational interactions. Petros highlights the challenges of adapting models to diverse XR scenarios and ensuring seamless cross-platform compatibility, underscoring CERTH’s commitment to pushing the boundaries of immersive technology.

What is your specific role within the VOXReality Project?

CERTH has been a key contributor to the project since its conception, since it has been among the founding members of the proposal team. As one of the primal research institutes in Europe, our involvement regards research conduction and technology provision to the team. In this project, specifically, we saw a chance we wouldn’t miss; to delve into the “brave new world” of Vision and Language Modeling. A relatively new field that lies at the intersection of Computer Vision, which is our lab’s expertise, and Natural Language Processing, an excessively flourishing field with the developments in Large Language Models and Generative AI (have you heard of ChatGPT? 😊). Additionally, we work on how to train and deploy all these models efficiently, an aspect extremely important due to the sheer size of the current model generation and the necessity for green transition. 

Could you share a bit about the models you're working on for VOXReality? What makes them magical in adding visual context to the experiences?

You set it nicely! Indeed, they enable interaction with the surrounding environment in a way that some years ago would seem magical. The models take an image or a short video as input (i.e. as seen from the user), and optionally a question about it, and provide a very human-like description of the scene or an answer to the question. This output can then be propagated to the other components of the VOXReality pipeline as “visual context”, endowing them with the ability to function knowing where they are and what is around them; effectively elevating their level of awareness. Speaking about the latter, what is novel about our approach is the introduction of inherent spatial reasoning, built deep into the models enabling them to fundamentally “think” spatially. 

Imagine we're using VOXReality applications in the future – how would your models make the XR experience better? Can you give us a glimpse of the exciting things we could see?

The possibilities are almost limitless and as experience has shown creators rarely grasp the full potential of their creations. The community has an almost “mysterious” way of stretching whatever is available to its limits, given enough visibility (thank you F6S!). Having said that, we envision a boom in the end-user XR applications integrating Large Language and Vision models, enabling users to interact with the applications in a more natural way, using primarily their voice in a conversational manner together with body language. We cannot, of course, predict how long this transition might take or to what extent the conventional Human-Computer Interaction interfaces, like keyboards, mice and touchscreens will be deprecated but the trend is obvious, nevertheless. 

In the world of XR, things can get pretty diverse. How do your models adapt to different situations and make sure they're always giving the right visual context?

It is true that in “pure” Vision-Language terms, a picture is worth a thousand words that some of them may be wrong 🤣!  For real, any Machine Learning model is only as good as the data it was trained on. The latest generation of AI models is undoubtedly exceptional, but largely due to learning from massive data. The standard practice today is to reuse pretrained models developed for another, sometimes generic, task and finetune them for the intended use-case, never letting them “forget” the knowledge they acquired from previous uses. In that sense, in VOXReality we seek to utilize models pretrained and then finetuned for a variety of tasks and data, which are innately competent to treat diverse input. 

In the future XR landscape, where cross-platform experiences are becoming increasingly important, how is VOXReality planning to ensure compatibility and seamless interaction across different XR devices and platforms?

Indeed, the rapid increase of edge-device capabilities we observe today is rapidly altering the notion of where the application logic should reside. Thus, models and code should be able to operate and perform on a variety of hardware and software platforms. VOXReality’s provision in this direction is two-fold. On one hand, we are developing an optimization framework that allows developers to fit initially large models to various deployment constraints. On the other hand, we definitely put emphasis on using as many platform-independent solutions as possible, in all stages of our development. Some examples of this include: the use of a RESTful API based model inference scheme, the release of all models in image-container form and the ability to export them into various cross-platform binary representations such as the ONNX. 

Picture of Petros Drakoulis

Petros Drakoulis

Research Associate, Project Manager & Software Develper at Visual Computing Lab (VCL)@CERTH/ITI

Twitter
LinkedIn
VOXReality template LinkedIn v3 (2).pdf (Instagram Post) (4)

Partner Interview #2 with MAASTRICHT UNIVERSITY

by Blog post

In our second Partner Interview, we had the opportunity to discuss the VOXReality project with Konstantia Zarkogianni, Associate Professor of Human-Centered AI at Maastricht University. As the scientific coordinator of VOXReality, Maastricht University plays a crucial role in the development and integration of neural machine translation and automatic speech recognition technologies. Konstantia shares her insights into how Natural Language Processing (NLP), Computer Vision (CV), and Artificial Intelligence (AI) are driving the future of Extended Reality (XR) by enabling more immersive and intuitive interactions within virtual environments. She also discusses the technical challenges the project aims to overcome, particularly in aligning language with visual understanding, and emphasizes the importance of balancing innovation with ethical considerations. Looking ahead, Konstantia highlights the project’s approach to scalability, ensuring that these cutting-edge models are optimized for next-generation XR applications.

What is your specific role within the VOXReality Project?

UM is the scientific coordinator of the project and responsible for implementing the neural machine translation and the automatic speech recognition. My role in the consortium is to monitor and supervise UM’s activities while providing my expertise in the ethical part of AI along with the execution of the pilots and the open calls.  

How do you perceive the role of Natural Language Processing (NLP), Computer Vision (CV), and Artificial Intelligence (AI) in shaping the future of Extended Reality (XR) as part of the VOXReality initiative?

The VOXReality’s technological advancements in the fields of Natural Language Processing, Computer Vision, and Artificial Intelligence pave the way for future XR applications capable of offering high level assistance and controlling. Language enhanced by visual understanding constitutes the VOXReality’s main medium for communication that it is implemented based on the combined use of NLPs, CV, and AI. The seamless fusion of linguistic expression and visual comprehension offers immersive communication and collaboration revolutionizing the way humans interact with virtual environments.  

What specific technical challenges is the project aiming to overcome in developing AI models that seamlessly integrate language and visual understanding?

Within the frame of the project, innovative cross-modal and multi-modal methods to integrate language and visual understanding will be developed. Cross-modal representation learning will be applied to capture both linguistic and visual information through encoding the semantic meaning of words and images in a cohesive manner. The generated word embeddings will be aligned with the visual features to ensure that the model can associate relevant linguistic concepts with corresponding visual elements. Multi-modal analysis involves the development of attention mechanisms that endorse the model with capabilities to focus on the most important and relevant parts of both modalities.  

How does the project balance technical innovation with ethical considerations in the development and deployment of XR applications?

VOXReality foresees the implementation of three use cases: (i) digital agent assisting the training of personnel in machine assembly, (ii) virtual conferencing offering a shared virtual environment that allows navigation and chatting among attendees speaking different languages, and (iii) theatre incorporating language translation and visual effects. Focus has been placed to take into consideration the ethical aspect of the implemented XR applications. Prior to initiating the pilots, the consortium identified specific ethical risks (e.g. misleading language translations), prepared relevant informed consents, and drafted a pilot study protocol ensuring safety and security. Ethical approval from the UM’s ethical review committee has been received to perform the pilots.   

Given the rapid evolution of XR technologies, how is VOXReality addressing challenges related to scalability and ensuring optimal performance in next-generation XR applications?

The VOXReality technological advancements in visual language models, automatic speech recognition, and neural machine translation feature scalability and are provided to support next-generation XR applications. Having as goal to deliver these models in the form of plug-n-play optimized models, modern data-driven techniques are applied to optimize models’ inference time and storage requirements. To this end, a variety of techniques are being investigated to transform unoptimized PyTorch models to ONNX hardware optimized ones. Except from the VOXReality pilot studies that implement three use cases, new XR applications will also be developed and evaluated within the frame of the VOXReality open calls. The new XR applications will be thoroughly assessed in terms of effectiveness, efficiency, and user acceptance.   

Picture of Konstantia Zarkogianni

Konstantia Zarkogianni

Associate Professor of Human-Centered AI, Maastricht University, MEng, MSc, PhD

Twitter
LinkedIn
VOXReality template LinkedIn v3 (2).pdf (Instagram Post) (3)

Partner Interview #1 with ADAPTIT S.A.

by Blog post

In our first Partner Interview, Spyros Polychronopoulos from ADAPTIT S.A. discusses their role in developing the AR Theatre application for the VOXReality project. As XR technology experts, ADAPTIT has been deeply involved in the design and deployment process, ensuring that the technology aligns with live theatre needs. They’ve focused on user-friendly interfaces, seamless integration with theatre systems, and secure data protocols to protect intellectual property. Spyros also highlights strategies for future-proofing the application, including modular design and cross-platform development, with plans to adapt to emerging XR technologies and broaden access to theatre through affordable AR devices.

What is your specific role within the VOXReality Project?

Our organization, in our capacity as XR technology experts, has undertaken the design, development and deployment of the AR Theatre application. We have been engaged in the design process since the early beginning, e.g. in discussing, interpreting and contextualizing the user requirements. Throughout the process, we have been in close contact with the theatrical partner and use case owner, offering technological knowledge transfer to their artistic and management team. This work frame for operations has proven critical to facilitating team-based decision-making during design, and thus keeping in view the needs of both the XR technology systems and the theatrical ecosystem. 

To facilitate our communication in an interdisciplinary team and to consolidate our mutual understanding, we have taken the lead in creating dedicated applications as deemed necessary. 

Firstly, to render the VOX Reality capabilities in tangible, everyday terms, we created an easily distributable mobile application which demonstrates the VOX Reality models one by one in a highly controlled environment. This application can also function as a dissemination contribution for the VOX Reality project goals. We proceeded with developing a non-VOX Reality related AR application to practically showcase the XR device capabilities to the theatrical partner, and more specifically, to the team’s theatrical and art director with a focus on the device’s audiovisual capabilities.

Furthermore, we combined the two previous projects in a new AR-empowered application to better contextualize the VOX Reality services to a general audience which is unfamiliar with AR. Since that milestone, we have been developing iterations of the theatrical application itself with increasing levels of complexity. Our first iteration was an independent application running on the XR device which simulates the theatrical play and user experience. It was produced in independent mode for increased mobility and testing and was used extensively for documenting footage and experientially evaluating design alternatives. The second iteration is a client-server system which can allow multiple XR applications to operate in sync with each other. This was performed for simulated testing in near-deployment conditions during development and was targeted on evaluating the more technical aspects of the system, like performance and stability. The third and last iteration will incorporate all the physical theatrical elements, specifically the actors and the stage, and will involve the introduction of yet new technology modules with their own challenges.  

In summary, this has been a creative and challenging journey so far, with tangible and verifiable indicators for our performance throughout, and with attention to reusability and multifunctionality of the developed modules to reinforce our future development tasks. 

As for my personal involvement, this has been a notably auspicious coincidence, since I myself am active in theatrical productions as a music producer and devoted to investigating the juncture of music creation and AI. 

What considerations went into selecting the technology stack for the theatre use case within VOXReality, and how does it align with the specific requirements of live theatrical performances?

Given the public nature of the theatrical use case, the user facing aspects of the system, specifically, the XR hardware and XR application user interface, were an important consideration.  

In terms of hardware, the form factor of the AR device was treated as a critical parameter. AR glasses are still a developing product with a limited range of devices that could support our needs. We opted for the most lightweight available option with a glass-like form to achieve improved comfort and acceptability. This option had the tradeoff of being cabled to a separate computing unit, which was considered of least concern to us given the seating and static arrangement in the theatre. In more practical terms, since the application should operate with minimal disturbance in terms of head and hand movement, in silence and in low light conditions, we had decided that any input to the application should be made using a dedicated controller and not hand tracking or voice commands. 

In terms of user interface design, we selected a persona with minimal or no XR familiarity and that defined our approach in two ways:  1) we chose the simplest possible user input methods on the controller and we implemented user guidance with visual cues and overlays. We added a visual highlight to the currently available button(s) at any point and in the next iteration, we will expand on this concept with a text prompt on the functionality of each button, triggered by user gaze tracking. 2) we tried to find the balance between providing user control which allows for customization/personalization and thus improved comfort, and limiting control which safeguards the application’s stability and removes cognitive strain and decision-making from the user. This was addressed by multiple design, testing and feedback iterations. 

How does the technical development ensure seamless integration with existing theatre systems, such as lighting, sound, and stage management, to create a cohesive and synchronized production environment?

As in most cases of innovative merging of technologies, adaptations from both sides of the domain spectrum will need to be made for a seamless merger. One problematic area involves the spatial mapping and tracking system needs for XR technology. Current best practices for its stable operation dictate conditions that typically do not match a theatrical setup: it requires well-lit conditions, stable throughout the experience, performs best in small/medium sized areas, needs surfaces with clear and definite traits that avoid specific textures, etc.  Failure of the spatial mapping and tracking system can lead to misplaced 3D content which no longer matches the scenography of the stage and thus breaks immersion and suspension of disbelief for the user. In some cases, failure may also lead to a non-detection or inaccurate detection of the XR device controller(s), thus impeding user input. 

To amend this, recommendations for the stage’s scenography can be provided from the technical team to the artistic team. Examples are to avoid reflective, transparent, or uniform in color (especially avoiding the color black) surfaces, or surfaces with strong repeating patterns.  Recommendations can also address non-tangible theatrical elements, like the lighting setup. Best practices advise avoiding strong lighting that produces intense shadows or dip areas in total or near-total darkness. 

Furthermore, there are spatial tracking support systems that a director may choose to integrate in experimental, narrative or artistic ways. One example is the incorporation of black-and-white markers (QR, ARUCO, etc) as scenography elements which have the practical function of supporting the accuracy of the XR tracking system or extending its capabilities (e.g. tracking moving objects).  

Going even further, an artistic team may even want to examine a non-typical theatre arrangement which can better match the XR technology needs and lead to innovative productions. On example is the round theatre setup, which has a smaller viewing distance between audience and actors and an inherently different approach to scenography (360° view). Other even more experimental physical setups can involve audience mobility, like standing or walking around, which can make even more use of the XR capabilities of the medium in innovative ways, like allowing the users to navigate a soundscape with invisible spatial audio sources or discover visual elements alongside pre-designed routes or from specific viewing angles. 

In terms of audio input, the merger has less parameters. Currently, users are listening to the audio feed from the theatre stage’s main speakers and are receiving no audio from the XR device. Innovative XR theatre design concepts around audio could involve making narrative and artistic use of the XR device speakers. This could e.g. be an audio recording of a thought or internal monologue that, instead of being broadcasted from the main stage, plays directly on the XR device speakers, and thus very close to the viewer and in low volume. It could be an audio effect that plays in waves rippling across the audience or plays with a spatialized effect somewhere in the hall, e.g. among the audience seating. Such effects could also make use of the left-right audio channels thus giving a stronger sense of directionality to the audio. 

The audio support could also be used in more practical terms. VOX Reality currently supports provision of subtitles in the user’s language of choice.  In the future, we could extend this functionality to provide a voice over narration using natural-sounded synthetic speech in their language of choice. This option would better accommodate people which prefer listening over reading for any physiological or neurological reason. This feature would require supplying XR devices with noise-cancelling headphones, so that the users may receive a clear audio feed from their XR devices, isolate the theatrical stage main speakers’ audio feed and not produce audio interference to each other.  

In summary, we are in the fortunate position to not only enact a functional merger of the XR technology and the art of theatre domains as we currently know them, but also to envision a redefinition of conventions that have shaped the public’s concept of theatrical experiences for centuries through the capabilities of XR. We would summarize these opening horizons in three broad directions: 1) an amplification of inclusivity by being able to provide customizable individualized access to a collectively shared experience, 2) an amplification and diversification of the audiovisual landscape in the theatrical domain and 3) an invigoration of previously niche or an invention of totally new ways for audience participation in the theatrical happenings.  

Given the sensitive nature of theatrical scripts, what security protocols have been implemented to protect against unauthorized access?

Although our use case does not manage personal or sensitive medical data as in the domains of healthcare or defense, we meticulously examined the security of our system in terms of data traffic and data storage with respect to the intellectual property protection needs of the theatrical content. To cover the needs of the theatre use case, we designed a client-server system with clients operating on the XR devices of the audience and the server operating on a workstation under the assignment of the interdisciplinary facilitation team (developer team and the theatre’s technical team). As context, core reasons for the existence of the client-server system in summary were 1) to centralize the audiovisual input from the scene (microphone and video input) in order to safeguard input media quality, 2) to simultaneously distribute the output to the end-user devices in order to assure synchronicity in the audience and 3) to offset the demanding computational needs to a more powerful device in order to avoid battery and overheating issues on the XR devices.  

In terms of data traffic security, the server and the clients are connected to the same local Wi-Fi network, protected by a WPA2 password, and communicate using a WebSocket protocol for frequent and fast communication. The local Wi-Fi network is for the explicit use of the AR theatre system and accesible only to the aforementioned devices, as a safeguarding measure against network bandwidth fluctuations, which could negatively affect the latency of the system and in turn the user experience during the performance, and as a security measure against data traffic inception. Furthermore, for the exact same reasons, the AI services are also operating locally in the same network and are accessed using RESTful API calls, with the added protection of a secure transport protocol (https).  In summary, the entire traffic is contained in a safe and isolated environment that can only be breached by an unauthorized network access violation. 

In terms of data storage, it was decided that in the release version of the application, no data logs will remain in the XR devices since safeguarding against unauthorized access of the data given the temporary provision of the devices to the public without supervision was not feasible. Any data stored will be in the server device, will hold no personalized information in any form, and will be used exclusively for technical purposes, like system monitoring and performance evaluation. 

Considering the rapid evolution of technology, how is the technical development future-proofed to accommodate emerging advancements, and what strategies are in place for seamless upgrades or integrations with future technologies?

In a rapidly changing technological domain like XR and AI, planning for change is an integral part of design and development. For us, this means asking questions in two directions: 1) what the future fate of the current product can be and 2) what can the product evolve to in the future with minimal effort. Answering these questions is enabled by the fact that we, as XR developers and producers of state-of-the-art XR applications, can create informed scenarios for the foreseeable future.  

One such scenario that is based on financial data and trends is the growth of the XR market, and specifically the AR sector. This is expected to diversify the device range and reduce purchase costs. In turn, this can affect us by enabling the selection of even more well-suited AR glasses for theatres, it can reduce the investment cost for adoption by theatrical establishments, and it can support the popularization of the XR theatre concept in artistic circles. At the same time, theatre-goes, in their role as individual consumers, can be expected to have increasing exposure and familiarity with this technology in general. Therefore, our evaluation for the first question is that we have good reasons to expect that our current product will have increasing potential for adoption. 

On the second question, our strategy is to vigorously uphold proper application design principles with explicit focus on modular, maintainable and expandable design. Operationally, we are adopting a cross-platform development approach to be able to target devices running on different operational systems using the same code base. We are prioritizing open frameworks to ensure compatibility with devices that are compliant with industry standards, thus minimizing intensive proprietary SDK use. In terms of system architecture, by separating the AI from the XR elements, we allow for independent development and evolution for each domain in their own speed and direction. By building the connections with well-established methods that are unlikely to change, like RESTful API calls, we ensure that our product is in the best position to adapt to potentially reworking of entire modules. Furthermore, we adopt a design approach with segmented “levels of XR technology” so as to be able to easily create spin-offs targeting various XR-enabled hardware as they emerge. This does not necessarily imply more powerful devices, but also more popular ones. One current example that we investigate is to single-pick the subtitles provision feature and target affordable 2D AR glasses (also called HUD glasses or smart glasses or wearable monitors) as a means of increasing theatre accessibility.  

Picture of Spyros Polychronopoulos

Spyros Polychronopoulos

Researcher on digital simulation of ancient Greek instruments, and lecturer, teaching music technology and image processing.

Twitter
LinkedIn
IMG_3853

VOXReality’s Immersive Experience at Laval Virtual 2024

by Blog post

Laval Virtual 2024 was an absolute blast, and VOXReality dove right into the heart of the action! Our mission? To scout for awesome SMEs ready to rock the XR world through our open call. But hey, it wasn’t all business—there was plenty of fun to be had!

Picture this: walking through the buzzing exhibition booths, learning a lot from the mind-blowing tech talks and conferences, and connecting with the coolest people in the European XR scene. 

One standout moment? The Women In Immersive Tech (WIIT) gathering. Talk about empowering! We connected with amazing colleagues, exchanged ideas, and celebrated diversity in the industry. It was all about making meaningful connections.

We also got hands-on with jaw-dropping XR content and demos, exploring the edge of innovation. From mind-bending projects to meeting a diverse bunch of XR aficionados, Laval Virtual was the ultimate playground for techies like us!

As we say goodbye to Laval Virtual 2024, VOXReality is pumped up and ready to rock the XR world. Armed with our insights and a bunch of new friends, we’re gearing up to take SMEs on a ride through the immersive tech universe. It’s gonna be one exciting journey!

Picture of Natalia Cardona

Natalia Cardona

Hey there! I'm Natalia and I'm a Corporate communications specialist, I also hold Master's degree in Journalism and Digital Content Innovation by the Autonomous University of Barcelona. I currently work in dissemination, communication, and marketing of technology, innovation, and science for projects funded by the European Commission at F6S.

Twitter
LinkedIn
Blog-1

The Accessibility of XR Events: Voice Commands for All

by Blog post

Inclusivity and accessibility have become increasingly important in the ever-evolving landscape of Extended Reality (XR) events. As these fully virtual and augmented worlds evolve further on, it’s crucial that no one is left behind. Voice-driven interactions are emerging as a powerful tool in making XR events more inclusive and user-friendly, ensuring that individuals of all abilities can fully participate in the immersive experiences offered. In the immersive world of XR events, accessibility plays a crucial role and how voice command interaction will revolutionise the experience, making sure that as many people as possible can partake in the magic of virtual and augmented gatherings.

The XR Event Revolution

XR events encompass a wide range of immersive experiences, from virtual conferences and trade shows to virtual art galleries and interactive social gatherings. These events have the potential to connect people from all corners of the world, transcending physical limitations and providing unique, enriching experiences. But, without proper consideration for accessibility, many individuals may find themselves excluded.

The Significance of Accessibility

Image generated by Midjourney

Accessibility isn’t merely a nice-to-have feature; it’s a fundamental necessity to ensure everyone has equal access to information and knowledge. In the digital age, more and more of this information is shared in online and virtual events. XR events offer the potential to bridge geographical and physical gaps, but to realise this potential, we must ensure that they are accessible to all. This means that all individuals can fully engage and enjoy the experience unimpaired by technical obstacles such as difficult controls, complex interfaces and language barriers. 

The Power of Voice Commands

Voice-driven interactions in XR events are a game-changer for accessibility. Here’s why:

1. Inclusivity: Voice commands allow individuals with limited mobility to navigate and interact with the XR environment easily. 

2. Enhanced Engagement: Voice interactions can make XR events more engaging for all users by presenting a more natural way of interacting in an environment, offering a new level of immersion and interactivity.

3. Language Diversity: XR events often attract participants from diverse linguistic backgrounds. Voice commands can accommodate a wide range of languages, ensuring everyone can participate comfortably.

4. Reducing Cognitive Load: Voice commands allow individuals with limited technical knowledge or XR experience to navigate and interact with the XR environment easily. Users who may have difficulty with complex navigation interfaces, voice commands simplify the interaction process, reducing cognitive load.

Image generated by Midjourney

Conclusion

In the fast-evolving world of XR events, accessibility must be a top priority. Voice-driven interactions are a powerful tool for making these events more inclusive and user-friendly. By implementing voice commands and adhering to best practices, XR event organiSers can ensure that everyone, regardless of their abilities or backgrounds, can fully participate in the immersive experiences that XR events have to offer. It’s not just about embracing technology; it’s about embracing everyone. The future of XR events must be accessible to all!

Picture of Regina Van Tongeren

Regina Van Tongeren

Hi, I'm Regina, Head of Marketing at VRDays Foundation. I help organise Immersive Tech Week in Rotterdam, a festival that brings together diverse voices to celebrate and explore immersive technologies' potential for a better world. I've always loved how films and games create new worlds and realities through stories, and I am fascinated by how immersive technologies are changing storytelling.

With a background in the games industry and teaching marketing, I believe immersive tech will revolutionise brand experiences, and I am curious to see the possibilities they offer for events. As a marketeer at Immersive Tech Week, I am passionate about bringing as many people as possible from all backgrounds and walks of life to Rotterdam so they can discover, experience and think about these new technologies.

Twitter
LinkedIn
2024-01-30 11.15.59-Photoroom

VOXReality Open Call Launches to Revolutionise XR Technology 

by Blog post

XR Innovators the day is officially here! 

We are thrilled to announce the launch of the VOXReality Open Call, an exhilarating opportunity for visionaries and pioneers like you to dive into the world of Extended Reality (XR) and shape its future like never before! 

At VOXReality, we’re passionate about pushing the boundaries of what’s possible in XR technology. That’s why we’re inviting Micro, small and medium-sized enterprises (SMEs) to join us on this exciting journey. Together, we’ll harness the power of advanced AI models and developer tools to create immersive, interactive, and mind-bending XR experiences that will leave audiences in awe. 

But that’s not all – we’re offering equity-free funding of 200K EUR to selected projects, providing the support and resources you need to turn your XR dreams into reality. Imagine the possibilities: from reimagining how we interact with virtual environments to revolutionising training, the potential for innovation is limitless. 

So, what are you waiting for? Dust off your imagination, fire up your creativity, and get ready to embark on the XR adventure of a lifetime. Submit your proposal and join us in reshaping the future of XR technology.  

To learn more about the VOXReality Open Call and how to apply click here. And don’t forget to download the official press release to share the excitement with your network. 

Let’s make magic happen in the world of XR together! 

The VOXReality Team 🚀 

GettyImages-971401274 © gorodenkoff - iStock Getty Images Plus 1920px

VOXReality insights on the European AI Act

by Blog post

Artificial intelligence (AI) is a fast-evolving technology, which can be adopted in an array of fields, such as healthcare, manufacturing, transport and energy, bringing forward multiple economic and societal benefits across Europe. However, while the EU has recognised the centrality of AI in the digital transformation of society, it has also highlighted that “the same elements and techniques that power socio-economic benefits of AI can also bring about new risks or negative consequences for individuals or the society”.

In light of these considerations, the EU has underlined the need for strong regulation to safeguard AI providers and deployers by providing them with straightforward requirements and obligations, without unduly hindering technological innovation and competitiveness in this sector.  

In December 2023, the Council of the European Union and the European Parliament reached a provisional agreement on the proposal for the regulation on harmonised rules on Artificial Intelligence, the so-called Artificial Intelligence Act, which was firstly presented by the European Commission in April 2021. The AI Act is the first legislative proposal of its kind in the world and is, consequently, destined to set a global standard for AI regulation.  

Carme Artigas, Spanish Secretary of state for digitalisation and artificial intelligence, has defined this agreement “a historical achievement and a huge milestone […]”, which joins a wider package of policy measures to promote the development and the circulation of safe and trustworthy AI in conformity with EU existing law on fundamental rights. The regulation also aims at encouraging investment and innovation in AI.  

Application and Implementation

The AI Act will be implemented in the twenty-seven Member States of the European Union. However, there are exceptions to its application, as the regulation clarifies that it will not interfere, in any case, with the Member States’ exclusive competence in national security. Systems used exclusively for military and defence purposes will therefore not be subjected to the new AI legislation. At the same time, the regulation will not apply to AI systems used for the sole purpose of research and innovation, or to people using AI for non-professional reasons.  

Prohibitions and rules for high-risk systems

Among the main objectives of the AI Act, the need to ensure that AI systems are safe and do not cause serious fundamental rights violations was at the heart of the negotiations. To this end, the regulation introduces a classification system that estimates the level of risk an AI technology potentially poses to the health and safety of an individual. 

Consequently, AI systems presenting only limited risk will be subject to very light transparency obligations, while a wide range of high-risk AI systems will be authorised, but subject to a set of requirements and obligations before they gain access to the EU market. However, certain high-risk AI applications will be banned from the EU altogether because they are deemed threatening to citizens’ rights. 

“The provisional agreement bans, for example, cognitive behavioural manipulation, the untargeted scraping of facial images from the internet or CCTV footage, emotion recognition in the workplace and educational institutions, social scoring, biometric categorisation to infer sensitive data, such as sexual orientation or religious beliefs, and some cases of predictive policing for individuals”.  

The Commission’s proposal was revised to include specificities for law enforcement authorities that deploy high-risk AI systems and tools to carry out sensitive operations and activities. However, an effective mechanism has been introduced to ensure the ever-present protection of fundamental rights against potential misuses of these high-risk systems. 

Moreover, when it comes to the use of real-time remote biometric identification systems in publicly accessible spaces, the provisional agreement defines the objectives “where such use is strictly necessary for law enforcement purposes and for which law enforcement authorities should therefore be exceptionally allowed to use such systems”.  

Rules on general-purpose AI models

The proposed legislation takes into account the so-called general purpose AI, that is general-purpose AI (GPAI) systems that can be used for varying purposes. Foundation models are capable of efficiently completing a wide range of performative tasks, such as “generating video, text, images, conversing in lateral language, computing, or generating computer code”. The provisional agreement states that foundation models must comply with specific transparency obligations before they are introduced on the market. For high-impact foundation models, a stricter regime was outlined since these models are more complex and advanced, deploying larger amounts of data and, therefore, increasing the probability of risks.  

System of governance and penalties

In addition, the AI Act provides for the creation of a common European AI governance system, through the establishment of an AI Office within the Commission as part of the administrative structure of the Directorate-General for Communication Networks, Content and Technology. The European AI Office will be the centre of AI expertise across the EU, by promoting the development and use of trustworthy AI. Alongside the AI Office, an AI Board will also be set up as a coordination platform and an advisory body to the Commission. The main purpose of the Board will be to contribute to the implementation of the AI Act in the Member States.  

In addition, the AI Act includes non-compliance penalties with fines ranging from 7.5 million to 35 million euros, which vary depending on the type of infringement and the firm’s size.  

Next steps

On Wednesday 13th March 2024, the European Parliament approved the AI Act with 523 votes in favour, 46 against and 49 abstentions. The regulation is still subject to a final lawyer-linguist check but is expected to be adopted before the end of the legislature, in May. The law also needs to be formally endorsed by the Council.  

The AI Act has generated some criticism since it introduces stricter rules and regulations that might hamper European competitiveness in the sector and, consequently,  lead other countries, like China and the USA, to dominate the industry as these countries have not yet issued a comprehensive regulation on AI.

The challenge for EU legislators has been to deliver a balanced legislation whose aim is to regulate AI but without curbing innovation potential. Experts in the field have raised concerns and doubts relating to certain aspects of the new legislation. The EU will consider the constructive suggestions provided and improve the legislation accordingly.  

Picture of Sabrina Bianchi

Sabrina Bianchi

Sabrina Bianchi is a graduate student in International Relations at the University of Bologna. She is currently working in the R&D Department of Maggioli Group, specialising in dissemination, communication and exploitation of European projects in the fields of Immersive Technologies, Sustainable Energy and Artificial Intelligence.

Twitter
LinkedIn
vonecia-carswell-0aMMMUjiiEQ-unsplash

Gender balance in the XR field: VOXReality’s commitment to women inclusion

by Blog post

Gender equity in Research and Innovation stands as a cornerstone of the European Commission’s  broader commitment to fostering equality across all European Union policies. This dedication to gender equality is particularly pertinent in the context of emerging technologies like Extended Reality (XR). 

As XR continues to reshape how we interact with the digital world, the European Commission’s focus on gender equity ensures that the transformative potential of XR is harnessed through diverse perspectives and inclusive practices. By promoting equal opportunities for women in research, development, and leadership roles within the XR field, the Commission aims to enrich the industry with varied insights, ultimately leading to the creation of XR applications that are more accessible, user-friendly, and reflective of the diverse needs and experiences of all individuals, irrespective of gender. 

In embracing gender equity, the European Commission not only addresses a crucial aspect of social justice but also enhances the innovation and societal impact of XR within the European Union.

The XR field, encompassing virtual reality (VR), augmented reality (AR), and mixed reality (MR), is rapidly evolving, promising ground-breaking innovations and transformative experiences. However, like many tech industries, gender imbalance persists within XR, with women underrepresented in key roles. 

In this article, we shed light on the importance of gender diversity in the XR industry and highlight the exemplary efforts of VOXReality in promoting women’s inclusion.

The XR gender gap: a closer look

The XR industry has witnessed remarkable growth in recent years, yet women remain a minority in the sector. From development and design to leadership positions, the gender gap persists, limiting the diversity of perspectives and ideas. This lack of representation not only hinders innovation but also perpetuates a cycle where women are discouraged from entering or advancing in the field.

Achieving gender balance in the XR field is crucial. First, diverse teams bring a variety of perspectives, leading to more creative and innovative solutions. In a technology landscape that aims to enhance human experiences, diverse voices ensure that XR applications are inclusive and considerate of the needs and preferences of all users.

Furthermore, gender diversity fosters a more inclusive and supportive work environment, encouraging women to pursue careers in XR without fear of isolation or bias. This inclusivity is essential for attracting and retaining talent, as well as for creating a positive and collaborative industry culture.

VOXReality's commitment to women inclusion

VOXReality, an Horizon Europe funded project in the XR domain, stands out for its unwavering commitment to promoting gender balance with a particular focus on women in leadership roles, actively championing the involvement of women in pivotal positions, recognising their invaluable contributions to the XR field. The project consortium recognises the importance of diverse perspectives in shaping the future of XR and actively works towards breaking down barriers for women in the field.

Moreover, VOXReality has taken significant strides in prioritising diversity in recruitment and leadership positions. Women are not merely present but are actively driving decision-making processes, shaping the trajectory of the project from its inception to implementation. The intentional inclusion of women in key roles sends a powerful message about the project’s commitment to breaking down gender barriers within the XR industry.

The impact of VOXReality's efforts

The impact of VOXReality’s efforts is transformative. The project has witnessed a notable increase in the number of women occupying crucial roles in XR-related activities and events. 

By showcasing the leadership and expertise of women within the XR community, VOXReality is dismantling stereotypes and highlighting the immense talent that women bring to the forefront of XR innovation.

The success stories of women who have assumed pivotal roles within the project stand as powerful testaments to the opportunities and support provided by VOXReality. These stories not only inspire the current generation of women in XR but also serve as beacons of encouragement for future female enthusiasts, illustrating that leadership roles are not only attainable but are actively encouraged within the project.

Meet the researchers from the VOXReality Team

Ana Luísa Alves is a Project Manager at F6S, with 8 years of experience, with 5 years of experience in international project management in various EC including framework contracts. 

Carina Pamminger is Head of Research at Holo-Light. Carina brings over ten years of research experience across several disciplines, ranging from the games and transportation industry to augmented and virtual reality sectors. Her main interest areas are in investigating novel ways of leveraging innovative XR technologies to further enable and enhance the Industrial Metaverse.  ​

Irene Viola is ​ tenure track researcher in CWI, in the Distributed and Interactive Systems laboratory. Her focus is on compression and Quality of Experience (QoE) for immersive media.

Jiahuan Pei obtained her PhD at University of Amsterdam and joined DIS group at the CWI (NWO-I) as a researcher. 

She focuses on generative dialogue systems in extended reality (XR) specifically for the VOXReality project. 

Moonisa Ahsan is scientific researcher and post-doctoral fellow in the DIS (Distributed & Interactive Systems) Group at CWI (NWO-I), the Dutch national research institute in the Netherlands.

In her current role within the VOXReality , she contributes her expertise to understanding next-generation applications in Extended Reality (XR). Her primary focus is on unraveling user needs and translating that knowledge into innovative solutions that significantly enhance the user experience across diverse scenarios.

Natalia Cardona is a corporate communications specialist, Master in Journalism and Digital Content Innovation by the Autonomous University of Barcelona. Currently working in dissemination, communication, and marketing of technology, innovation, and science for projects funded by the European Commission at F6S.

Olga Chatzifoti expertise lies in digital and digitised spatiality with a focus on extended reality (XR) environments currently working at Gruppo Maggioli. 

Her academic background includes an MSc in the interdisciplinary program “Geoinformatics” in NTUA, Greece, and an MSc in the interdisciplinary program “Serious Games & Virtual Reality” in the Glasgow School of Art, Scotland. She has since worked as a researcher for the Glasgow School of Art and Heriot-Watt university and collaborated on projects with the Cyprus Institute and Stanford University. 

She is currently a PhD candidate in the Department of Informatics & Telecommunications of the University of Athens with a focus on Human-Computer Interaction in XR environments.

Regina Van Tongeren is marketing and brand manager in the entertainment industry at Immersive Tech Week by VRDays Foundation. She’s had the privilege to construct some amazing stories for consumers and work with wonderful brands loved by many people. Since September 2015 she is also working as a part-time lecturer marketing and branding and enjoys giving students the tools and knowledge they need to become successful professionals and create some amazing stories of their own.

Stavroula Bourou received her Master of Engineering from National Technical University of Athens (NTUA) in 2015. Additionally, she holds a MSc Degree on Geoinformation Science with specialization in Computer Vision from Technical University Berlin in 2019.​

From 2019, she is working as Machine Learning Engineer at Synelixis. She has working experience in conducting research on different aspects of AI, including among others Federated Learning and Privacy Preserving Deep Learning, GAN models, NLP solutions. Additionally, she has hands-on experience on building complete AI workflows, from data collection and model creation until deployment to production. ​

Our reflection

Gender balance in the XR field is not just an ethical imperative but a strategic advantage. The commitment of EU funded projects like VOXReality to women’s inclusion is a beacon of hope for a more diverse and vibrant XR industry. VOXReality’s commitment to women’s leadership and inclusion is not just a commendable ethical stance but a strategic imperative. 

By championing diversity, fostering inclusivity, and actively supporting women in leadership roles, projects like VOXReality are steering the XR industry towards a future that authentically represents the diverse talents and experiences of its participants. As we celebrate the strides made by projects like VOXReality, we continue striving for a more equitable and inclusive future for the XR industry, where the leadership roles of women are not only acknowledged but celebrated.

Picture of Ana Luisa Alves

Ana Luisa Alves

Twitter
LinkedIn
IMG_2740 2

VOXReality Takes Mobile World Congress 2024 and 4YFN by Storm!

by Blog post

Hey tech enthusiasts! Guess who just made a splash at Mobile World Congress 2024 and 4YFN? That’s right, it’s none other than VOXReality! 🚀

Amidst the bustling corridors and vibrant booths, VOXReality brought its A-game to one of the biggest tech events of the year. But what exactly were we up to? Let us spill the beans!

First things first, networking was our middle name! We were on a mission to connect with startups and SMEs, scouting for fresh talent and innovative ideas for our open call. The energy was electric as we exchanged visions and dreams with fellow trailblazers in the tech ecosystem.

But wait, there’s more! We weren’t just mingling with the startup crowd. Oh no, we were on a quest to reach out to every ecosystem actor we could find. From investors to accelerators, we left no stone unturned in spreading the word about VOXReality and our mission to revolutionise the extended reality landscape.

And did we mention our use cases? We showcased the endless possibilities of XR technology. Of course, no tech extravaganza would be complete without meeting with partners. We caught up with old friends from Immersive Tech Week and renewed our alliances, solidifying our commitment to driving innovation forward.

So there you have it, folks! VOXReality came, saw, and conquered Mobile World Congress 2024 and 4YFN. But this is just the beginning of our journey.

Stay tuned as we continue to push the boundaries of XR technology and make waves in the tech world. Until next time, keep dreaming big and embracing the revolution! ✨🌐

Picture of Natalia Cardona

Natalia Cardona

Hey there! I'm Natalia and I'm a Corporate communications specialist, I also hold Master's degree in Journalism and Digital Content Innovation by the Autonomous University of Barcelona. I currently work in dissemination, communication, and marketing of technology, innovation, and science for projects funded by the European Commission at F6S.

Twitter
LinkedIn
nordwood-themes-q8U1YgBaRQk-unsplash (1)

Economic Insights into the Growth of Immersive Technologies Markets

by Blog post

The evolution from the early days of computers to our current era marks a noticeable shift towards more straightforward and engaging interactions with technology. This transformation has been particularly propelled by the rise of augmented and virtual reality (AR/VR) technologies, along with the profound impact of the COVID-19 pandemic, hastening the acceptance of digital realms and the concept of the metaverse.

As of 2021, the global immersive technology market reached a valuation of USD 21.66 billion, and projections indicate a substantial increase to approximately USD 134.18 billion by 2030. This signifies a robust compound annual growth rate (CAGR) of 22.46% from 2022 to 2030. Deloitte Global forecasts a remarkable 50% increase in the virtual reality market, predicting a revenue of US$ 7 billion globally in 2023, up from 2022’s US$ 4.7 billion. 

These figures present a promising outlook for a continually expanding market, resilient even in the face of the recent COVID-19 pandemic. Notably, as a survey conducted in 2020 by McKinsey & Co. highlights, the global pandemic has accelerated the development and has encouraged the adoption of VR and AR technology, which facilitated the completion of activities previously only done in person. Companies have started to use these technologies more intensively because they allow daily working tasks to be performed remotely, while “offering endless possibilities for better learning, productivity and creativity outcomes in every way” .

The role of Europe

Looking at the European XR industry, it is anticipated to reach between €35 billion and €65 billion by 2025, with a gross added value ranging from €20 billion to €40 billion. This growth is expected to directly create employment for up to 860,000 people. The momentum in the VR/AR sector can be attributed to two key factors: the availability of advanced XR technologies, including more comfortable and affordable headsets, and the rising demand for enterprise XR solutions as businesses recognise their potential benefits.

The European Union has been a staunch supporter of digitalisation and XR technology development, evident through funding innovative digital research projects under the Horizon 2020 and Horizon Europe programs. Additionally, the European XR ecosystem thrives with events, initiatives, and associations like EuroXR, uniting national associations, individual members, and companies interested in XR. The EU’s overarching goal is to enhance digital literacy, transforming Europe into a thriving and highly competitive hub for XR activities.

However, recent geopolitical events, particularly the war in Ukraine, have left a significant impact on the immersive technology market. Sanctions and the withdrawal of companies from the Russian market, including <Microsoft’s suspension of HoloLens sales, have disrupted the AR landscape. The effects have rippled into the European market, with Central and Eastern European countries experiencing the most substantial consequences, given their slower and more price-sensitive AR market.

Conclusion

The trajectory of immersive technology, fuelled by advancements in XR technologies and the transformative influence of the COVID-19 pandemic, showcases a robust and continually expanding global market. Projections underscore the sector’s resilience and potential for substantial growth, with the European Union playing a pivotal role in fostering digital innovation and XR development. 

Despite geopolitical challenges impacting the industry, the commitment to digital literacy and strategic support for XR activities position Europe as a competitive hub. As we navigate this dynamic landscape, it becomes evident that immersive technologies are not merely trends but integral components shaping the future of how we interact with and perceive the digital realm.

References

1 https://www.precedenceresearch.com/immersive-technology-market
2 Lee P., Arkenberg C., Stanton B., Cook A., Will VR go from niche to mainstream? It all depends on compelling VR content, in Deloitte’s Technology,
Media and Telecommunications Predictions, 2023, p.71
3 https://www.mckinsey.com/capabilities/strategy-and-corporate-finance/our-insights/how-covid-19-has-pushed-companies-over-the-technology- tipping-point-and-transformed-business-forever#/
4 Globally Cool B.V., The European market potential for VR and AR services, 2021
5 World Economic Forum, Immersive Media Technologies: The Acceleration of Augmented and Virtual Reality in the Wake of COVID-19, 2022, p. 8
6 Ecorys, XR and its potential for Europe, 2021; Vigkos A., Bevacqua D., Turturro L., Kuehl S., The Virtual and Augmented Reality Industrial Coalition,
Ecorys, 2022
7 https://blog-idceurope.com/how-the-russia-ukraine-war-is-impacting-the-human-augmentation-market-in-europe/

Picture of Alberto Casanova

Alberto Casanova

Alberto Casanova is an EU Project Manager in the R&D department of Maggioli Group, one of Italy's foremost companies providing software and digital services for Public Administrations. With more than five years of experience in this role, Alberto is engaged in proposal preparation and project management; he specializes in Business and Exploitation activities, with a specific focus on European Projects. He has successfully led numerous projects in diverse fields, including e-Health, Security, Industry 4.0, Cloud Technologies, and Immersive Technologies. Alberto is currently involved in the project coordination of the VOXReality project, where he takes the lead in overseeing exploitation activities.

Twitter
LinkedIn