WELD-E, one of the winners of the VOXReality Open Call, is a project that’s pioneering the future of human-robot collaboration in the welding industry. By integrating artificial intelligence (AI) with extended reality (XR), WELD-E has created a safer, more efficient, and more intuitive welding environment. The project’s team recently presented two key papers that detail the system’s advancements.
Published Papers
The WELD-E team has published two papers outlining their work. Each paper details a different aspect of the project’s technology and its implications for the future of manufacturing.
Paper 1: “WELD-E: Enhanced-XR Human-Robot Interactive Collaboration for Welding Operations “
- Lead Authors: Andreas El Saer, George Tsakiris, Leonidas Valavanis, Aristea M. Zafeiropoulou, Konstantinos Loupos, George Argyropoulos, Petros Tsampiras
- Publication: EEITE 2025, 6th International Conference In Electronic Engineering & Information Technology
- Abstract: This paper introduces WEld-e, an end-to-end solution for human-robot collaboration in welding. It leverages AI and XR technologies, including Microsoft HoloLens, to address challenges like a lack of effective guidance and real-time monitoring. The system uses a multimodal interface with voice commands, spatial awareness, and automated decision-making. At its core, it employs deep learning models—such as Automatic Speech Recognition (ASR), Neural Machine Translation (NMT), a domain-specific Welding Large Language Model (WeLLM), and a Visual Language Model—to execute welding commands with high precision and safety. The framework supports multilingual, context-aware interactions and improves operational efficiency and safety in hazardous industrial settings. The paper validates that WEld-e can significantly reduce setup times, training requirements, and error rates, aligning with Industry 5.0 objectives.
- Key Breakthrough: By combining AI and Extended Reality, welders can interact naturally with robots using voice and gestures. This makes welding operations faster and more intuitive by reducing errors and improving safety, all without the need for complex programming.
Paper 2: “Edge AI IoT Immersive Applications”
- Lead Authors: Andreas El Saer, George Tsakiris, Leonidas Valavanis, Aristea M. Zafeiropoulou, Konstantinos Loupos, George Argyropoulos, Petros Tsampiras
- Publication: AIOTI (The European Alliance for Innovation)
- Abstract: This paper presents the WEld-e system as an immersive, AI-driven platform for human-robot collaboration in robotic welding. It integrates Mixed Reality (MR) via Microsoft HoloLens, along with advanced voice and gesture control, and a suite of AI models (ASR, NMT, TTS, and WeLLM). The system enables real-time, multilingual communication between human experts and robotic welders. It also features a digital twin interface for spatially contextual feedback and safety monitoring of a UR10e robotic arm. The paper highlights a key innovation in its knowledge distillation pipeline for computer vision, which refines object detection. Built on a modular architecture using ROS2 and Unity3D, the system supports low-latency interactions, improves precision, and enhances operator awareness, aligning with Industry 5.0 goals.
- Key Breakthrough: How an expert can remotely guide a welding robot using only spoken commands and hand gestures, all while seeing a real-time view through special glasses. This breakthrough simplifies complex tasks, making them faster and safer, and allows experts to supervise operations from a distance.
A Real-World Example
Imagine a welder in a factory wearing HoloLens glasses. Instead of manually programming a robot or using a complex control panel, they simply say, in their native language, “Start welding using template 3.” The system, powered by the AI described in the papers, understands the voice command, translates it if necessary, and directs the robotic arm to begin welding immediately.
But the system’s intelligence doesn’t stop there. Safety is a top priority. If another worker accidentally walks into the designated welding area, the system’s sensors instantly detect the person. It then stops the robotic arm, and a visual and auditory alert pops up in the welder’s glasses, saying something like, “Unauthorized person detected. Operation canceled.” This real-time, intuitive interaction not only streamlines the workflow but also creates a significantly safer working environment.
The WELD-E project is a powerful example of how cutting-edge technology can transform traditional industries.