Extended Reality based Digital Twin of Arc Welding

Abstract

Arc welding is an important permanent joining process widely used across multiple industries such as transportation, infrastructure construction and equipment manufacturing. However, the inherent high-energy physics and toxic fumes generated during the process poses serious health risks for the human operator. In addition, there is an acute shortage of welding professionals across the world leading to significant disruptions in industry. This risk is partly reduced from the use of automation and welding robots which provide stability and accuracy in movement. However, welding robots need to be taught by humans and this requires significant programming efforts and on-site presence of the skilled operator. This thesis proposes a solution to these problems by leveraging the complementary strengths of Digital Twin (DT) and Extended Reality (XR). The advancements in connected and immersive technologies have led to new reality in design and manufacturing. The recent pandemic and hybrid work culture have pushed companies to adopt new ways of digital transformation and global collaboration. This work starts with the development of Virtual (VR) and Mixed Reality (MR) interfaces for facilitating remote welding scenarios. The differences in VR and MR were investigated with an ISO 9241 pointing and selection task and a weld path definition task with actual robot movement. Comprehensive user studies with an array of quantitative measures such as task completion time, qualitative scores such as NASA TLX and SUS, and physiological parameters such as EEG and ocular parameters revealed crucial insights about users’ cognitive workload and preferences. Trajectory analysis with curve similarity measures uncovered important considerations for path planning scenarios in XR environments. To improve the realism of the VR environment in the next phase, the influence of multi-modal elements such as vibrotactile feedback and spatial audio cues was studied with the raycast based pointing and selection task. Results from user studies were used to derive design guidelines for implementation and selection of frequency and duration of these elements. In the final phase, a multi-modal VR based digital twin was developed by leveraging the obtained results. A novel multi-modal collision-based path definition method for welding was developed and evaluated with various mapping functions. A novel and intuitive robot simulation method was also developed to plan and simulate various weld paths for reachability and safety prior to actual execution by the robot. The developed framework was then integrated with a robotic welding station at an industrial plant and was found to be successful in producing welds of acceptable quality. The system was built using commercially available devices and tools and is suitable for industrial deployment at various stages of engineering development and operations. Thus, this user centered immersive digital twin facilitates the creation of a high-performance Human Robot Team (HRT) which utilizes the intelligence and skill of the human operator and the stability and accuracy of the robot.