Light-perception-based interactive control of an underwater digital twin hand
Fig. 1 | Overview of the all-optical-perception-based underwater digital twin hand interactive control system (LP-UDTHS). (a) Conceptual diagram of the proposed LP-UDTHS system, showing multiple application scenarios where the FBG-SenseGlove performs tas
Fig. 2 | Design, working principles, and performance characterization of the BPSM and FBG-SenseGlove. (a) The BPSM consists of an elephant-trunk-inspired soft arm, a rotational wrist, and a five-finger anthropomorphic dexterous hand. (b) Modular component
Fig. 3 | Underwater digital twin interactive control. (a) System diagram of the underwater digital twin human–robot interactive control. Both the simulated underwater operator and the BPSM wear FBG-SenseGloves, enabling remote digital twin monitoring of b
SHANNON, CLARE, IRELAND, July 24, 2026 /EINPresswire.com/ --
Announcing a new publication from Opto-Electronic Advances; DOI 10.29026/oea.2026.250311.
As global resource exploitation expands further into the ocean, deep-sea exploration and operations are emerging as a new frontier in technological competition. However, the underwater environment is extremely complex, presenting challenges such as high pressure, low temperature, low visibility, and strong disturbances. Traditional rigid robotic hands, limited by poor flexibility and weak environmental adaptability, struggle to perform delicate underwater tasks. In recent years, the emergence of soft robotics has provided a new approach to addressing this challenge. Leveraging their compliant structures, continuous deformation capabilities, and low-damage characteristics toward marine life, soft robotic hands show great potential for performing precision operations in complex underwater environments.
Nevertheless, the practical application of soft robotic hands underwater still faces three core challenges: first, achieving high-precision posture perception under disturbances such as high hydrostatic pressure and turbid currents; second, constructing natural and efficient human–machine interaction (HMI) that allows operators to remotely and intuitively control the soft hand; third, implementing closed-loop, adaptive control strategies in dynamic environments to ensure operational stability and robustness. Traditional sensing methods, such as electrical sensors, are susceptible to electromagnetic interference and require complex encapsulation underwater, while vision systems often fail in turbid water conditions. Therefore, there is an urgent need to develop a new perception and interaction method that is anti-interference, highly sensitive, and suitable for underwater environments.
In response to these challenges, the research team proposed a light-perception-based underwater digital twin hand interactive control system. The system uses fiber Bragg grating (FBG) sensors as the core sensing elements, employing light as the information carrier to construct an integrated closed-loop chain that spans human gesture acquisition, soft hand control, and digital twin visualization. FBG sensors inherently offer advantages such as immunity to electromagnetic interference, high precision, and ease of multiplexing, making them particularly suitable for achieving stable and reliable posture monitoring in complex underwater environments. By integrating the FBG-based data glove (FBG-SenseGlove) with multimodal sensors such as RGB cameras, and optimizing the control strategy with reinforcement learning, the system maintains high-precision gesture recognition and dynamic response even under complex underwater conditions involving turbidity and disturbances. Centered on “light perception,” this research opens a new technological pathway for the development of underwater intelligent equipment.
The research team designed and constructed a complete light-perception-based underwater digital twin hand interactive control system, achieving high-precision, low-latency, and robust human–robot collaborative master–slave control from human gestures to the underwater soft robotic hand, with full-process visual monitoring and bidirectional feedback enabled by digital twin technology.
The team developed a biomimetic pneumatic soft manipulator (BPSM) comprising an elephant-trunk-inspired soft arm, a rotational wrist, and a five-finger anthropomorphic soft hand. Utilizing a rigid–flexible hybrid and multi-layer composite structure, the manipulator possesses multi-degree-of-freedom bending capabilities and can replicate 33 typical human grasp postures, demonstrating excellent compliance and operational flexibility.
To achieve high-precision posture control, the team designed an FBG-SenseGlove with sensors integrated at the interphalangeal joints. It exhibits a strain sensitivity of 1.148 pm/με and a bending sensitivity of 26.667 pm/°, with linearities of 0.9996 and 0.9791, respectively. The FBG sensors demonstrate stable responses in underwater environments. The glove can be worn on both a diver’s hand and the underwater soft hand, enabling bidirectional posture sensing and forming a complete human–robot collaborative closed loop.
In terms of multimodal perception, the team collected FBG signals for six common underwater interaction gestures, constructing a dataset of 2,400 samples. By integrating multi-view RGB images with FBG multimodal data and employing a cross-attention mechanism for data fusion, the system achieved a recognition accuracy of 99.2% under multimodal input, significantly outperforming unimodal approaches. For control strategy, the team fused the FBG-SenseGlove, RGB cameras, and inertial measurement units (IMUs) to build a reinforcement learning optimization framework based on URDF modeling and simulation. Simulation results show that, compared to traditional PID control and model predictive control (MPC), the Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm reduces position error by 5.4 mm and 2.1 mm, respectively, and velocity error by 8.1 mm/frame and 4.6 mm/frame, significantly improving the dynamic response speed and stability of the digital twin hand in complex tasks.
The system realizes quadruple intelligent interactive control encompassing “human, machine, virtual space, and environment.” In practical underwater experiments, an operator wearing the FBG-SenseGlove remotely controlled the BPSM mounted on a remotely operated vehicle (ROV) via gestures to complete tasks such as grasping biological models and transporting targets. The system exhibited fast response times, with the digital twin interface synchronizing postures in real time. Even under complex conditions such as turbid currents and disturbances, the system maintained high-precision tracking and stable control. This study provides a new paradigm of “light perception + digital twin” for underwater intelligent equipment, offering advantages such as high precision, strong robustness, low cost, and ease of deployment. It holds broad prospects for future applications in deep-sea scientific exploration, underwater rescue, marine engineering, and seabed equipment maintenance. Next, the team will focus on integrating integrated multi-modal tactile sensors suitable for underwater intelligent robotic hands, enhancing multi-modal sensor fusion, and exploring brain–computer interface control to further improve operational capability and intelligence in extreme deep-sea environments.
Keywords: fiber Bragg gratings (FBGs), human-machine interaction, underwater soft robotic hand, digital twin, deep learning
# # # # # #
Yang Yu is an Associate Researcher and Master’s Supervisor at the College of Science, National University of Defense Technology. He serves as a Youth Committee Member of the Laser Technology and Application Committee of the Chinese Society for Optical Engineering, a Member of the Marine Remote Sensing and Sensing Technology and Application Group of the Marine Information Network Committee, and a Council Member of the Hunan Optical Society. His research focuses on fiber optic sensing, multi-physics field monitoring in extreme environments, biosensing, and human–machine interaction technologies. He has published over 50 academic papers as first or corresponding author in journals such as Opto-Electronic Advances, IEEE Journal of Lightwave Technology, and Optics Express, filed more than 20 patents (with 15 granted), and received one Hunan Optical Science and Technology Progress Award. His work has been recognized as being among the top 1‰ of highly cited papers globally.
The Micro/Nano Optoelectronics and Intelligent Perception Research Group, affiliated with the College of Science at the National University of Defense Technology, is driven by application demands in complex environment target recognition, interactive control of underwater unmanned platforms, and intelligent monitoring of high-end power machinery. The group focuses on advancing key technologies in micro/nano optoelectronic devices, intelligent algorithms and optical computing, biochemical sensing, human–machine interaction, and extreme environment perception. Over the years, supported by projects from the National Key Research and Development Program, the National Natural Science Foundation of China, the Basic Enhancement Key Project, and the Pre-research Key Project, the group has received multiple national and military science and technology achievement awards. The group has filed and been granted over 50 national invention patents and has published more than 300 papers in high-impact journals such as Opto-Electronic Advances, Laser & Photonics Reviews, Advanced Optical Materials, Nanophotonics, Photonics Research, and Carbon, with numerous research results ranked among the top 1% and 1‰ of ESI.
# # # # # #
Opto-Electronic Advances (OEA) is a high-impact, open access, peer reviewed SCI journal with an impact factor of 22.4 (Journal Citation Reports 2024). OEA has been indexed in SCI, EI, DOAJ, Scopus, CA and ICI databases, and expanded its Editorial Board to 41 members from 17 countries.
# # # # # #
More information: http://www.oejournal.org/oea
Editorial Board: http://www.oejournal.org/oea/editorialboard/list
All issues available in the online archive (http://www.oejournal.org/oea/archive).
Submissions to OEA may be made using ScholarOne (https://mc03.manuscriptcentral.com/oea).
ISSN: 2096-4579
CN: 51-1781/TN
Contact Us: oea@ioe.ac.cn
Twitter: @OptoElectronAdv (https://twitter.com/OptoElectronAdv?lang=en)
WeChat: OE_Journal
# # # # # #
Lu JL, Zhang C, Nong HC et al. Light-perception-based interactive control of an underwater digital twin hand. Opto-Electron Adv 9, 250311 (2026). DOI: 10.29026/oea.2026.250311
Opto-Electronic Advances Editorial Office
Opto-Electronic Advances
+86 591 2286 1222
email us here
Legal Disclaimer:
EIN Presswire provides this news content "as is" without warranty of any kind. We do not accept any responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you have any complaints or copyright issues related to this article, kindly contact the author above.
