Development of a Robotic Teleoperation, Communication, and Visualization System in a Ground Testbed for Lunar and Planetary Exploration

Introduction

We are developing a ground demonstration field for lunar and planetary exploration in Fukushima Prefecture, Japan. This initiative is part of the Moon/Mars Garden (Hakoniwa) Project, led by the University of Aizu (UoA). Specifically, we are constructing a lunar analog environment at the Fukushima Robot Test Field (RTF). This study focuses on building a communication system infrastructure to support robotic teleoperation. Our aim is to address key challenges and contribute to solutions for deploying and operating robotic systems in extreme environments, such as those encountered during lunar and planetary missions.

Methods

To enable remote operation of robots at the Fukushima Robot Test Field (RTF), we have developed a relay-based communication system connecting three primary sites: RTF, the University of Aizu, and remote user locations. This system allows robots deployed at RTF to be operated and tested by users at distant locations. The communication architecture emulates actual mission scenarios—such as multi-robot operations on the Moon relayed through the Lunar Gateway to Earth-based mission control centers.

We have integrated a communication emulator into the relay system to simulate Earth–Moon and other space communication environments. The system allows for the controlled insertion of artificial delays, packet loss, and noise. It is designed to support multi-user and multi-robot operations, enabling each robot to be tested under different communication conditions independently.

In parallel, we are developing a 3D telepresence visualization system to enhance robotic teleoperation. As an initial step, we have reconstructed a 3D model of the test environment at RTF using data acquired from cameras and LiDAR scanners. We evaluated multiple reconstruction methods, including 3D Gaussian Splatting and Neural Radiance Fields (NeRF), and incorporated the results into a virtual reality (VR) interface. While these processes are relatively fast, they do not yet achieve real-time performance. Therefore, we mounted a 360-degree camera on a reference rover, streamed the video to remote users, and transformed it into real-time VR content using Unity.

Experiments

We conducted a series of experiments to evaluate the remote operation of rovers at RTF, specifically around a newly created lunar analog crater (approximately 22 meters in diameter and over 2 meters in depth). A Starlink network was deployed to provide full wireless coverage of the crater area. Multiple rovers equipped with onboard sensors were operated remotely, and the relay-based communication system was validated under various artificial delay conditions, with each delay independently controlled by remote operators. These experiments were also demonstrated during semi-public events, allowing registered attendees to experience remote robotic teleoperation.

Regarding the visualization system, we successfully reconstructed a high-resolution 3D model of the lunar analog crater. Among the various methods tested, the fusion of depth camera and LiDAR data produced the most accurate results, although further quantitative evaluation is ongoing. We also streamed live 360-degree video from the rover to remote users through the communication relay server at UoA. Remote users viewed the scene in real-time using Meta Quest 3 VR headsets, significantly enhancing their situational awareness and operational control.

Discussion and Summary

The experiments conducted at the RTF's lunar analog crater demonstrated the feasibility and effectiveness of our communication and visualization systems for robotic teleoperation. We successfully established a communication infrastructure capable of supporting remote operations under simulated lunar conditions.

As part of future development, we are working to enhance system security through multi-session support and to upgrade the communication emulator to allow packet-level delay control. Additionally, we are exploring communication system designs for actual lunar applications, including the use of Low Power Wide Area (LPWA) technologies to enable reliable communication on the Moon. Such communication may be made feasible through coordinated multi-robot operations, with each robot equipped with a LoRa module.

The 3D visualization system plays a critical role in facilitating accurate and intuitive teleoperation. In actual missions, such systems are expected to support path planning, obstacle avoidance, and spatial awareness. We are currently investigating methods to generate 3D models in quasi-real time, with the ultimate goal of to enable remote operators to control robotic systems as if physically present at the test site.

At RTF, a realistic lunar analog crater with regolith-like sand has been constructed, which serves as the deployment site for this infrastructure. We plan to open the facility to potential users, providing a practical testbed for robotic experiments under realistic lunar analog conditions.

Acknowledgement

This research is supported by a subsidy from Fukushima Prefecture, Japan, and is being conducted by the University of Aizu in collaboration with fellow researchers and Japanese partner companies.