The project "AI-Aided-XR: AI Aided eXtended Reality applications" aims to combine artificial intelligence (AI) and extended reality (XR) technologies to explore new ways in which one can help create applications involving the other and vice versa. Specifically, it focuses on the use case of training and testing autonomous driving artificial intelligences on lunar soil by, among other objectives, creating pseudo-random terrain simulations of the lunar surface, which will contribute to research in the underexplored field of space AI.
The initiative, funded by the European Space Agency (ESA), runs from October 2023 to June 2024 and involves the participation of IRTIC through the ARTEC group to faithfully recreate the diversity of lunar soils. The project consortium also includes GMV NSL Ltd from the UK, GMV Soluciones Globales Internet SAU from Spain, GMV Innovating Solutions Sp from Poland and ESA's European Space Operations Centre from Germany.
AI-Aided-XR addresses the lack of data in spatial AI applications, such as images of the lunar surface, through a simulated environment that is procedurally generated by fractal algorithms, that is, without the need for the user to configure each of its details beforehand.
It also addresses this issue through an AI platform based on ESA's AI4OPs technology (artificial intelligence for information technology operations, enabling operational workflows to run smoothly and automatically), trained on the scenario generated to classify planetary terrain. The tool builds on the success of ESA's ViBEKO (Vision-Based Knowledge Extraction using Artificial Intelligence) activity, also from the same organisation, using Google's DeepLabV3+ model.
In fact, the initiative was presented at the 9th European Mission Operations Data System Architecture Workshop (ESAW) that ESA held on June 13th and 14th in Germany. The meeting was part of a series of events held since 2005 inviting contributions covering all aspects of mission operations data systems, focusing on collaboration and common solutions for traditional space missions, but also for newer areas such as Big Data, augmented and virtual reality, machine learning and quantum computing.
The artificial intelligence and extended reality sectors have grown rapidly, with space-related applications taking advantage of recent innovations in sectors such as healthcare, where AI models are used to create three-dimensional models of organs from medical images, which experts can then visualise in virtual environments.
In this line, ESA has driven the integration of AI into its mission operations data systems through the preparation of the Artificial Intelligence for Automation (A2I) Roadmap and has dedicated a prominent place to AI and XR in its Agenda 2025.
The international community's potential uses for AI and XR range from educational devices using AR to interact with satellite imagery for commercial tourism to the ability for mission operators to view automatically labelled and classified images of terrain taken by planetary rovers.
AI also plays a key role in the research opportunities arising from the Mars Sample Return (MSR) mission due to the importance of computer vision techniques for estimating the position of objects.
As with many other AI applications in the space domain, there are limitations to the availability of data to allow for adequate training, so the momentum of initiatives that offer to fuss virtual reality and artificial intelligence to synthetically create datasets in a simulator is noteworthy.
Images:
