The team formed by members of IRTIC, ETSE-UV and the Faculty of Geography and History of the Universitat de València (UV) coordinates the project «ChemiNova: Novel Technologies for On-Site and Remote Collaborative Enriched Monitoring to Detect Structural and Chemical Damages in Cultural Heritage Assets». The Horizon Europe initiative, which will run from February 2024 to January 2027, aims to develop an intelligent computational system that goes beyond current technologies to address the structural and chemical damage to European monuments and heritage collections caused by two specific human-made threats, such as climate change and civil conflicts.
The project takes its name and is inspired by the chemical experimentation game of the 1960s that included the necessary tools to have a home laboratory, such as test tubes and chemicals. Thanks to the augmented reality (AR) and mixed reality tools that ChemiNova will generate, among other tools that will allow the incorporation of a large amount of data, the curators will be able to observe in situ the types of degradation mentioned above.
ChemiNova has a multidisciplinary team with expertise in fields such as art history, architecture, chemical characterization, biology, photogrammetry, topography and 3D reconstruction, among others, and seeks to prioritize the human being as the central axis of technological innovations. The Ramón y Cajal researcher, director of the DINA group at IRTIC and coordinator of the project, Cristina Portalés, highlights the importance of "putting technology at the service of citizens by giving it an applied use that meets real needs" such as "the conservation, analysis and monitoring of European cultural heritage assets".
Portalés also emphasized that the project "allows us to follow the line of collaboration between interdisciplinary groups" adopted in previous initiatives such as Silknow and highlights "the international dimension that the project makes possible to achieve", since it involves entities from several European countries.
The augmented reality tool developed as part of the project, ChemiInspection, specifically developed by IRTIC members, will make a preliminary record of the condition of the part to be studied, the deterioration of which the group of experts will be able to examine. On the one hand, it will consider enriched 3D (e3D) models embedded, making it possible for conservation experts to see pre-calculated defect types superimposed on the real object. On the other hand, it will embed status reports for in situ inspection, while allowing conservation experts to add real-time annotations to the e3D models.
To this end, the consortium members will develop ChemiModel, a software module that will apply 3D reconstruction methods of objects from images to generate these e3D models, which can be enhanced with more data.
Regarding the mixed reality tool, ChemiSensing, which will also be carried out by IRTIC, it will consist of an immersive system that will allow curators to remotely study and assess the state of cultural heritage. For this purpose, elements such as 3D tablets with real-time capture and video cameras with drones will be used to broadcast from the place to be studied.
Seeing, learning and predicting
As a basis for the development of the aforementioned devices, the initiative aims to design a method based on machine learning algorithms, or deep learning, called ChemiAI to predict various types of damage from textured 3D meshes of artifacts, buildings and monuments contained in the e3D models. This artificial intelligence (AI)-based technology will be developed to automatically produce chemical and structural damage characterization, allowing professionals and others an easy-to-use method to infer information based on the reutilization of commonly and widely used devices such as cell phones.
The ChemiSee tool will focus on the negative impact of climate change and conflict zones on cultural heritage, turning them from an abstract fact into a tragic tangible reality. The concept is based on an immersive experience that directly connects people with cultural heritage and the impact of climate change and the effects of war. Through simple on-site or digital interactions, the public will be able to directly explore data and narrations about places and objects.
Four pilot tests have been selected in different European countries because it is possible to collect the necessary data to train the AI model, there is pre-existing data that can be used in ChemiNova to start with the technological implementations as soon as possible, they involve studying a set of artifacts, buildings and monuments, and they allow testing in the corresponding locations the different interactive technologies developed in the project.
One of the pilots will be carried out at the St. Sophia Cathedral in Kyiv, Ukraine, as it is surrounded by an urbanized environment and exposed to anthropogenic and technogenic factors. In times of war, the site requires specific conservation measures.
Another will take place at the Collection of the Università degli Studi di Palermo, in Italy, and features treatises, manuals, encyclopedias and anatomical tables, made of different constituent materials from the mid-sixteenth century to the eighteenth century.
Schönbrunn Palace in Vienna, Austria, where Empress Sissi lived, is another of the chosen monuments. The former imperial summer residence of the Habsburgs is now a museum of Baroque architecture, and the interior of the castle is filled with ornaments, frescos and works of art.
La Nau of the University of Valencia, where another of the pilot tests will take place, is the oldest building of the institution and exemplifies the Valencian neoclassical architecture. In addition to the Cloister, the Aula Magna and the Chapel of Wisdom, the building houses one of the most significant collections of contemporary Valencian artists.
The ChemiNova consortium is coordinated by the University of Valencia. From Italy, the International Centre for the Study of the Preservation and Restoration of Cul-tural Property (ICCROM), the Università degli Studi di Palermo (UNIPA) and Consiglio Nazionale delle Ricerche - Institute of Atmospheric Sciences and Climate (CNR-ISAC), and from Ukraine, the National Conservation Area "St. Sophia of Kyiv" (NCA-SSK).
Also participating in the project are Gottfried Wilhelm Leibniz Universität Hannover (LUH) and ART+COM AG (ARTCO) from Germany; 4D-IT GmbH (4D-IT) and Schloß Schönbrunn Kultur- und Betriebsges.m.b.H. (SKB), Austria; University of Burgundy (UB), France; DIADRASIS Interdisciplinary research on Archaeological & Architectural Conservation (DIADRASIS), Greece; and Universitatea Tehnica Cluj-Napoca (UTC), Romania.
This action funded by the Horizon Europe program is part of the call HORIZON-CL2-2023-HERITAGE-01-01 - Advanced technologies for remote monitoring of heritage monuments and artefacts. The budget amounts to almost 3.5 million euros, of which more than 850,000 euros correspond to the University of Valencia.
Also in:
ChemiNova is funded by the European Union through grant agreement number 101132442. The views expressed are the sole responsibility of the authors and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them.
