Publications
Google Scholar
Scopus
Publons
ORCID
PrePrint articles
Jan 2025,
Understanding Flood Detection Models Across Sentinel-1 and Sentinel-2 Modalities and Benchmark Datasets
Enrique Portalés-Julià, Gonzalo Mateo-García and Luis Gómez-Chova
DOI🔗
BibTeX▼
@misc{portales-julia_understanding_2025,
address = {Rochester, NY},
type = {{SSRN} {Scholarly} {Paper}},
title = {Understanding {Flood} {Detection} {Models} {Across} {Sentinel}-1 and {Sentinel}-2 {Modalities} and {Benchmark} {Datasets}},
doi = {10.2139/ssrn.5118486},
language = {en},
urldate = {2025-02-02},
publisher = {Social Science Research Network},
author = {Portalés-Julià, Enrique and Mateo-García, Gonzalo and Gómez-Chova, Luis},
month = jan,
year = {2025},
keywords = {Deep learning, Sentinel-2, multispectral, Sentinel-1, flood detection, multimodal fusion, sar},
}
Abstract▼
In recent years, deep learning has emerged as the dominant approach for flood mapping from remote sensing satellite imagery. While new flood segmentation models are increasingly being proposed, most of these works focus on advancing architectures trained on single datasets. Therefore, these studies overlook the intrinsic limitations and biases of the available training and evaluation data. This often leads to poor generalization and overconfident predictions when these models are used in real-world scenarios. To address this gap, the objective of this work is twofold. First, we train and evaluate flood segmentation models on five publicly available datasets including data from Sentinel-1, Sentinel-2, and both SAR and multispectral modalities. Our findings reveal that models achieving high detection accuracy on a single dataset (intra-dataset validation) do not necessarily generalize well to unseen datasets. In contrast, models trained on more diverse samples from multiple datasets demonstrate greater robustness and generalization ability. Furthermore, we present a dual-stream multimodal architecture that can be independently trained and tested on both single-modality and dual-modality datasets. This enables the integration of all the diversity and richness of available data into a single unified framework. The results emphasize the need for a more comprehensive validation using diverse and well-designed datasets, particularly for multimodal approaches. If not adequately addressed, the shortcomings of current datasets can significantly limit the potential of deep learning-based flood mapping approaches.
Aug 2024,
AI for operational methane emitter monitoring from space
Anna Vaughan, Gonzalo Mateo-Garcia, Itziar Irakulis-Loitxate, Marc Watine, Pablo Fernandez-Poblaciones, Richard E. Turner, James Requeima, Javier Gorroño, Cynthia Randles, Manfredi Caltagirone and Claudio Cifarelli
DOI🔗
BibTeX▼
@misc{vaughan_ai_2024,
title = {{AI} for operational methane emitter monitoring from space},
url = {http://arxiv.org/abs/2408.04745},
doi = {10.48550/arXiv.2408.04745},
urldate = {2024-08-27},
publisher = {arXiv},
author = {Vaughan, Anna and Mateo-Garcia, Gonzalo and Irakulis-Loitxate, Itziar and Watine, Marc and Fernandez-Poblaciones, Pablo and Turner, Richard E. and Requeima, James and Gorroño, Javier and Randles, Cynthia and Caltagirone, Manfredi and Cifarelli, Claudio},
month = aug,
year = {2024},
note = {arXiv:2408.04745 [physics]},
keywords = {Computer Science - Artificial Intelligence, Physics - Atmospheric and Oceanic Physics},
}
Abstract▼
Mitigating methane emissions is the fastest way to stop global warming in the short-term and buy humanity time to decarbonise. Despite the demonstrated ability of remote sensing instruments to detect methane plumes, no system has been available to routinely monitor and act on these events. We present MARS-S2L, an automated AI-driven methane emitter monitoring system for Sentinel-2 and Landsat satellite imagery deployed operationally at the United Nations Environment Programme's International Methane Emissions Observatory. We compile a global dataset of thousands of super-emission events for training and evaluation, demonstrating that MARS-S2L can skillfully monitor emissions in a diverse range of regions globally, providing a 216\% improvement in mean average precision over a current state-of-the-art detection method. Running this system operationally for six months has yielded 457 near-real-time detections in 22 different countries of which 62 have already been used to provide formal notifications to governments and stakeholders.
Journal articles
Oct 2024,
Onboard Cloud Detection and Atmospheric Correction With Efficient Deep Learning Models
Cesar Aybar, Gonzalo Mateo-García, Giacomo Acciarini, Vít Růžička, Gabriele Meoni, Nicolas Longépé and Luis Gómez-Chova
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
DOI🔗
Code
Web page
Trained models 🤖
BibTeX▼
@article{aybar_onboard_2024,
title = {Onboard {Cloud} {Detection} and {Atmospheric} {Correction} {With} {Efficient} {Deep} {Learning} {Models}},
volume = {17},
issn = {2151-1535},
doi = {10.1109/JSTARS.2024.3480520},
urldate = {2024-11-12},
journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
author = {Aybar, Cesar and Mateo-García, Gonzalo and Acciarini, Giacomo and Růžička, Vít and Meoni, Gabriele and Longépé, Nicolas and Gómez-Chova, Luis},
month = oct,
year = {2024},
note = {Conference Name: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
keywords = {Atmospheric correction, Atmospheric modeling, cloud detection (CD), Clouds, CloudSEN12, Data models, deep learning, onboard satellite processing, Optical imaging, Optical reflection, Reflectivity, Remote sensing, Satellites, Sentinel-2 (S-2), Surface treatment, Training},
pages = {19518--19529},
}
Abstract▼
Nano and microsatellites have expanded the acquisition of satellite images with higher spatial, temporal, and spectral resolutions. Nevertheless, downlinking all this data to the ground for processing becomes challenging as the amount of remote sensing data rises. Custom onboard algorithms are designed to make real-time decisions and to prioritize and reduce the amount of data transmitted to the ground. However, these onboard algorithms frequently require cloud-free bottom-of-atmosphere surface reflectance (SR) estimations as inputs to operate. In this context, this article presents the data transformations and autocalibration for Sentinel-2 (S-2) network (DTACSNet), an onboard cloud detection and atmospheric correction processor based on lightweight convolutional neural networks. DTACSNet provides cloud and cloud shadow masks and SR estimates 10× faster than the operational S-2 L2A processor in dedicated space-tested hardware: 7 mins versus 1 h for a 10 980 × 10 980 scene. The DTACSNet cloud masking, based on a lightweight neural network, obtains the highest F2-score (0.81), followed by the state-of-the-art KappaMask (0.74), Fmask (0.72), and Sen2Cor v.2.8 (0.51) algorithms. In addition, validation results on independent datasets show that DTACSNet can efficiently replicate Sen2Cor SR estimates, reporting a competitive accuracy with differences below 2\%.
Aug 2024,
CloudSEN12+: The largest dataset of expert-labeled pixels for cloud and cloud shadow detection in Sentinel-2
Cesar Aybar, Lesly Bautista, David Montero, Julio Contreras, Daryl Ayala, Fernando Prudencio, Jhomira Loja, Luis Ysuhuaylas, Fernando Herrera, Karen Gonzales, Jeanett Valladares, Lucy A. Flores, Evelin Mamani, Maria Quiñonez, Rai Fajardo, Wendy Espinoza, Antonio Limas, Roy Yali, Alejandro Alcántara, Martin Leyva, Raúl Loayza-Muro, Bram Willems, Gonzalo Mateo-García and Luis Gómez-Chova
Data in Brief
DOI🔗
Code
Web page
Dataset 🤗
Trained models 🤗
BibTeX▼
@article{aybar_cloudsen12_2024,
title = {{CloudSEN12}+: {The} largest dataset of expert-labeled pixels for cloud and cloud shadow detection in {Sentinel}-2},
issn = {2352-3409},
shorttitle = {{CloudSEN12}+},
url = {https://www.sciencedirect.com/science/article/pii/S2352340924008163},
doi = {10.1016/j.dib.2024.110852},
urldate = {2024-08-27},
journal = {Data in Brief},
author = {Aybar, Cesar and Bautista, Lesly and Montero, David and Contreras, Julio and Ayala, Daryl and Prudencio, Fernando and Loja, Jhomira and Ysuhuaylas, Luis and Herrera, Fernando and Gonzales, Karen and Valladares, Jeanett and Flores, Lucy A. and Mamani, Evelin and Quiñonez, Maria and Fajardo, Rai and Espinoza, Wendy and Limas, Antonio and Yali, Roy and Alcántara, Alejandro and Leyva, Martin and Loayza-Muro, Raúl and Willems, Bram and Mateo-García, Gonzalo and Gómez-Chova, Luis},
month = aug,
year = {2024},
keywords = {Sentinel-2, Cloud shadow, U-net, Global dataset, IRIS, Thin cloud},
pages = {110852},
}
Abstract▼
Detecting and screening clouds is the first step in most optical remote sensing analyses. Cloud formation is diverse, presenting many shapes, thicknesses, and altitudes. This variety poses a significant challenge to the development of effective cloud detection algorithms, as most datasets lack an unbiased representation. To address this issue, we have built CloudSEN12+, a significant expansion of the CloudSEN12 dataset. This new dataset doubles the expert-labeled annotations, making it the largest cloud and cloud shadow detection dataset for Sentinel-2 imagery up to date. We have carefully reviewed and refined our previous annotations to ensure maximum trustworthiness. We expect CloudSEN12+ will be a valuable resource for the cloud detection research community.
May 2024,
CH4Net: a deep learning model for monitoring methane super-emitters with Sentinel-2 imagery
Anna Vaughan, Gonzalo Mateo-García, Luis Gómez-Chova, Vít Růžička, Luis Guanter and Itziar Irakulis-Loitxate
Atmospheric Measurement Techniques
DOI🔗
Code
Dataset 🤗
BibTeX▼
@article{vaughan_ch4net_2024,
title = {{CH4Net}: a deep learning model for monitoring methane super-emitters with {Sentinel}-2 imagery},
volume = {17},
issn = {1867-1381},
shorttitle = {{CH4Net}},
url = {https://amt.copernicus.org/articles/17/2583/2024/},
doi = {10.5194/amt-17-2583-2024},
language = {English},
number = {9},
urldate = {2024-05-06},
journal = {Atmospheric Measurement Techniques},
author = {Vaughan, Anna and Mateo-García, Gonzalo and Gómez-Chova, Luis and Růžička, Vít and Guanter, Luis and Irakulis-Loitxate, Itziar},
month = may,
year = {2024},
note = {Publisher: Copernicus GmbH},
pages = {2583--2593},
}
Abstract▼
We present a deep learning model, CH4Net, for automated monitoring of methane super-emitters from Sentinel-2 data. When trained on images of 23 methane super-emitter locations from 2017–2020 and evaluated on images from 2021, this model detects 84 \% of methane plumes compared with 24 \% of plumes for a state-of-the-art baseline while maintaining a similar false positive rate. We present an in-depth analysis of CH4Net over the complete dataset and at each individual super-emitter site. In addition to the CH4Net model, we compile and make open source a hand-annotated training dataset consisting of 925 methane plume masks as a machine learning baseline to drive further research in this field.
Nov 2023,
Global flood extent segmentation in optical satellite images
Enrique Portalés-Julià, Gonzalo Mateo-García, Cormac Purcell and Luis Gómez-Chova
Scientific Reports
DOI🔗
Code
Web page
Dataset 🤗
Trained models 🤗
El Confidencial 📰
BibTeX▼
@article{portales-julia_global_2023,
title = {Global flood extent segmentation in optical satellite images},
volume = {13},
copyright = {2023 The Author(s)},
issn = {2045-2322},
url = {https://www.nature.com/articles/s41598-023-47595-7},
doi = {10.1038/s41598-023-47595-7},
language = {en},
number = {1},
urldate = {2023-11-30},
journal = {Scientific Reports},
author = {Portalés-Julià, Enrique and Mateo-García, Gonzalo and Purcell, Cormac and Gómez-Chova, Luis},
month = nov,
year = {2023},
note = {Number: 1
Publisher: Nature Publishing Group},
keywords = {Natural hazards, Scientific data, Computational science, Imaging and sensing},
pages = {20316},
}
Abstract▼
Floods are among the most destructive extreme events that exist, being the main cause of people affected by natural disasters. In the near future, estimated flood intensity and frequency are projected to increase. In this context, automatic and accurate satellite-derived flood maps are key for fast emergency response and damage assessment. However, current approaches for operational flood mapping present limitations due to cloud coverage on acquired satellite images, the accuracy of flood detection, and the generalization of methods across different geographies. In this work, a machine learning framework for operational flood mapping from optical satellite images addressing these problems is presented. It is based on a clouds-aware segmentation model trained in an extended version of the WorldFloods dataset. The model produces accurate and fast water segmentation masks even in areas covered by semitransparent clouds, increasing the coverage for emergency response scenarios. The proposed approach can be applied to both Sentinel-2 and Landsat 8/9 data, which enables a much higher revisit of the damaged region, also key for operational purposes. Detection accuracy and generalization of proposed model is carefully evaluated in a novel global dataset composed of manually labeled flood maps. We provide evidence of better performance than current operational methods based on thresholding spectral indices. Moreover, we demonstrate the applicability of our pipeline to map recent large flood events that occurred in Pakistan, between June and September 2022, and in Australia, between February and April 2022. Finally, the high-resolution (10-30m) flood extent maps are intersected with other high-resolution layers of cropland, building delineations, and population density. Using this workflow, we estimated that approximately 10 million people were affected and 700k buildings and 25,000 km\$\${\textasciicircum}2\$\$of cropland were flooded in 2022 Pakistan floods.
Nov 2023,
Semantic segmentation of methane plumes with hyperspectral machine learning models
Vít Růžička, Gonzalo Mateo-Garcia, Luis Gómez-Chova, Anna Vaughan, Luis Guanter and Andrew Markham
Scientific Reports
DOI🔗
Code
Dataset 🗺️
Trained models 🤗
Oxford news 📰
BibTeX▼
@article{ruzicka_semantic_2023,
title = {Semantic segmentation of methane plumes with hyperspectral machine learning models},
volume = {13},
copyright = {2023 The Author(s)},
issn = {2045-2322},
url = {https://www.nature.com/articles/s41598-023-44918-6},
doi = {10.1038/s41598-023-44918-6},
language = {en},
number = {1},
urldate = {2023-12-01},
journal = {Scientific Reports},
author = {Růžička, Vít and Mateo-Garcia, Gonzalo and Gómez-Chova, Luis and Vaughan, Anna and Guanter, Luis and Markham, Andrew},
month = nov,
year = {2023},
note = {Number: 1
Publisher: Nature Publishing Group},
keywords = {Natural hazards, Computer science, Environmental sciences, Scientific data, Software, Atmospheric science, Climate change},
pages = {19999},
}
Abstract▼
Methane is the second most important greenhouse gas contributor to climate change; at the same time its reduction has been denoted as one of the fastest pathways to preventing temperature growth due to its short atmospheric lifetime. In particular, the mitigation of active point-sources associated with the fossil fuel industry has a strong and cost-effective mitigation potential. Detection of methane plumes in remote sensing data is possible, but the existing approaches exhibit high false positive rates and need manual intervention. Machine learning research in this area is limited due to the lack of large real-world annotated datasets. In this work, we are publicly releasing a machine learning ready dataset with manually refined annotation of methane plumes. We present labelled hyperspectral data from the AVIRIS-NG sensor and provide simulated multispectral WorldView-3 views of the same data to allow for model benchmarking across hyperspectral and multispectral sensors. We propose sensor agnostic machine learning architectures, using classical methane enhancement products as input features. Our HyperSTARCOP model outperforms strong matched filter baseline by over 25\% in F1 score, while reducing its false positive rate per classified tile by over 41.83\%. Additionally, we demonstrate zero-shot generalisation of our trained model on data from the EMIT hyperspectral instrument, despite the differences in the spectral and spatial resolution between the two sensors: in an annotated subset of EMIT images HyperSTARCOP achieves a 40\% gain in F1 score over the baseline.
Jun 2023,
In-orbit demonstration of a re-trainable machine learning payload for processing optical imagery
Gonzalo Mateo-Garcia, Josh Veitch-Michaelis, Cormac Purcell, Nicolas Longepe, Simon Reid, Alice Anlind, Fredrik Bruhn, James Parr and Pierre Philippe Mathieu
Scientific Reports
DOI🔗
Web page
ESA press release 📰
BibTeX▼
@article{mateo-garcia_-orbit_2023,
title = {In-orbit demonstration of a re-trainable machine learning payload for processing optical imagery},
volume = {13},
copyright = {2023 The Author(s)},
issn = {2045-2322},
url = {https://www.nature.com/articles/s41598-023-34436-w},
doi = {10.1038/s41598-023-34436-w},
language = {en},
number = {1},
urldate = {2023-06-27},
journal = {Scientific Reports},
author = {Mateo-Garcia, Gonzalo and Veitch-Michaelis, Josh and Purcell, Cormac and Longepe, Nicolas and Reid, Simon and Anlind, Alice and Bruhn, Fredrik and Parr, James and Mathieu, Pierre Philippe},
month = jun,
year = {2023},
note = {Number: 1
Publisher: Nature Publishing Group},
keywords = {Natural hazards, Hydrology, Engineering},
pages = {10391},
}
Abstract▼
Cognitive cloud computing in space (3CS) describes a new frontier of space innovation powered by Artificial Intelligence, enabling an explosion of new applications in observing our planet and enabling deep space exploration. In this framework, machine learning (ML) payloads—isolated software capable of extracting high level information from onboard sensors—are key to accomplish this vision. In this work we demonstrate, in a satellite deployed in orbit, a ML payload called ‘WorldFloods’ that is able to send compressed flood maps from sensed images. In particular, we perform a set of experiments to: (1) compare different segmentation models on different processing variables critical for onboard deployment, (2) show that we can produce, onboard, vectorised polygons delineating the detected flood water from a full Sentinel-2 tile, (3) retrain the model with few images of the onboard sensor downlinked to Earth and (4) demonstrate that this new model can be uplinked to the satellite and run on new images acquired by its camera. Overall our work demonstrates that ML-based models deployed in orbit can be updated if new information is available, paving the way for agile integration of onboard and onground processing and “on the fly” continuous learning.
Jan 2023,
Multi-spectral multi-image super-resolution of Sentinel-2 with radiometric consistency losses and its effect on building delineation
Muhammed T. Razzak, Gonzalo Mateo-García, Gurvan Lecuyer, Luis Gómez-Chova, Yarin Gal and Freddie Kalaitzis
ISPRS Journal of Photogrammetry and Remote Sensing
DOI🔗
Code
Visualization
Video
BibTeX▼
@article{razzak_multi-spectral_2023,
title = {Multi-spectral multi-image super-resolution of {Sentinel}-2 with radiometric consistency losses and its effect on building delineation},
volume = {195},
issn = {0924-2716},
url = {https://www.sciencedirect.com/science/article/pii/S0924271622002878},
doi = {10.1016/j.isprsjprs.2022.10.019},
language = {en},
urldate = {2022-11-14},
journal = {ISPRS Journal of Photogrammetry and Remote Sensing},
author = {Razzak, Muhammed T. and Mateo-García, Gonzalo and Lecuyer, Gurvan and Gómez-Chova, Luis and Gal, Yarin and Kalaitzis, Freddie},
month = jan,
year = {2023},
keywords = {Building detection, Multi-image super-resolution, Segmentation, Sentinel 2, Super-resolution},
pages = {1--13},
}
Abstract▼
High resolution remote sensing imagery is used in a broad range of tasks, including detection and classification of objects. High-resolution imagery is however expensive to obtain, while lower resolution imagery is often freely available and can be used for a range of social good applications. To that end, we curate a multi-spectral multi-image dataset for super-resolution of satellite images. We use PlanetScope imagery from the SpaceNet-7 challenge as the high resolution reference and multiple Sentinel-2 revisits of the same location as the low-resolution imagery. We present the first results of applying multi-image super-resolution (MISR) to multi-spectral remote sensing imagery. We, additionally, introduce a radiometric-consistency module into the MISR model to preserve the high radiometric resolution and quality of the Sentinel-2 sensor. We show that MISR is superior to single-image super-resolution (SISR) and other baselines on a range of image fidelity metrics. Furthermore, we present the first assessment of the utility of multi-image super-resolution on a semantic and instance segmentation – common remote sensing tasks – showing that utilizing multiple images results in better performance in these downstream tasks, but MISR pre-processing is non-essential.
Dec 2022,
CloudSEN12, a global dataset for semantic understanding of cloud and cloud shadow in Sentinel-2
Cesar Aybar, Luis Ysuhuaylas, Jhomira Loja, Karen Gonzales, Fernando Herrera, Lesly Bautista, Roy Yali, Angie Flores, Lissette Diaz, Nicole Cuenca, Wendy Espinoza, Fernando Prudencio, Valeria Llactayo, David Montero, Martin Sudmanns, Dirk Tiede, Gonzalo Mateo-García and Luis Gómez-Chova
Scientific Data
DOI🔗
Code
Visualization
Web page
Dataset 🗺️
Trained models 🤖
BibTeX▼
@article{aybar_cloudsen12_2022,
title = {{CloudSEN12}, a global dataset for semantic understanding of cloud and cloud shadow in {Sentinel}-2},
volume = {9},
copyright = {2022 The Author(s)},
issn = {2052-4463},
url = {https://www.nature.com/articles/s41597-022-01878-2},
doi = {10.1038/s41597-022-01878-2},
language = {en},
number = {1},
urldate = {2023-01-02},
journal = {Scientific Data},
author = {Aybar, Cesar and Ysuhuaylas, Luis and Loja, Jhomira and Gonzales, Karen and Herrera, Fernando and Bautista, Lesly and Yali, Roy and Flores, Angie and Diaz, Lissette and Cuenca, Nicole and Espinoza, Wendy and Prudencio, Fernando and Llactayo, Valeria and Montero, David and Sudmanns, Martin and Tiede, Dirk and Mateo-García, Gonzalo and Gómez-Chova, Luis},
month = dec,
year = {2022},
note = {Number: 1
Publisher: Nature Publishing Group},
keywords = {Atmospheric dynamics, Geodynamics},
pages = {782},
}
Abstract▼
Accurately characterizing clouds and their shadows is a long-standing problem in the Earth Observation community. Recent works showcase the necessity to improve cloud detection methods for imagery acquired by the Sentinel-2 satellites. However, the lack of consensus and transparency in existing reference datasets hampers the benchmarking of current cloud detection methods. Exploiting the analysis-ready data offered by the Copernicus program, we created CloudSEN12, a new multi-temporal global dataset to foster research in cloud and cloud shadow detection. CloudSEN12 has 49,400 image patches, including (1) Sentinel-2 level-1C and level-2A multi-spectral data, (2) Sentinel-1 synthetic aperture radar data, (3) auxiliary remote sensing products, (4) different hand-crafted annotations to label the presence of thick and thin clouds and cloud shadows, and (5) the results from eight state-of-the-art cloud detection algorithms. At present, CloudSEN12 exceeds all previous efforts in terms of annotation richness, scene variability, geographic distribution, metadata complexity, quality control, and number of samples.
Oct 2022,
RaVÆn: unsupervised change detection of extreme events using ML on-board satellites
Vít Růžička, Anna Vaughan, Daniele De Martini, James Fulton, Valentina Salvatelli, Chris Bridges, Gonzalo Mateo-Garcia and Valentina Zantedeschi
Scientific Reports
DOI🔗
Code
ESA press release 📰
BibTeX▼
@article{ruzicka_ravaen_2022,
title = {{RaVÆn}: unsupervised change detection of extreme events using {ML} on-board satellites},
volume = {12},
copyright = {2022 The Author(s)},
issn = {2045-2322},
shorttitle = {{RaVÆn}},
url = {https://www.nature.com/articles/s41598-022-19437-5},
doi = {10.1038/s41598-022-19437-5},
language = {en},
number = {1},
urldate = {2022-10-09},
journal = {Scientific Reports},
author = {Růžička, Vít and Vaughan, Anna and De Martini, Daniele and Fulton, James and Salvatelli, Valentina and Bridges, Chris and Mateo-Garcia, Gonzalo and Zantedeschi, Valentina},
month = oct,
year = {2022},
note = {Number: 1
Publisher: Nature Publishing Group},
keywords = {Natural hazards, Computer science, Environmental sciences, Scientific data, Software},
pages = {16939},
}
Abstract▼
Applications such as disaster management enormously benefit from rapid availability of satellite observations. Traditionally, data analysis is performed on the ground after being transferred—downlinked—to a ground station. Constraints on the downlink capabilities, both in terms of data volume and timing, therefore heavily affect the response delay of any downstream application. In this paper, we introduce RaVÆn, a lightweight, unsupervised approach for change detection in satellite data based on Variational Auto-Encoders (VAEs), with the specific purpose of on-board deployment. RaVÆn pre-processes the sampled data directly on the satellite and flags changed areas to prioritise for downlink, shortening the response time. We verified the efficacy of our system on a dataset—which we release alongside this publication—composed of time series containing a catastrophic event, demonstrating that RaVÆn outperforms pixel-wise baselines. Finally, we tested our approach on resource-limited hardware for assessing computational and memory limitations, simulating deployment on real hardware.
Jun 2022,
Cloud Mask Intercomparison eXercise (CMIX): An evaluation of cloud masking algorithms for Landsat 8 and Sentinel-2
Sergii Skakun, Jan Wevers, Carsten Brockmann, Georgia Doxani, Matej Aleksandrov, Matej Batič, David Frantz, Ferran Gascon, Luis Gómez-Chova, Olivier Hagolle, Dan López-Puigdollers, Jérôme Louis, Matic Lubej, Gonzalo Mateo-García, Julien Osman, Devis Peressutti, Bringfried Pflug, Jernej Puc, Rudolf Richter, Jean-Claude Roger, Pat Scaramuzza, Eric Vermote, Nejc Vesel, Anže Zupanc and Lojze Žust
Remote Sensing of Environment
DOI🔗
Web page
BibTeX▼
@article{skakun_cloud_2022,
title = {Cloud {Mask} {Intercomparison} {eXercise} ({CMIX}): {An} evaluation of cloud masking algorithms for {Landsat} 8 and {Sentinel}-2},
volume = {274},
issn = {0034-4257},
shorttitle = {Cloud {Mask} {Intercomparison} {eXercise} ({CMIX})},
url = {https://www.sciencedirect.com/science/article/pii/S0034425722001043},
doi = {10.1016/j.rse.2022.112990},
language = {en},
urldate = {2022-03-22},
journal = {Remote Sensing of Environment},
author = {Skakun, Sergii and Wevers, Jan and Brockmann, Carsten and Doxani, Georgia and Aleksandrov, Matej and Batič, Matej and Frantz, David and Gascon, Ferran and Gómez-Chova, Luis and Hagolle, Olivier and López-Puigdollers, Dan and Louis, Jérôme and Lubej, Matic and Mateo-García, Gonzalo and Osman, Julien and Peressutti, Devis and Pflug, Bringfried and Puc, Jernej and Richter, Rudolf and Roger, Jean-Claude and Scaramuzza, Pat and Vermote, Eric and Vesel, Nejc and Zupanc, Anže and Žust, Lojze},
month = jun,
year = {2022},
keywords = {Sentinel-2, CEOS, Cloud, CMIX, Intercomparison, Landsat 8, Validation},
pages = {112990},
}
Abstract▼
Cloud cover is a major limiting factor in exploiting time-series data acquired by optical spaceborne remote sensing sensors. Multiple methods have been developed to address the problem of cloud detection in satellite imagery and a number of cloud masking algorithms have been developed for optical sensors but very few studies have carried out quantitative intercomparison of state-of-the-art methods in this domain. This paper summarizes results of the first Cloud Masking Intercomparison eXercise (CMIX) conducted within the Committee Earth Observation Satellites (CEOS) Working Group on Calibration \& Validation (WGCV). CEOS is the forum for space agency coordination and cooperation on Earth observations, with activities organized under working groups. CMIX, as one such activity, is an international collaborative effort aimed at intercomparing cloud detection algorithms for moderate-spatial resolution (10–30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS) and Sentinel-2 (ESA) missions. Ten algorithms developed by nine teams from fourteen different organizations representing universities, research centers and industry, as well as space agencies (CNES, ESA, DLR, and NASA), are evaluated within the CMIX. Those algorithms vary in their approach and concepts utilized which were based on various spectral properties, spatial and temporal features, as well as machine learning methods. Algorithm outputs are evaluated against existing reference cloud mask datasets. Those datasets vary in sampling methods, geographical distribution, sample unit (points, polygons, full image labels), and generation approaches (experts, machine learning, sky images). Overall, the performance of algorithms varied depending on the reference dataset, which can be attributed to differences in how the reference datasets were produced. The algorithms were in good agreement for thick cloud detection, which were opaque and had lower uncertainties in their identification, in contrast to thin/semi-transparent clouds detection. Not only did CMIX allow identification of strengths and weaknesses of existing algorithms and potential areas of improvements, but also the problems associated with the existing reference datasets. The paper concludes with recommendations on generating new reference datasets, metrics, and an analysis framework to be further exploited and additional input datasets to be considered by future CMIX activities.
Nov 2021,
Towards a novel approach for Sentinel-3 synergistic OLCI/SLSTR cloud and cloud shadow detection based on stereo cloud-top height estimation
Roberto Fernandez-Moran, Luis Gómez-Chova, Luis Alonso, Gonzalo Mateo-García and Dan López-Puigdollers
ISPRS Journal of Photogrammetry and Remote Sensing
DOI🔗
BibTeX▼
@article{fernandez-moran_towards_2021,
title = {Towards a novel approach for {Sentinel}-3 synergistic {OLCI}/{SLSTR} cloud and cloud shadow detection based on stereo cloud-top height estimation},
volume = {181},
issn = {0924-2716},
url = {https://www.sciencedirect.com/science/article/pii/S0924271621002458},
doi = {10.1016/j.isprsjprs.2021.09.013},
language = {en},
urldate = {2021-10-14},
journal = {ISPRS Journal of Photogrammetry and Remote Sensing},
author = {Fernandez-Moran, Roberto and Gómez-Chova, Luis and Alonso, Luis and Mateo-García, Gonzalo and López-Puigdollers, Dan},
month = nov,
year = {2021},
keywords = {Cloud shadow, Cloud detection, Cloud mask, Cloud top height, OLCI, Sentinel-3, SLSTR},
pages = {238--253},
}
Abstract▼
Sentinel-3 is an Earth observation satellite constellation launched by the European Space Agency. Each satellite carries two optical multispectral instruments: the Ocean and Land Colour Instrument (OLCI) and the Sea and Land Surface Temperature Radiometer (SLSTR). OLCI and SLSTR sensors produce images covering the visible and infrared spectrum that can be collocated in order to generate synergistic products. In Earth observation, a particular weakness of optical sensors is their high sensitivity to clouds and their shadows. An incorrect cloud and cloud shadow detection leads to mistakes in both land and ocean retrievals of biophysical parameters. In order to exploit both OLCI and SLSTR capabilities, image co-registration at ground level is needed. However, applying such collocation of the images results in cloud location mismatches due to the different viewing angles of OLCI and SLSTR, which complicates the synergistic cloud detection. This study seeks to provide a solution to correctly obtain the projected clouds based on the estimation of cloud top heights in order to better collocate clouds between sensors and detect their shadows. The study presents a forward and backward method to estimate the real nadir position of a cloud on the satellite image starting from an existing cloud mask, as well as the corresponding cloud projections on the surface depending on the solar and sensor viewing angles. The estimation of cloud top heights is based on differences in the cloud projections from SLSTR nadir and oblique views. Experimental results show that the stereo cloud matching based on maximum cross-correlation between SLSTR nadir and oblique spectra was the most robust method to match SLSTR clouds for both nadir and oblique views as compared to spectral distance and spectral angle minimization. We test the method over several images around the world, leading to higher overall accuracy (OA) as compared to Sentinel-3 official products, both in detecting SLSTR clouds and OLCI cloud shadows (SLSTR nadir OA = 93.6\%, SLSTR oblique OA = 88.7\%, OLCI cloud shadow OA = 93.9\% for the stereo matcher, against 82.2\%, 81.3\% and 90.5\%, respectively, for the official Sentinel-3 products). This study also provides a starting point in the development of a cloud screening approach for the upcoming Fluorescence Explorer (FLEX) satellite mission, expected to fly in tandem with Sentinel-3.
Mar 2021,
Towards global flood mapping onboard low cost satellites with machine learning
Gonzalo Mateo-Garcia, Joshua Veitch-Michaelis, Lewis Smith, Silviu Vlad Oprea, Guy Schumann, Yarin Gal, Atılım Güneş Baydin and Dietmar Backes
Scientific Reports
DOI🔗
Code
Web page
Dataset 🗺️
Trained models 🤖
Video
BibTeX▼
@article{mateo-garcia_towards_2021,
title = {Towards global flood mapping onboard low cost satellites with machine learning},
volume = {11},
copyright = {2021 The Author(s)},
issn = {2045-2322},
url = {https://www.nature.com/articles/s41598-021-86650-z},
doi = {10.1038/s41598-021-86650-z},
language = {en},
number = {1},
urldate = {2021-04-01},
journal = {Scientific Reports},
author = {Mateo-Garcia, Gonzalo and Veitch-Michaelis, Joshua and Smith, Lewis and Oprea, Silviu Vlad and Schumann, Guy and Gal, Yarin and Baydin, Atılım Güneş and Backes, Dietmar},
month = mar,
year = {2021},
note = {Number: 1
Publisher: Nature Publishing Group},
pages = {7249},
}
Abstract▼
Spaceborne Earth observation is a key technology for flood response, offering valuable information to decision makers on the ground. Very large constellations of small, nano satellites— ’CubeSats’ are a promising solution to reduce revisit time in disaster areas from days to hours. However, data transmission to ground receivers is limited by constraints on power and bandwidth of CubeSats. Onboard processing offers a solution to decrease the amount of data to transmit by reducing large sensor images to smaller data products. The ESA’s recent PhiSat-1 mission aims to facilitate the demonstration of this concept, providing the hardware capability to perform onboard processing by including a power-constrained machine learning accelerator and the software to run custom applications. This work demonstrates a flood segmentation algorithm that produces flood masks to be transmitted instead of the raw images, while running efficiently on the accelerator aboard the PhiSat-1. Our models are trained on WorldFloods: a newly compiled dataset of 119 globally verified flooding events from disaster response organizations, which we make available in a common format. We test the system on independent locations, demonstrating that it produces fast and accurate segmentation masks on the hardware accelerator, acting as a proof of concept for this approach.
Jan 2021,
Benchmarking Deep Learning Models for Cloud Detection in Landsat-8 and Sentinel-2 Images
Dan López-Puigdollers, Gonzalo Mateo-García and Luis Gómez-Chova
Remote Sensing
DOI🔗
Code
Trained models 🤖
BibTeX▼
@article{lopez-puigdollers_benchmarking_2021,
title = {Benchmarking {Deep} {Learning} {Models} for {Cloud} {Detection} in {Landsat}-8 and {Sentinel}-2 {Images}},
volume = {13},
copyright = {http://creativecommons.org/licenses/by/3.0/},
url = {https://www.mdpi.com/2072-4292/13/5/992},
doi = {10.3390/rs13050992},
language = {en},
number = {5},
urldate = {2021-04-01},
journal = {Remote Sensing},
author = {López-Puigdollers, Dan and Mateo-García, Gonzalo and Gómez-Chova, Luis},
month = jan,
year = {2021},
note = {Number: 5
Publisher: Multidisciplinary Digital Publishing Institute},
keywords = {deep learning, Sentinel-2, Landsat-8, convolutional neural networks, transfer learning, cloud detection, inter-dataset comparison, multispectral sensors},
pages = {992},
}
Abstract▼
The systematic monitoring of the Earth using optical satellites is limited by the presence of clouds. Accurately detecting these clouds is necessary to exploit satellite image archives in remote sensing applications. Despite many developments, cloud detection remains an unsolved problem with room for improvement, especially over bright surfaces and thin clouds. Recently, advances in cloud masking using deep learning have shown significant boosts in cloud detection accuracy. However, these works are validated in heterogeneous manners, and the comparison with operational threshold-based schemes is not consistent among many of them. In this work, we systematically compare deep learning models trained on Landsat-8 images on different Landsat-8 and Sentinel-2 publicly available datasets. Overall, we show that deep learning models exhibit a high detection accuracy when trained and tested on independent images from the same Landsat-8 dataset (intra-dataset validation), outperforming operational algorithms. However, the performance of deep learning models is similar to operational threshold-based ones when they are tested on different datasets of Landsat-8 images (inter-dataset validation) or datasets from a different sensor with similar radiometric characteristics such as Sentinel-2 (cross-sensor validation). The results suggest that (i) the development of cloud detection methods for new satellites can be based on deep learning models trained on data from similar sensors and (ii) there is a strong dependence of deep learning models on the dataset used for training and testing, which highlights the necessity of standardized datasets and procedures for benchmarking cloud detection models in the future.
Oct 2020,
Cross-Sensor Adversarial Domain Adaptation of Landsat-8 and Proba-V Images for Cloud Detection
G. Mateo-García, V. Laparra, D. López-Puigdollers and L. Gómez-Chova
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
DOI🔗
Code
Visualization
Video
BibTeX▼
@article{mateo-garcia_cross-sensor_2020,
title = {Cross-{Sensor} {Adversarial} {Domain} {Adaptation} of {Landsat}-8 and {Proba}-{V} {Images} for {Cloud} {Detection}},
volume = {14},
issn = {2151-1535},
doi = {10.1109/JSTARS.2020.3031741},
journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
author = {Mateo-García, G. and Laparra, V. and López-Puigdollers, D. and Gómez-Chova, L.},
month = oct,
year = {2020},
note = {Conference Name: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
keywords = {Satellites, Landsat-8, Remote sensing, Cloud computing, Earth, Data models, convolutional neural networks, Adaptation models, Artificial satellites, Proba-V, cloud detection, domain adaptation, Generative adversarial networks},
pages = {747--761},
}
Abstract▼
The number of Earth observation satellites carrying optical sensors with similar characteristics is constantly growing. Despite their similarities and the potential synergies among them, derived satellite products are often developed for each sensor independently. Differences in retrieved radiances lead to significant drops in accuracy, which hampers knowledge and information sharing across sensors. This is particularly harmful for machine learning algorithms, since gathering new ground-truth data to train models for each sensor is costly and requires experienced manpower. In this work, we propose a domain adaptation transformation to reduce the statistical differences between images of two satellite sensors in order to boost the performance of transfer learning models. The proposed methodology is based on the cycle consistent generative adversarial domain adaptation framework that trains the transformation model in an unpaired manner. In particular, Landsat-8 and Proba-V satellites, which present different but compatible spatio-spectral characteristics, are used to illustrate the method. The obtained transformation significantly reduces differences between the image datasets while preserving the spatial and spectral information of adapted images, which is, hence, useful for any general purpose cross-sensor application. In addition, the training of the proposed adversarial domain adaptation model can be modified to improve the performance in a specific remote sensing application, such as cloud detection, by including a dedicated term in the cost function. Results show that, when the proposed transformation is applied, cloud detection models trained in Landsat-8 data increase cloud detection accuracy in Proba-V.
Feb 2020,
Estimating and understanding crop yields with explainable deep learning in the Indian Wheat Belt
Aleksandra Wolanin, Gonzalo Mateo-García, Gustau Camps-Valls, Luis Gómez-Chova, Michele Meroni, Gregory Duveiller, You Liangzhi and Luis Guanter
Environmental Research Letters
DOI🔗
BibTeX▼
@article{wolanin_estimating_2020,
title = {Estimating and understanding crop yields with explainable deep learning in the {Indian} {Wheat} {Belt}},
volume = {15},
issn = {1748-9326},
url = {https://doi.org/10.1088%2F1748-9326%2Fab68ac},
doi = {10.1088/1748-9326/ab68ac},
language = {en},
number = {2},
urldate = {2020-02-12},
journal = {Environmental Research Letters},
author = {Wolanin, Aleksandra and Mateo-García, Gonzalo and Camps-Valls, Gustau and Gómez-Chova, Luis and Meroni, Michele and Duveiller, Gregory and Liangzhi, You and Guanter, Luis},
month = feb,
year = {2020},
pages = {024019},
}
Abstract▼
Forecasting crop yields is becoming increasingly important under the current context in which food security needs to be ensured despite the challenges brought by climate change, an expanding world population accompanied by rising incomes, increasing soil erosion, and decreasing water resources. Temperature, radiation, water availability and other environmental conditions influence crop growth, development, and final grain yield in a complex nonlinear manner. Machine learning (ML) techniques, and deep learning (DL) methods in particular, can account for such nonlinear relations between yield and its covariates. However, they typically lack transparency and interpretability, since the way the predictions are derived is not directly evident. Yet, in the context of yield forecasting, understanding which are the underlying factors behind both a predicted loss or gain is of great relevance. Here, we explore how to benefit from the increased predictive performance of DL methods while maintaining the ability to interpret how the models achieve their results. To do so, we applied a deep neural network to multivariate time series of vegetation and meteorological data to estimate the wheat yield in the Indian Wheat Belt. Then, we visualized and analyzed the features and yield drivers learned by the model with the use of regression activation maps. The DL model outperformed other tested models (ridge regression and random forest) and facilitated the interpretation of variables and processes that lead to yield variability. The learned features were mostly related to the length of the growing season, and temperature and light conditions during this time. For example, our results showed that high yields in 2012 were associated with low temperatures accompanied by sunny conditions during the growing period. The proposed methodology can be used for other crops and regions in order to facilitate application of DL models in agriculture.
Feb 2020,
Transferring deep learning models for cloud detection between Landsat-8 and Proba-V
Gonzalo Mateo-García, Valero Laparra, Dan López-Puigdollers and Luis Gómez-Chova
ISPRS Journal of Photogrammetry and Remote Sensing
DOI🔗
Code
PDF
BibTeX▼
@article{mateo-garcia_transferring_2020,
title = {Transferring deep learning models for cloud detection between {Landsat}-8 and {Proba}-{V}},
volume = {160},
issn = {0924-2716},
url = {https://www.sciencedirect.com/science/article/abs/pii/S0924271619302801},
doi = {10.1016/j.isprsjprs.2019.11.024},
language = {en},
urldate = {2019-12-12},
journal = {ISPRS Journal of Photogrammetry and Remote Sensing},
author = {Mateo-García, Gonzalo and Laparra, Valero and López-Puigdollers, Dan and Gómez-Chova, Luis},
month = feb,
year = {2020},
keywords = {Convolutional neural networks, Deep learning, Transfer learning, Cloud masking, Domain adaptation, Multispectral sensors},
pages = {1--17},
}
Abstract▼
Accurate cloud detection algorithms are mandatory to analyze the large streams of data coming from the different optical Earth observation satellites. Deep learning (DL) based cloud detection schemes provide very accurate cloud detection models. However, training these models for a given sensor requires large datasets of manually labeled samples, which are very costly or even impossible to create when the satellite has not been launched yet. In this work, we present an approach that exploits manually labeled datasets from one satellite to train deep learning models for cloud detection that can be applied (or transferred) to other satellites. We take into account the physical properties of the acquired signals and propose a simple transfer learning approach using Landsat-8 and Proba-V sensors, whose images have different but similar spatial and spectral characteristics. Three types of experiments are conducted to demonstrate that transfer learning can work in both directions: (a) from Landsat-8 to Proba-V, where we show that models trained only with Landsat-8 data produce cloud masks 5 points more accurate than the current operational Proba-V cloud masking method, (b) from Proba-V to Landsat-8, where models that use only Proba-V data for training have an accuracy similar to the operational FMask in the publicly available Biome dataset (87.79–89.77\% vs 88.48\%), and (c) jointly from Proba-V and Landsat-8 to Proba-V, where we demonstrate that using jointly both data sources the accuracy increases 1–10 points when few Proba-V labeled images are available. These results highlight that, taking advantage of existing publicly available cloud masking labeled datasets, we can create accurate deep learning based cloud detection models for new satellites, but without the burden of collecting and labeling a large dataset of images.
May 2019,
Estimating crop primary productivity with Sentinel-2 and Landsat 8 using machine learning methods trained with radiative transfer simulations
Aleksandra Wolanin, Gustau Camps-Valls, Luis Gómez-Chova, Gonzalo Mateo-García, Christiaan van der Tol, Yongguang Zhang and Luis Guanter
Remote Sensing of Environment
DOI🔗
BibTeX▼
@article{wolanin_estimating_2019,
title = {Estimating crop primary productivity with {Sentinel}-2 and {Landsat} 8 using machine learning methods trained with radiative transfer simulations},
volume = {225},
issn = {0034-4257},
url = {http://www.sciencedirect.com/science/article/pii/S0034425719300938},
doi = {10.1016/j.rse.2019.03.002},
urldate = {2019-04-01},
journal = {Remote Sensing of Environment},
author = {Wolanin, Aleksandra and Camps-Valls, Gustau and Gómez-Chova, Luis and Mateo-García, Gonzalo and van der Tol, Christiaan and Zhang, Yongguang and Guanter, Luis},
month = may,
year = {2019},
keywords = {Landsat 8, C3 crops, Gross primary productivity (GPP), Hybrid approach, Machine learning (ML), Neural networks (NN), Radiative transfer modeling (RTM), Sentinel-2 (S2), Soil-canopy-observation of photosynthesis and the energy balance (SCOPE)},
pages = {441--457},
}
Abstract▼
Satellite remote sensing has been widely used in the last decades for agricultural applications, both for assessing vegetation condition and for subsequent yield prediction. Existing remote sensing-based methods to estimate gross primary productivity (GPP), which is an important variable to indicate crop photosynthetic function and stress, typically rely on empirical or semi-empirical approaches, which tend to over-simplify photosynthetic mechanisms. In this work, we take advantage of all parallel developments in mechanistic photosynthesis modeling and satellite data availability for an advanced monitoring of crop productivity. In particular, we combine process-based modeling with the soil-canopy energy balance radiative transfer model (SCOPE) with Sentinel-2 and Landsat 8 optical remote sensing data and machine learning methods in order to estimate crop GPP. With this approach, we by-pass the need for an intermediate step to retrieve the set of vegetation biophysical parameters needed to accurately model photosynthesis, while still accounting for the complex processes of the original physically-based model. Several implementations of the machine learning models are tested and validated using simulated and flux tower-based GPP data. Our final neural network model is able to estimate GPP at the tested flux tower sites with r2 of 0.92 and RMSE of 1.38 gC d−1 m−2, which outperforms empirical models based on vegetation indices. The first test of applicability of this model to Landsat 8 data showed good results (r2 of 0.82 and RMSE of 1.97 gC d−1 m−2), which suggests that our approach can be further applied to other sensors. Modeling and testing is restricted to C3 crops in this study, but can be extended to C4 crops by producing a new training dataset with SCOPE that accounts for the different photosynthetic pathways. Our model successfully estimates GPP across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
Jul 2018,
Multitemporal Cloud Masking in the Google Earth Engine
Gonzalo Mateo-García, Luis Gómez-Chova, Julia Amorós-López, Jordi Muñoz-Marí and Gustau Camps-Valls
Remote Sensing
DOI🔗
Code
Visualization
BibTeX▼
@article{mateo-garcia_multitemporal_2018,
title = {Multitemporal {Cloud} {Masking} in the {Google} {Earth} {Engine}},
volume = {10},
copyright = {http://creativecommons.org/licenses/by/3.0/},
url = {http://www.mdpi.com/2072-4292/10/7/1079},
doi = {10.3390/rs10071079},
language = {en},
number = {7},
urldate = {2018-07-10},
journal = {Remote Sensing},
author = {Mateo-García, Gonzalo and Gómez-Chova, Luis and Amorós-López, Julia and Muñoz-Marí, Jordi and Camps-Valls, Gustau},
month = jul,
year = {2018},
keywords = {Landsat-8, change detection, cloud masking, Google Earth Engine (GEE), image time series, multitemporal analysis},
pages = {1079},
}
Abstract▼
The exploitation of Earth observation satellite images acquired by optical instruments requires an automatic and accurate cloud detection. Multitemporal approaches to cloud detection are usually more powerful than their single scene counterparts since the presence of clouds varies greatly from one acquisition to another whereas surface can be assumed stationary in a broad sense. However, two practical limitations usually hamper their operational use: the access to the complete satellite image archive and the required computational power. This work presents a cloud detection and removal methodology implemented in the Google Earth Engine (GEE) cloud computing platform in order to meet these requirements. The proposed methodology is tested for the Landsat-8 mission over a large collection of manually labeled cloud masks from the Biome dataset. The quantitative results show state-of-the-art performance compared with mono-temporal standard approaches, such as FMask and ACCA algorithms, yielding improvements between 4\–5\% in classification accuracy and 3\–10\% in commission errors. The algorithm implementation within the Google Earth Engine and the generated cloud masks for all test images are released for interested readers.
May 2018,
Machine Learning Regression Approaches for Colored Dissolved Organic Matter (CDOM) Retrieval with S2-MSI and S3-OLCI Simulated Data
Ana Belen Ruescas, Martin Hieronymi, Gonzalo Mateo-Garcia, Sampsa Koponen, Kari Kallio and Gustau Camps-Valls
Remote Sensing
DOI🔗
Code
BibTeX▼
@article{ruescas_machine_2018,
title = {Machine {Learning} {Regression} {Approaches} for {Colored} {Dissolved} {Organic} {Matter} ({CDOM}) {Retrieval} with {S2}-{MSI} and {S3}-{OLCI} {Simulated} {Data}},
volume = {10},
copyright = {http://creativecommons.org/licenses/by/3.0/},
url = {http://www.mdpi.com/2072-4292/10/5/786},
doi = {10.3390/rs10050786},
language = {en},
number = {5},
urldate = {2018-05-20},
journal = {Remote Sensing},
author = {Ruescas, Ana Belen and Hieronymi, Martin and Mateo-Garcia, Gonzalo and Koponen, Sampsa and Kallio, Kari and Camps-Valls, Gustau},
month = may,
year = {2018},
keywords = {remote sensing, Sentinel 2, machine learning, linear regression, CDOM, optically complex waters, Sentinel 3},
pages = {786},
}
Abstract▼
The colored dissolved organic matter (CDOM) variable is the standard measure of humic substance in waters optics. CDOM is optically characterized by its spectral absorption coefficient, a C D O M at at reference wavelength (e.g., \≈ 440 nm). Retrieval of CDOM is traditionally done using bio-optical models. As an alternative, this paper presents a comparison of five machine learning methods applied to Sentinel-2 and Sentinel-3 simulated reflectance ( R r s ) data for the retrieval of CDOM: regularized linear regression (RLR), random forest regression (RFR), kernel ridge regression (KRR), Gaussian process regression (GPR) and support vector machines (SVR). Two different datasets of radiative transfer simulations are used for the development and training of the machine learning regression approaches. Statistics comparison with well-established polynomial regression algorithms shows optimistic results for all models and band combinations, highlighting the good performance of the methods, especially the GPR approach, when all bands are used as input. Application to an atmospheric corrected OLCI image using the reflectance derived form the alternative neural network (Case 2 Regional) is also shown. Python scripts and notebooks are provided to interested users.
Dec 2017,
HyperLabelMe: A Web Platform for Benchmarking Remote-Sensing Image Classifiers
J. Munoz-Mari, E. Izquierdo-Verdiguier, M. Campos-Taberner, A. Perez-Suay, L. Gomez-Chova, G. Mateo-Garcia, A. B. Ruescas, V. Laparra, J. A. Padron, J. Amoros-Lopez and G. Camps-Valls
IEEE Geoscience and Remote Sensing Magazine
DOI🔗
BibTeX▼
@article{munoz-mari_hyperlabelme:_2017,
title = {{HyperLabelMe}: {A} {Web} {Platform} for {Benchmarking} {Remote}-{Sensing} {Image} {Classifiers}},
volume = {5},
issn = {2473-2397},
shorttitle = {{HyperLabelMe}},
doi = {10.1109/MGRS.2017.2762476},
number = {4},
journal = {IEEE Geoscience and Remote Sensing Magazine},
author = {Munoz-Mari, J. and Izquierdo-Verdiguier, E. and Campos-Taberner, M. and Perez-Suay, A. and Gomez-Chova, L. and Mateo-Garcia, G. and Ruescas, A. B. and Laparra, V. and Padron, J. A. and Amoros-Lopez, J. and Camps-Valls, G.},
month = dec,
year = {2017},
keywords = {Hyperspectral imaging, Benchmark testing, Classification algorithms, Image classification, Web and Internet services},
pages = {79--85},
}
Abstract▼
HyperLabelMe is a web platform that allows the automatic benchmarking of remote-sensing image classifiers. To demonstrate this platform's attributes, we collected and harmonized a large data set of labeled multispectral and hyperspectral images with different numbers of classes, dimensionality, noise sources, and levels. The registered user can download training data pairs (spectra and land cover/use labels) and submit the predictions for unseen testing spectra. The system then evaluates the accuracy and robustness of the classifier, and it reports different scores as well as a ranked list of the best methods and users. The system is modular, scalable, and ever-growing in data sets and classifier results.
Jan 2017,
Cloud masking and removal in remote sensing image time series
Luis Gómez-Chova, Julia Amorós-López, Gonzalo Mateo-García, Jordi Muñoz-Marí and Gustau Camps-Valls
Journal of Applied Remote Sensing
DOI🔗
Code
Visualization
PDF
BibTeX▼
@article{gomez-chova_cloud_2017,
title = {Cloud masking and removal in remote sensing image time series},
volume = {11},
copyright = {All rights reserved},
issn = {1931-3195, 1931-3195},
url = {https://www.spiedigitallibrary.org/journals/Journal-of-Applied-Remote-Sensing/volume-11/issue-1/015005/Cloud-masking-and-removal-in-remote-sensing-image-time-series/10.1117/1.JRS.11.015005.short},
doi = {10.1117/1.JRS.11.015005},
number = {1},
urldate = {2017-10-09},
journal = {Journal of Applied Remote Sensing},
author = {Gómez-Chova, Luis and Amorós-López, Julia and Mateo-García, Gonzalo and Muñoz-Marí, Jordi and Camps-Valls, Gustau},
month = jan,
year = {2017},
pages = {015005},
}
Abstract▼
Automatic cloud masking of Earth observation images is one of the first required steps in optical remote sensing data processing since the operational use and product generation from satellite image time series might be hampered by undetected clouds. The high temporal revisit of current and forthcoming missions and the scarcity of labeled data force us to cast cloud screening as an unsupervised change detection problem in the temporal domain. We introduce a cloud screening method based on detecting abrupt changes along the time dimension. The main assumption is that image time series follow smooth variations over land (background) and abrupt changes will be mainly due to the presence of clouds. The method estimates the background surface changes using the information in the time series. In particular, we propose linear and nonlinear least squares regression algorithms that minimize both the prediction and the estimation error simultaneously. Then, significant differences in the image of interest with respect to the estimated background are identified as clouds. The use of kernel methods allows the generalization of the algorithm to account for higher-order (nonlinear) feature relations. After the proposed cloud masking and cloud removal, cloud-free time series at high spatial resolution can be used to obtain a better monitoring of land cover dynamics and to generate more elaborated products. The method is tested in a dataset with 5-day revisit time series from SPOT-4 at high resolution and with Landsat-8 time series. Experimental results show that the proposed method yields more accurate cloud masks when confronted with state-of-the-art approaches typically used in operational settings. In addition, the algorithm has been implemented in the Google Earth Engine platform, which allows us to access the full Landsat-8 catalog and work in a parallel distributed platform to extend its applicability to a global planetary scale.
Conference articles
Mar 2024,
UNEP's Methane Alert and Response System (MARS): current status, new developments and case studies
Itziar Irakulis-Loitxate, Cynthia Randles, Marc Watine-Guiu, Gonzalo Mateo-García, Anna Vaughan, Meghan Demeter, Claudio Cifarelli, Luis Guanter, Joannes D. Maasakkers, Ilse Aben, Tobias A. de Jong, Shubham Sharma, Alexis Groshenry, Quentin Peyle, Antoine Benoit and Manfredi Caltagirone
EGU General Assembly 2024
DOI🔗
Web page
BibTeX▼
@inproceedings{irakulis-loitxate_uneps_2024,
address = {Vienna, Austria},
series = {{EGU24}},
title = {{UNEP}'s {Methane} {Alert} and {Response} {System} ({MARS}): current status, new developments and case studies},
volume = {16048},
shorttitle = {{UNEP}'s {Methane} {Alert} and {Response} {System} ({MARS})},
url = {https://meetingorganizer.copernicus.org/EGU24/EGU24-16048.html},
doi = {10.5194/egusphere-egu24-16048},
During the year-long pilot phase, more than 600 plumes from the energy sector were detected with high-resolution satellites, and more than a hundred were notified. In December 2023, MARS entered the nominal phase with the launch of a data portal including information about the plumes detected and notified by MARS. In its current form, MARS is focused on the detection of strong point sources ({\textasciitilde}{\textgreater}1 ton/h) from the oil and gas production sector, but the system is expected to develop and integrate observations from new satellites as they become available and extend to the notification of smaller sources, also from other sectors such as coal mining, waste, or agriculture.
In this contribution, we will provide a brief overview of the MARS satellite-based plume detection and monitoring system, with the updates made since the launch of the nominal phase. Furthermore, we will describe some examples of real source detection and notification efforts and discuss the next steps planned for MARS in 2024.},
language = {en},
urldate = {2024-05-06},
booktitle = {{EGU} {General} {Assembly} 2024},
publisher = {Copernicus Meetings},
author = {Irakulis-Loitxate, Itziar and Randles, Cynthia and Watine-Guiu, Marc and Mateo-García, Gonzalo and Vaughan, Anna and Demeter, Meghan and Cifarelli, Claudio and Guanter, Luis and Maasakkers, Joannes D. and Aben, Ilse and Jong, Tobias A. de and Sharma, Shubham and Groshenry, Alexis and Peyle, Quentin and Benoit, Antoine and Caltagirone, Manfredi},
month = mar,
year = {2024},
}
Abstract▼
UNEP's Methane Alert and Response System (MARS) is a satellite-based system for the detection and mitigation of methane emissions around the world. As part of the International Methane Emissions Observatory (IMEO), MARS is the first global system connecting satellite methane detection to transparent notification processes intended to trigger mitigation efforts. MARS harnesses state-of-the-art satellite data to identify major emissions, activate its partners to notify relevant stakeholders, and support and track progress toward mitigation. During the year-long pilot phase, more than 600 plumes from the energy sector were detected with high-resolution satellites, and more than a hundred were notified. In December 2023, MARS entered the nominal phase with the launch of a data portal including information about the plumes detected and notified by MARS. In its current form, MARS is focused on the detection of strong point sources ({\textasciitilde}{\textgreater}1 ton/h) from the oil and gas production sector, but the system is expected to develop and integrate observations from new satellites as they become available and extend to the notification of smaller sources, also from other sectors such as coal mining, waste, or agriculture. In this contribution, we will provide a brief overview of the MARS satellite-based plume detection and monitoring system, with the updates made since the launch of the nominal phase. Furthermore, we will describe some examples of real source detection and notification efforts and discuss the next steps planned for MARS in 2024.
Jul 2023,
Lessons Learned From Cloudsen12 Dataset: Identifying Incorrect Annotations in Cloud Semantic Segmentation Datasets
Cesar Aybar, David Montero, Gonzalo Mateo-García and Luis Gómez-Chova
IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
Dataset 🤗
Trained models 🤗
Code
BibTeX▼
@inproceedings{aybar_lessons_2023,
title = {Lessons {Learned} {From} {Cloudsen12} {Dataset}: {Identifying} {Incorrect} {Annotations} in {Cloud} {Semantic} {Segmentation} {Datasets}},
shorttitle = {Lessons {Learned} {From} {Cloudsen12} {Dataset}},
url = {https://ieeexplore.ieee.org/document/10282381},
doi = {10.1109/IGARSS52108.2023.10282381},
urldate = {2023-10-23},
booktitle = {{IGARSS} 2023 - 2023 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Aybar, Cesar and Montero, David and Mateo-García, Gonzalo and Gómez-Chova, Luis},
month = jul,
year = {2023},
note = {ISSN: 2153-7003},
pages = {892--895},
}
Abstract▼
In Earth observation, deep learning models rely heavily on comprehensive datasets for training and evaluation. However, the relevance of data quality is often underestimated, leading to subpar generalization in real-world remote sensing scenarios. This study aims to bridge this gap by proposing a straightforward method to identify critical human annotation errors in semantic segmentation datasets. The approach is based on two indices: trustworthiness and hardness. By implementing these indices, we estimate the extent of human annotation errors in CloudSEN12, a global dataset specifically designed for cloud detection in Sentinel-2 imagery. Considering only the trustworthiness index, our approach identified 1794 potential labelling errors among 10,000 image patches. Out of these, 106 were confirmed as human errors, resulting in a true positive rate of 9.86\%. When this method was applied to other extensive cloud masking datasets, such as KappaSet and Sentinel-2 Cloud Mask Catalogue, it was found that over 44\% of the human labels were inaccurate. These results do not imply the inferior quality of these datasets, instead, they highlight the considerable shift between the annotation protocols, making inter-dataset benchmarking exercises inequitable.
Jul 2023,
Onboard Cloud Detection and Atmospheric Correction with Deep Learning Emulators
Gonzalo Mateo-García, Cesar Aybar, Giacomo Acciarini, Vít Růžička, Gabriele Meoni, Nicolas Longépé and Luis Gómez-Chova
IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
Dataset 🗺️
Trained models 🤗
Code
BibTeX▼
@inproceedings{mateo-garcia_onboard_2023,
title = {Onboard {Cloud} {Detection} and {Atmospheric} {Correction} with {Deep} {Learning} {Emulators}},
url = {https://ieeexplore.ieee.org/document/10282605},
doi = {10.1109/IGARSS52108.2023.10282605},
urldate = {2023-10-23},
booktitle = {{IGARSS} 2023 - 2023 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Mateo-García, Gonzalo and Aybar, Cesar and Acciarini, Giacomo and Růžička, Vít and Meoni, Gabriele and Longépé, Nicolas and Gómez-Chova, Luis},
month = jul,
year = {2023},
note = {ISSN: 2153-7003},
pages = {1875--1878},
}
Abstract▼
This paper introduces DTACSNet, a Convolutional Neural Network (CNN) model specifically developed for efficient onboard atmospheric correction and cloud detection in optical Earth observation satellites. The model is developed with Sentinel-2 data. Through a comparative analysis with the operational Sen2Cor processor, DTACSNet demonstrates a significantly better performance in cloud scene classification (F2 score of 0.89 for DTACSNet compared to 0.51 for Sen2Cor v2.8) and a surface reflectance estimation with average absolute error below 2\% in reflectance units. Moreover, we tested DTACSNet on hardware-constrained systems similar to recent deployed missions and show that DTACSNet is 11 times faster than Sen2Cor with a significantly lower memory consumption footprint. These preliminary results highlight the potential of DTACSNet to provide enhanced efficiency, autonomy, and responsiveness in onboard data processing for Earth observation satellite missions.
Jul 2023,
Fast Model Inference and Training On-Board of Satellites
Vít Růžička, Gonzalo Mateo-García, Chris Bridges, Chris Brunskill, Cormac Purcell, Nicolas Longépé and Andrew Markham
IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
ESA press release 📰
Code
BibTeX▼
@inproceedings{ruzicka_fast_2023,
title = {Fast {Model} {Inference} and {Training} {On}-{Board} of {Satellites}},
url = {https://ieeexplore.ieee.org/document/10282715},
doi = {10.1109/IGARSS52108.2023.10282715},
urldate = {2023-10-23},
booktitle = {{IGARSS} 2023 - 2023 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Růžička, Vít and Mateo-García, Gonzalo and Bridges, Chris and Brunskill, Chris and Purcell, Cormac and Longépé, Nicolas and Markham, Andrew},
month = jul,
year = {2023},
note = {ISSN: 2153-7003},
pages = {2002--2005},
}
Abstract▼
Artificial intelligence onboard satellites has the potential to reduce data transmission requirements, enable real-time decision-making and collaboration within constellations. This study deploys a lightweight foundational model called RaVAEn on D-Orbit’s ION SCV004 satellite. RaVAEn is a variational auto-encoder (VAE) that generates compressed latent vectors from small image tiles, enabling several downstream tasks. In this work we demonstrate the reliable use of RaVAEn onboard a satellite, achieving an encoding time of 0.110s for tiles of a 4.8x4.8 km2 area. In addition, we showcase fast few-shot training onboard a satellite using the latent representation of data. We compare the deployment of the model on the on-board CPU and on the available Myriad vision processing unit (VPU) accelerator. To our knowledge, this work shows for the first time the deployment of a multitask model onboard a CubeSat and the onboard training of a machine learning model.
Dec 2020,
Pix2Streams: Dynamic Hydrology Maps from Satellite-LiDAR Fusion
Dolores Garcia, Gonzalo Mateo-Garcia, Hannes Bernhardt, Ron Hagensieker, Ignacio G. Lopez-Francos, Jonathan Stock, Guy Schumann, Kevin Dobbs and Alfredo Kalaitzis
AI for Earth Scienes Workshop, 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada
URL🔗
Slides
Visualization
Video
BibTeX▼
@inproceedings{garcia_pix2streams_2020,
title = {{Pix2Streams}: {Dynamic} {Hydrology} {Maps} from {Satellite}-{LiDAR} {Fusion}},
shorttitle = {{Pix2Streams}},
url = {http://arxiv.org/abs/2011.07584},
urldate = {2019-10-23},
booktitle = {{AI} for {Earth} {Scienes} {Workshop}, 34th {Conference} on {Neural} {Information} {Processing} {Systems} ({NeurIPS} 2020), {Vancouver}, {Canada}},
author = {Garcia, Dolores and Mateo-Garcia, Gonzalo and Bernhardt, Hannes and Hagensieker, Ron and Lopez-Francos, Ignacio G. and Stock, Jonathan and Schumann, Guy and Dobbs, Kevin and Kalaitzis, Alfredo},
month = dec,
year = {2020},
note = {arXiv: 1910.03019},
keywords = {Computer Science - Machine Learning, Electrical Engineering and Systems Science - Image and Video Processing, Statistics - Machine Learning},
}
Abstract▼
Where are the Earth's streams flowing right now? Inland surface waters expand with floods and contract with droughts, so there is no one map of our streams. Current satellite approaches are limited to monthly observations that map only the widest streams. These are fed by smaller tributaries that make up much of the dendritic surface network but whose flow is unobserved. A complete map of our daily waters can give us an early warning for where droughts are born: the receding tips of the flowing network. Mapping them over years can give us a map of impermanence of our waters, showing where to expect water, and where not to. To that end, we feed the latest high-res sensor data to multiple deep learning models in order to map these flowing networks every day, stacking the times series maps over many years. Specifically, i) we enhance water segmentation to \$50\$ cm/pixel resolution, a 60\${\textbackslash}times\$ improvement over previous state-of-the-art results. Our U-Net trained on 30-40cm WorldView3 images can detect streams as narrow as 1-3m (30-60\${\textbackslash}times\$ over SOTA). Our multi-sensor, multi-res variant, WasserNetz, fuses a multi-day window of 3m PlanetScope imagery with 1m LiDAR data, to detect streams 5-7m wide. Both U-Nets produce a water probability map at the pixel-level. ii) We integrate this water map over a DEM-derived synthetic valley network map to produce a snapshot of flow at the stream level. iii) We apply this pipeline, which we call Pix2Streams, to a 2-year daily PlanetScope time-series of three watersheds in the US to produce the first high-fidelity dynamic map of stream flow frequency. The end result is a new map that, if applied at the national scale, could fundamentally improve how we manage our water resources around the world.
Dec 2019,
Flood Detection On Low Cost Orbital Hardware
Gonzalo Mateo-Garcia, Silviu Oprea, Lewis Smith, Josh Veitch-Michaelis, Guy Schumann, Yarin Gal, Atılım Güneş Baydin and Dietmar Backes
Artificial Intelligence for Humanitarian Assistance and Disaster Response Workshop, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada
URL🔗
Slides
Poster
Code
BibTeX▼
@inproceedings{mateo-garcia_flood_2019,
title = {Flood {Detection} {On} {Low} {Cost} {Orbital} {Hardware}},
url = {http://arxiv.org/abs/1910.03019},
urldate = {2019-10-23},
booktitle = {Artificial {Intelligence} for {Humanitarian} {Assistance} and {Disaster} {Response} {Workshop}, 33rd {Conference} on {Neural} {Information} {Processing} {Systems} ({NeurIPS} 2019), {Vancouver}, {Canada}},
author = {Mateo-Garcia, Gonzalo and Oprea, Silviu and Smith, Lewis and Veitch-Michaelis, Josh and Schumann, Guy and Gal, Yarin and Baydin, Atılım Güneş and Backes, Dietmar},
month = dec,
year = {2019},
note = {arXiv: 1910.03019},
keywords = {Computer Science - Machine Learning, Electrical Engineering and Systems Science - Image and Video Processing, Statistics - Machine Learning},
}
Abstract▼
Satellite imaging is a critical technology for monitoring and responding to natural disasters such as flooding. Despite the capabilities of modern satellites, there is still much to be desired from the perspective of first response organisations like UNICEF. Two main challenges are rapid access to data, and the ability to automatically identify flooded regions in images. We describe a prototypical flood segmentation system, identifying cloud, water and land, that could be deployed on a constellation of small satellites, performing processing on board to reduce downlink bandwidth by 2 orders of magnitude. We target PhiSat-1, part of the FSSCAT mission, which is planned to be launched by the European Space Agency (ESA) near the start of 2020 as a proof of concept for this new technology.
Jul 2019,
Convolutional Long Short-Term Memory Network for Multitemporal Cloud Detection Over Landmarks
Gonzalo Mateo-García, Jose E. Adsuara, Adrián Pérez-Suay and Luis Gómez-Chova
IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
Slides
PDF
BibTeX▼
@inproceedings{mateo-garcia_convolutional_2019,
title = {Convolutional {Long} {Short}-{Term} {Memory} {Network} for {Multitemporal} {Cloud} {Detection} {Over} {Landmarks}},
doi = {10.1109/IGARSS.2019.8897832},
booktitle = {{IGARSS} 2019 - 2019 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Mateo-García, Gonzalo and Adsuara, Jose E. and Pérez-Suay, Adrián and Gómez-Chova, Luis},
month = jul,
year = {2019},
note = {ISSN: 2153-6996},
keywords = {CNN, convolutional neural networks, cloud detection, landmarks, long short-term memory, LSTM, MSG/SEVIRI},
pages = {210--213},
}
Abstract▼
In this work, we propose to exploit both the temporal and spatial correlations in Earth observation satellite images through deep learning methods. In particular, the combination of a U-Net convolutional neural network together with a convolutional long short-term memory (LSTM) layer is proposed. This model is applied for cloud detection on MSG/SEVIRI image time series over selected landmarks. Implementation details are provided and our proposal is compared against a standard SVM and a U-Net without the convolutional LSTM layer but including temporal information too. Experimental results show that this combination of networks exploits both the spatial and temporal dependence and provides state-of-the-art classification results on this dataset.
Jul 2019,
Domain Adaptation of Landsat-8 and Proba-V Data Using Generative Adversarial Networks for Cloud Detection
Gonzalo Mateo-García, Valero Laparra and Luis Gómez-Chova
IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
Slides
BibTeX▼
@inproceedings{mateo-garcia_domain_2019,
title = {Domain {Adaptation} of {Landsat}-8 and {Proba}-{V} {Data} {Using} {Generative} {Adversarial} {Networks} for {Cloud} {Detection}},
doi = {10.1109/IGARSS.2019.8899193},
booktitle = {{IGARSS} 2019 - 2019 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Mateo-García, Gonzalo and Laparra, Valero and Gómez-Chova, Luis},
month = jul,
year = {2019},
note = {ISSN: 2153-6996},
keywords = {Landsat-8, Cloud Detection, Convolutional Neural Networks, Domain Adaptation, Proba-V, Generative Adversarial Networks},
pages = {712--715},
}
Abstract▼
Training machine learning algorithms for new satellites requires collecting new data. This is a critical drawback for most remote sensing applications and specially for cloud detection. A sensible strategy to mitigate this problem is to exploit available data from a similar sensor, which involves transforming this data to resemble the new sensor data. However, even taking into account the technical characteristics of both sensors to transform the images, statistical differences between data distributions still remain. This results in a poor performance of the methods trained on one sensor and applied to the new one. In this this work, we propose to use the generative adversarial networks (GANs) framework to adapt the data from the new satellite. In particular, we use Landsat-8 images, with the corresponding ground truth, to perform cloud detection in Proba-V. Results show that the GANs adaptation significantly improves the detection accuracy.
Jul 2018,
Retrieval of Case 2 Water Quality Parameters with Machine Learning
A. B. Ruescas, G. Mateo-Garcia, G. Camps-Valls and M. Hieronymi
IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
Code
BibTeX▼
@inproceedings{ruescas_retrieval_2018,
title = {Retrieval of {Case} 2 {Water} {Quality} {Parameters} with {Machine} {Learning}},
doi = {10.1109/IGARSS.2018.8518810},
booktitle = {{IGARSS} 2018 - 2018 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Ruescas, A. B. and Mateo-Garcia, G. and Camps-Valls, G. and Hieronymi, M.},
month = jul,
year = {2018},
keywords = {remote sensing, Data models, radiative transfer, learning (artificial intelligence), Machine learning, absorbing waters, Case 2 Absorbing Waters, Case 2 water quality parameters, Case2eXtreme dataset, CDOM concentrations, coloured dissolved organic matter, Estimation, EUMETSAT-ESA, Gaussian process, Gaussian processes, Ground penetrating radar, hydrolight in-water radiative transfer simulations, hydrological techniques, independent simulation dataset, Kernel ridge, Machine Learning Regression, machine learning regression methods, Numerical models, Oceans, OLCI Neural Network Swarm, random forest, regression analysis, regression approaches, regularized linear, Remote Sensing, Sentinel-3 OLCI wavebands, standard OLCI product, Standards, support vector machines, support vector regressors, water quality, Water Quality Parameters},
pages = {124--127},
}
Abstract▼
Water quality parameters are derived applying several machine learning regression methods on the Case2eXtreme dataset (C2X). The used data are based on Hydrolight in-water radiative transfer simulations at Sentinel-3 OLCI wavebands, and the application is done exclusively for absorbing waters with high concentrations of coloured dissolved organic matter (CDOM). The regression approaches are: regularized linear, random forest, Kernel ridge, Gaussian process and support vector regressors. The validation is made with and an independent simulation dataset. A comparison with the OLCI Neural Network Swarm (ONSS) is made as well. The best approached is applied to a sample scene and compared with the standard OLCI product delivered by EUMETSAT/ESA.
Jul 2018,
Optimizing Kernel Ridge Regression for Remote Sensing Problems
G. Mateo-García, V. Laparra and L. Gómez-Chova
IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
Slides
PDF
Code
BibTeX▼
@inproceedings{mateo-garcia_optimizing_2018,
title = {Optimizing {Kernel} {Ridge} {Regression} for {Remote} {Sensing} {Problems}},
doi = {10.1109/IGARSS.2018.8518016},
booktitle = {{IGARSS} 2018 - 2018 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Mateo-García, G. and Laparra, V. and Gómez-Chova, L.},
month = jul,
year = {2018},
keywords = {Atmospheric modeling, Remote sensing, Training, Biological system modeling, IASI, Kernel, kernel methods, kernel ridge regression, KRR, Orbits, Proposals, SGD, temperature retrieval},
pages = {4007--4010},
}
Abstract▼
Kernel methods have been very successful in remote sensing problems because of their ability to deal with high dimensional non-linear data. However, they are computationally expensive to train when a large amount of samples are used. In this context, while the amount of available remote sensing data has constantly increased, the size of training sets in kernel methods is usually restricted to few thousand samples. In this work, we modified the kernel ridge regression (KRR) training procedure to deal with large scale datasets. In addition, the basis functions in the reproducing kernel Hilbert space are defined as parameters to be also optimized during the training process. This extends the number of free parameters from two (in the standard KRR with an RBF kernel) to more than fifty thousand in our experiments. The effectiveness of the proposal is illustrated in the problem of surface temperature estimation from MetOp-IASI hyperspectral infrared sounding data. The data set used contains more than one million samples, but the proposed method could potentially be trained with much more data.
Jul 2018,
Convolutional Neural Networks for Cloud Screening: Transfer Learning from Landsat-8 to Proba-V
G. Mateo-García and L. Gómez-Chova
IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
Slides
Poster
PDF
BibTeX▼
@inproceedings{mateo-garcia_convolutional_2018,
title = {Convolutional {Neural} {Networks} for {Cloud} {Screening}: {Transfer} {Learning} from {Landsat}-8 to {Proba}-{V}},
shorttitle = {Convolutional {Neural} {Networks} for {Cloud} {Screening}},
doi = {10.1109/IGARSS.2018.8517975},
booktitle = {{IGARSS} 2018 - 2018 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Mateo-García, G. and Gómez-Chova, L.},
month = jul,
year = {2018},
keywords = {Landsat-8, Remote sensing, Clouds, Earth, Data models, CNN, Adaptation models, Artificial satellites, Cloud detection, Biological system modeling, Domain Adaptation, Proba-V, Transfer Learning},
pages = {2103--2106},
}
Abstract▼
Cloud detection is a key issue for exploiting the information from Earth observation satellites multispectral sensors. For Proba-V, cloud detection is challenging due to the limited number of spectral bands. Advanced machine learning methods, such as convolutional neural networks (CNN), have shown to work well on this problem provided enough labeled data. However, simultaneous collocated information about the presence of clouds is usually not available or requires a great amount of manual labor. In this work, we propose to learn from the available Landsat −8 cloud masks datasets and transfer this learning to solve the Proba-V cloud detection problem. CNN are trained with Landsat images adapted to resemble Proba-V characteristics and tested on a large set of real Proba-V scenes. Developed models outperform current operational Proba-V cloud detection without being trained with any real Proba-V data. Moreover, cloud detection accuracy can be further increased if the CNN are fine-tuned using a limited amount of Proba-V data.
Sep 2017,
Fair Kernel Learning
Adrián Pérez-Suay, Valero Laparra, Gonzalo Mateo-García, Jordi Muñoz-Marí, Luis Gómez-Chova and Gustau Camps-Valls
Machine Learning and Knowledge Discovery in Databases
DOI🔗
BibTeX▼
@inproceedings{perez-suay_fair_2017,
series = {Lecture {Notes} in {Computer} {Science}},
title = {Fair {Kernel} {Learning}},
isbn = {978-3-319-71248-2 978-3-319-71249-9},
url = {https://link.springer.com/chapter/10.1007/978-3-319-71249-9_21},
doi = {10.1007/978-3-319-71249-9_21},
language = {en},
urldate = {2018-03-14},
booktitle = {Machine {Learning} and {Knowledge} {Discovery} in {Databases}},
publisher = {Springer, Cham},
author = {Pérez-Suay, Adrián and Laparra, Valero and Mateo-García, Gonzalo and Muñoz-Marí, Jordi and Gómez-Chova, Luis and Camps-Valls, Gustau},
month = sep,
year = {2017},
pages = {339--355},
}
Abstract▼
New social and economic activities massively exploit big data and machine learning algorithms to do inference on people’s lives. Applications include automatic curricula evaluation, wage determination, and risk assessment for credits and loans. Recently, many governments and institutions have raised concerns about the lack of fairness, equity and ethics in machine learning to treat these problems. It has been shown that not including sensitive features that bias fairness, such as gender or race, is not enough to mitigate the discrimination when other related features are included. Instead, including fairness in the objective function has been shown to be more efficient.We present novel fair regression and dimensionality reduction methods built on a previously proposed fair classification framework. Both methods rely on using the Hilbert Schmidt independence criterion as the fairness term. Unlike previous approaches, this allows us to simplify the problem and to use multiple sensitive variables simultaneously. Replacing the linear formulation by kernel functions allows the methods to deal with nonlinear problems. For both linear and nonlinear formulations the solution reduces to solving simple matrix inversions or generalized eigenvalue problems. This simplifies the evaluation of the solutions for different trade-off values between the predictive error and fairness terms. We illustrate the usefulness of the proposed methods in toy examples, and evaluate their performance on real world datasets to predict income using gender and/or race discrimination as sensitive variables, and contraceptive method prediction under demographic and socio-economic sensitive descriptors.
Sep 2017,
Advances in statistical cloud screening: the Proba-V case study
Gonzalo Mateo-García, Luis Gómez-Chova, Jordi Muñoz-Marí and Gustau Camps-Valls
Fifth Recent Advances in Quantitative Remote Sensing, Sept 2017
URL🔗
Dataset 🗺️
Poster
PDF
BibTeX▼
@inproceedings{mateo-garcia_advances_2017,
address = {Valencia},
title = {Advances in statistical cloud screening: the {Proba}-{V} case study},
volume = {V},
copyright = {All rights reserved},
isbn = {978-84-9133-201-5},
url = {https://cloud.uv.es/owncloud/index.php/s/4DHonL5QjZtLZWd},
The objective of the cloud masking algorithms is to detect clouds accurately providing a cloud flag per pixel. We first approach the problem from the classical machine learning perspective based on feature extraction plus supervised classification with neural networks (NN). Using this approach we significantly improve the cloud detection accuracy compared to the operational Proba-V algorithm. Then we approach the problem using deep learning methods based on convolutional neural networks (CNN). Experimental results show that CNN are a promising alternative for solving cloud masking problems.},
booktitle = {Fifth {Recent} {Advances} in {Quantitative} {Remote} {Sensing}, {Sept} 2017},
author = {Mateo-García, Gonzalo and Gómez-Chova, Luis and Muñoz-Marí, Jordi and Camps-Valls, Gustau},
month = sep,
year = {2017},
keywords = {Sensors, Satellites, Aerosols, artificial satellites, aerosols, AD 2011 10 28, ADL, Aerosol EDR, Aerosol Environmental Data Record Algorithm, Algorithm Development Library, Cloud Mask algorithms, geophysical equipment, Joint Polar Satellite System satellite, JPSS, JPSS VIIRS aerosol EDR algorithms, NOAA Interface Data Processing Segment, Ocean temperature, radiometers, Radiometry, Satellite broadcasting, scanning radiometer, Sea surface, upstream-downstream effects, VIIRS, VIIRS Sensor Data Record, Visible Infrared Imaging Radiometer Suite},
pages = {391--395},
}
Abstract▼
Accurate and automatic detection of clouds in optical Earth observation satellite scenes is a key issue for a wide range of remote sensing applications. With no accurate cloud masking, undetected clouds are one of the most significant sources of error in both sea and land biophysical parameter retrieval. We present recent advances in statistical cloud detection for Proba-V imagery. The objective of the cloud masking algorithms is to detect clouds accurately providing a cloud flag per pixel. We first approach the problem from the classical machine learning perspective based on feature extraction plus supervised classification with neural networks (NN). Using this approach we significantly improve the cloud detection accuracy compared to the operational Proba-V algorithm. Then we approach the problem using deep learning methods based on convolutional neural networks (CNN). Experimental results show that CNN are a promising alternative for solving cloud masking problems.
Jul 2017,
Convolutional neural networks for multispectral image cloud masking
G. Mateo-García, L. Gómez-Chova and G. Camps-Valls
IGARSS 2017 - 2017 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
Slides
BibTeX▼
@inproceedings{mateo-garcia_convolutional_2017,
title = {Convolutional neural networks for multispectral image cloud masking},
url = {https://ieeexplore.ieee.org/document/8127438},
doi = {10.1109/IGARSS.2017.8127438},
booktitle = {{IGARSS} 2017 - 2017 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Mateo-García, G. and Gómez-Chova, L. and Camps-Valls, G.},
month = jul,
year = {2017},
keywords = {remote sensing, Convolutional neural networks, deep learning, Instruments, Remote sensing, Clouds, Training, convolutional neural networks, image classification, learning (artificial intelligence), feature extraction, Feature extraction, geophysical image processing, neural nets, Proba-V, cloud masking, art methods, cloud detection, cloud masking problems, custom feature extraction, different CNN architectures, end-to-end learning, image classification tasks, multispectral image cloud masking, Proba-V multispectral images, remote sensing problems, V},
pages = {2255--2258},
}
Abstract▼
Convolutional neural networks (CNN) have proven to be state of the art methods for many image classification tasks and their use is rapidly increasing in remote sensing problems. One of their major strengths is that, when enough data is available, CNN perform an end-to-end learning without the need of custom feature extraction methods. In this work, we study the use of different CNN architectures for cloud masking of Proba-V multispectral images. We compare such methods with the more classical machine learning approach based on feature extraction plus supervised classification. Experimental results suggest that CNN are a promising alternative for solving cloud masking problems.
Jul 2017,
Cloud detection on the Google Earth engine platform
G. Mateo-García, J. Muñoz-Marí and L. Gómez-Chova
IGARSS 2017 - 2017 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
Poster
Code
BibTeX▼
@inproceedings{mateo-garcia_cloud_2017,
title = {Cloud detection on the {Google} {Earth} engine platform},
doi = {10.1109/IGARSS.2017.8127359},
booktitle = {{IGARSS} 2017 - 2017 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Mateo-García, G. and Muñoz-Marí, J. and Gómez-Chova, L.},
month = jul,
year = {2017},
keywords = {remote sensing, Landsat-8, Remote sensing, Clouds, Earth, Google, Artificial satellites, change detection, clouds, geophysical image processing, cloud masking, Google Earth Engine (GEE), multitemporal analysis, time series, Time series analysis, satellite images, cloud detection algorithm, Engines, GEE platform capabilities, google Earth engine platform, Google Earth Engine platform, high resolution Earth observation satellites, long time series, technical challenges, temporal dimension},
pages = {1942--1945},
}
Abstract▼
The vast amount of data acquired by current high resolution Earth observation satellites implies some technical challenges to be faced. Google Earth Engine (GEE) platform provides a framework for the development of algorithms and products built over this data in an easy and scalable manner. In this paper, we take advantage of the GEE platform capabilities to exploit the wealth of information in the temporal dimension by processing a long time series of satellite images. A cloud detection algorithm for Landsat-8, which uses previous images of the same location to detect clouds, is implemented and tested on the GEE platform.
Jul 2017,
Cloud detection machine learning algorithms for PROBA-V
L. Gómez-Chova, G. Mateo-García, J. Muñoz-Marí and G. Camps-Valls
IGARSS 2017 - 2017 IEEE International Geoscience and Remote Sensing Symposium
DOI🔗
Dataset 🗺️
BibTeX▼
@inproceedings{gomez-chova_cloud_2017-1,
title = {Cloud detection machine learning algorithms for {PROBA}-{V}},
doi = {10.1109/IGARSS.2017.8127437},
booktitle = {{IGARSS} 2017 - 2017 {IEEE} {International} {Geoscience} and {Remote} {Sensing} {Symposium}},
author = {Gómez-Chova, L. and Mateo-García, G. and Muñoz-Marí, J. and Camps-Valls, G.},
month = jul,
year = {2017},
keywords = {remote sensing, Clouds, Training, Support vector machines, ML, learning (artificial intelligence), Feature extraction, Machine learning algorithms, clouds, geophysical image processing, remote sensing applications, Proba-V, cloud masking, land cover, cloud detection, Brightness, Snow, statistical analysis, automatic cloud detection, cloud detection machine learning algorithms, land cover biophysical parameter retrieval, PROBA-V images, PROBA-V products, satellite scenes, sea, statistical machine learning techniques},
pages = {2251--2254},
}
Abstract▼
This paper presents the development and implementation of a cloud detection algorithm for Proba-V. Accurate and automatic detection of clouds in satellite scenes is a key issue for a wide range of remote sensing applications. With no accurate cloud masking, undetected clouds are one of the most significant sources of error in both sea and land cover biophysical parameter retrieval. The objective of the algorithms presented in this paper is to detect clouds accurately providing a cloud flag per pixel. For this purpose, the method exploits the information of Proba-V using statistical machine learning techniques to identify the clouds present in Proba-V products. The effectiveness of the proposed method is successfully illustrated using a large number of real Proba-V images.
Jun 2017,
Proba-V cloud detection Round Robin: Validation results and recommendations
R. Q. Iannone, F. Niro, P. Goryl, S. Dransfeld, B. Hoersch, K. Stelzer, G. Kirches, M. Paperin, C. Brockmann, L. Gómez-Chova, G. Mateo-García, R. Preusker, J. Fischer, U. Amato, C. Serio, U. Gangkofner, B. Berthelot, M. D. Iordache, L. Bertels, E. Wolters, W. Dierckx, I. Benhadj and E. Swinnen
2017 9th International Workshop on the Analysis of Multitemporal Remote Sensing Images (MultiTemp)
DOI🔗
BibTeX▼
@inproceedings{iannone_proba-v_2017,
title = {Proba-{V} cloud detection {Round} {Robin}: {Validation} results and recommendations},
copyright = {All rights reserved},
shorttitle = {Proba-{V} cloud detection {Round} {Robin}},
doi = {10.1109/Multi-Temp.2017.8035219},
Clouds detection is a critical issue for satellite optical remote sensing, since potential errors in cloud masking directly translates into significant uncertainty in the retrieved downstream geophysical products. Cloud detection is particularly challenging for Proba-V due to the presence of a limited number of spectral bands and the lack of thermal infrared bands. The main objective of the project was the inter-comparison of several cloud detection algorithms for Proba-V over a wide range of surface types and environmental conditions. Proba-V Level 2a products have been distributed to six different algorithm providers representing companies and research institutes in several European countries. The considered cloud detection approaches are based on different strategies: Neural Network, Discriminant Analysis, Multi-spectral and Multi-textural Thresholding, Self-Organizing Feature Maps, Dynamic Thresholding, and physically-based retrieval of Cloud Optical Thickness. The results from all algorithms were analysed and compared against a reference dataset, consisting of a large number (more than fifty thousands) of visually classified pixels.
The quality assessment was performed according to a uniform methodology and the results provide clear indication on the potential best-suited approach for next Proba-V cloud detection algorithm.},
booktitle = {2017 9th {International} {Workshop} on the {Analysis} of {Multitemporal} {Remote} {Sensing} {Images} ({MultiTemp})},
author = {Iannone, R. Q. and Niro, F. and Goryl, P. and Dransfeld, S. and Hoersch, B. and Stelzer, K. and Kirches, G. and Paperin, M. and Brockmann, C. and Gómez-Chova, L. and Mateo-García, G. and Preusker, R. and Fischer, J. and Amato, U. and Serio, C. and Gangkofner, U. and Berthelot, B. and Iordache, M. D. and Bertels, L. and Wolters, E. and Dierckx, W. and Benhadj, I. and Swinnen, E.},
month = jun,
year = {2017},
keywords = {remote sensing, Reflectivity, Satellites, Clouds, MODIS, clouds, atmospheric techniques, Proba-V, cloud masking, neural network, Cloud Detection Algorithm, cloud detection approaches, cloud optical thickness, Detection algorithms, discriminant analysis, downstream geophysical products, environmental conditions, European countries, Land surface, multispectral thresholding, multitextural thresholding, Proba-V cloud detection algorithm, Proba-V cloud detection Round Robin, Proba-V level 2a products, Round robin, Round Robin, satellite optical remote sensing, self-organizing feature maps, spectral bands, surface types, thermal infrared bands},
pages = {1--8},
}
Abstract▼
This paper discusses results from 12 months of a Round Robin exercise aimed at the inter-comparison of different cloud detection algorithms for Proba-V. Clouds detection is a critical issue for satellite optical remote sensing, since potential errors in cloud masking directly translates into significant uncertainty in the retrieved downstream geophysical products. Cloud detection is particularly challenging for Proba-V due to the presence of a limited number of spectral bands and the lack of thermal infrared bands. The main objective of the project was the inter-comparison of several cloud detection algorithms for Proba-V over a wide range of surface types and environmental conditions. Proba-V Level 2a products have been distributed to six different algorithm providers representing companies and research institutes in several European countries. The considered cloud detection approaches are based on different strategies: Neural Network, Discriminant Analysis, Multi-spectral and Multi-textural Thresholding, Self-Organizing Feature Maps, Dynamic Thresholding, and physically-based retrieval of Cloud Optical Thickness. The results from all algorithms were analysed and compared against a reference dataset, consisting of a large number (more than fifty thousands) of visually classified pixels. The quality assessment was performed according to a uniform methodology and the results provide clear indication on the potential best-suited approach for next Proba-V cloud detection algorithm.
Book chapters and Thesis
Jan 2025,
Understanding Flood Detection Models Across Sentinel-1 and Sentinel-2 Modalities and Benchmark Datasets
Enrique Portalés-Julià, Gonzalo Mateo-García and Luis Gómez-Chova
DOI🔗
BibTeX▼
@misc{portales-julia_understanding_2025,
address = {Rochester, NY},
type = {{SSRN} {Scholarly} {Paper}},
title = {Understanding {Flood} {Detection} {Models} {Across} {Sentinel}-1 and {Sentinel}-2 {Modalities} and {Benchmark} {Datasets}},
doi = {10.2139/ssrn.5118486},
language = {en},
urldate = {2025-02-02},
publisher = {Social Science Research Network},
author = {Portalés-Julià, Enrique and Mateo-García, Gonzalo and Gómez-Chova, Luis},
month = jan,
year = {2025},
keywords = {Deep learning, Sentinel-2, multispectral, Sentinel-1, flood detection, multimodal fusion, sar},
}
Abstract▼
In recent years, deep learning has emerged as the dominant approach for flood mapping from remote sensing satellite imagery. While new flood segmentation models are increasingly being proposed, most of these works focus on advancing architectures trained on single datasets. Therefore, these studies overlook the intrinsic limitations and biases of the available training and evaluation data. This often leads to poor generalization and overconfident predictions when these models are used in real-world scenarios. To address this gap, the objective of this work is twofold. First, we train and evaluate flood segmentation models on five publicly available datasets including data from Sentinel-1, Sentinel-2, and both SAR and multispectral modalities. Our findings reveal that models achieving high detection accuracy on a single dataset (intra-dataset validation) do not necessarily generalize well to unseen datasets. In contrast, models trained on more diverse samples from multiple datasets demonstrate greater robustness and generalization ability. Furthermore, we present a dual-stream multimodal architecture that can be independently trained and tested on both single-modality and dual-modality datasets. This enables the integration of all the diversity and richness of available data into a single unified framework. The results emphasize the need for a more comprehensive validation using diverse and well-designed datasets, particularly for multimodal approaches. If not adequately addressed, the shortcomings of current datasets can significantly limit the potential of deep learning-based flood mapping approaches.
Aug 2024,
AI for operational methane emitter monitoring from space
Anna Vaughan, Gonzalo Mateo-Garcia, Itziar Irakulis-Loitxate, Marc Watine, Pablo Fernandez-Poblaciones, Richard E. Turner, James Requeima, Javier Gorroño, Cynthia Randles, Manfredi Caltagirone and Claudio Cifarelli
DOI🔗
BibTeX▼
@misc{vaughan_ai_2024,
title = {{AI} for operational methane emitter monitoring from space},
url = {http://arxiv.org/abs/2408.04745},
doi = {10.48550/arXiv.2408.04745},
urldate = {2024-08-27},
publisher = {arXiv},
author = {Vaughan, Anna and Mateo-Garcia, Gonzalo and Irakulis-Loitxate, Itziar and Watine, Marc and Fernandez-Poblaciones, Pablo and Turner, Richard E. and Requeima, James and Gorroño, Javier and Randles, Cynthia and Caltagirone, Manfredi and Cifarelli, Claudio},
month = aug,
year = {2024},
note = {arXiv:2408.04745 [physics]},
keywords = {Computer Science - Artificial Intelligence, Physics - Atmospheric and Oceanic Physics},
}
Abstract▼
Mitigating methane emissions is the fastest way to stop global warming in the short-term and buy humanity time to decarbonise. Despite the demonstrated ability of remote sensing instruments to detect methane plumes, no system has been available to routinely monitor and act on these events. We present MARS-S2L, an automated AI-driven methane emitter monitoring system for Sentinel-2 and Landsat satellite imagery deployed operationally at the United Nations Environment Programme's International Methane Emissions Observatory. We compile a global dataset of thousands of super-emission events for training and evaluation, demonstrating that MARS-S2L can skillfully monitor emissions in a diverse range of regions globally, providing a 216\% improvement in mean average precision over a current state-of-the-art detection method. Running this system operationally for six months has yielded 457 near-real-time detections in 22 different countries of which 62 have already been used to provide formal notifications to governments and stakeholders.
Dec 2022,
PhD thesis: Transfer learning of Deep Learning Models for Cloud Detection in Optical Satellite Images
Gonzalo Mateo-García
PhD Thesis
URL🔗
BibTeX▼
@phdthesis{mateo-garcia_phd_2022,
type = {{PhD} thesis},
title = {{PhD} thesis: {Transfer} learning of {Deep} {Learning} {Models} for {Cloud} {Detection} in {Optical} {Satellite} {Images}},
url = {https://roderic.uv.es/items/4402c3d6-7533-4f1a-9555-6dd7c4c882cf},
Henceforth, the overreaching goal of this Thesis is to develop accurate cloud detection models that exploit the different properties of the satellite images and to develop methodologies to transfer those models across different sensors. The four contributions of this Thesis are stepping stones in that direction. In the first contribution, "Multitemporal cloud masking in the Google Earth Engine", we implemented a lightweight multitemporal cloud detection model that runs on the Google Earth Engine platform and which outperforms the operational models for Landsat-8. The second contribution, "Transferring deep learning models for Cloud Detection between Landsat-8 and Proba-V", is a case-study of transferring a deep learning based cloud detection algorithm from Landsat-8 (30m resolution, 12 spectral bands and very good radiometric quality) to Proba-V which has a lower 333m resolution, only four bands and less good radiometric quality. The third paper "Cross sensor adversarial domain adaptation of Landsat-8 and Proba-V images for cloud detection" propose a learning-based domain adaptation transformation to Proba-V images to resemble those taken by Landsat-8 with the objective of transferring products designed on Landsat-8 to Proba-V. Finally, the fourth contribution, "Towards global flood mapping onboard low cost satellites with machine learning", tackles simultaneously cloud and flood water detection with a single deep learning model; in this case the model is implemented so that it could run onboard of a CubeSat (Phi-Sat) with an AI accelerator chip; the model is trained on Sentinel-2 images and we demonstrate how to transfer this model to the Phi-Sat camera. We trained this model in a newly compiled dataset of more than 100 verified flood events called WorldFloods. This model was launched on June 2021 onboard the Wild Ride D-Orbit mission and we are testing now its performance in space.},
school = {Universitat de Valencia},
author = {Mateo-García, Gonzalo},
month = dec,
year = {2022},
}
Abstract▼
Earth observation through remote sensing sensors in orbiting satellites provide us with a great capacity to monitor our planet at high spatial and temporal resolutions. Nevertheless, to process all this ever-growing amount of data, we need to develop fast and accurate models adapted to the specific characteristics of the data of each sensor. For optical sensors, detecting the clouds in the image is an unavoidable first step to most of the land and ocean applications. Although detecting bright and opaque clouds is relatively easy, automatically identifying thin semi-transparent clouds or differentiating clouds from snow or bright surfaces is much more challenging. In addition, in the current scenario where the number of sensors in space is constantly growing, developing methodologies to transfer models across different satellite data is a pressing need. Henceforth, the overreaching goal of this Thesis is to develop accurate cloud detection models that exploit the different properties of the satellite images and to develop methodologies to transfer those models across different sensors. The four contributions of this Thesis are stepping stones in that direction. In the first contribution, "Multitemporal cloud masking in the Google Earth Engine", we implemented a lightweight multitemporal cloud detection model that runs on the Google Earth Engine platform and which outperforms the operational models for Landsat-8. The second contribution, "Transferring deep learning models for Cloud Detection between Landsat-8 and Proba-V", is a case-study of transferring a deep learning based cloud detection algorithm from Landsat-8 (30m resolution, 12 spectral bands and very good radiometric quality) to Proba-V which has a lower 333m resolution, only four bands and less good radiometric quality. The third paper "Cross sensor adversarial domain adaptation of Landsat-8 and Proba-V images for cloud detection" propose a learning-based domain adaptation transformation to Proba-V images to resemble those taken by Landsat-8 with the objective of transferring products designed on Landsat-8 to Proba-V. Finally, the fourth contribution, "Towards global flood mapping onboard low cost satellites with machine learning", tackles simultaneously cloud and flood water detection with a single deep learning model; in this case the model is implemented so that it could run onboard of a CubeSat (Phi-Sat) with an AI accelerator chip; the model is trained on Sentinel-2 images and we demonstrate how to transfer this model to the Phi-Sat camera. We trained this model in a newly compiled dataset of more than 100 verified flood events called WorldFloods. This model was launched on June 2021 onboard the Wild Ride D-Orbit mission and we are testing now its performance in space.
Jan 2021,
Generative Adversarial Networks in the Geosciences
Gonzalo Mateo-García, Valero Laparra, Christian Requena-Mesa and Luis Gómez-Chova
Deep Learning for the Earth Sciences
DOI🔗
BibTeX▼
@incollection{mateo-garcia_generative_2021,
title = {Generative {Adversarial} {Networks} in the {Geosciences}},
isbn = {978-1-119-64618-1},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/9781119646181.ch3},
language = {en},
urldate = {2022-08-09},
booktitle = {Deep {Learning} for the {Earth} {Sciences}},
publisher = {John Wiley \& Sons, Ltd},
author = {Mateo-García, Gonzalo and Laparra, Valero and Requena-Mesa, Christian and Gómez-Chova, Luis},
month = jan,
year = {2021},
doi = {10.1002/9781119646181.ch3},
note = {Section: 3
\_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/9781119646181.ch3},
keywords = {remote sensing, deep learning, Earth observation images, generative adversarial networks, geosciences applications},
pages = {24--36},
}
Abstract▼
One of the most exciting trends in machine learning nowadays is the use of deep networks to learn density properties. In this direction, the field of generative models has become a hot topic in deep learning, and different approaches have been proposed. In this chapter, we are going to overview some of the generative models based on deep learning and focus on one of the most used ones: the Generative Adversarial Networks (GANs). We review the main different families of GANs, and how these families have been applied to remote sensing problems. Finally, in the last section, we illustrate the use of GANs in remote sensing problems with two different applications.
Jan 2020,
Chapter 2.13 - Statistical biophysical parameter retrieval and emulation with Gaussian processes
Gustau Camps-Valls, Luis Gómez-Chova, Valero Laparra, Luca Martino, Gonzalo Mateo-García, Jordi Muñoz-Marí, Daniel H. Svendsen and Jochem Verrelst
Data Handling in Science and Technology
DOI🔗
BibTeX▼
@incollection{camps-valls_chapter_2020,
series = {Hyperspectral {Imaging}},
title = {Chapter 2.13 - {Statistical} biophysical parameter retrieval and emulation with {Gaussian} processes},
volume = {32},
url = {http://www.sciencedirect.com/science/article/pii/B9780444639776000158},
urldate = {2019-10-02},
booktitle = {Data {Handling} in {Science} and {Technology}},
publisher = {Elsevier},
author = {Camps-Valls, Gustau and Gómez-Chova, Luis and Laparra, Valero and Martino, Luca and Mateo-García, Gonzalo and Muñoz-Marí, Jordi and Svendsen, Daniel H. and Verrelst, Jochem},
editor = {Amigo, José Manuel},
month = jan,
year = {2020},
doi = {10.1016/B978-0-444-63977-6.00015-8},
keywords = {Machine Learning, Emulation, Earth observation, Gaussian process, Bayesian Inference, Earth Observing System, Fractional vegetation cover, Linear model of coregionalization, Model Inversion, Parameter Retrieval, Radiative transfer model},
pages = {333--368},
}
Abstract▼
Earth observation from satellites poses challenging problems where machine learning is being widely adopted as a key player. Perhaps the most challenging scenario that we are facing nowadays is to provide accurate estimates of particular variables of interest characterizing the Earth's surface. This chapter introduces some recent advances in statistical bio-geophysical parameter retrieval from satellite data. In particular, we will focus on Gaussian process regression (GPR) that has excelled in parameter estimation as well as in modeling complex radiative transfer processes. GPR is based on solid Bayesian statistics and generally yields efficient and accurate parameter estimates, accompanied by confidence intervals for the estimations. We will first review the standard application of GPs for parameter retrieval and inverse modeling, which is based on regressing the parameter of interest on the concurrent observations. Secondly, noting that very often several parameters need to be estimated simultaneously, we review the field of multioutput GPR and illustrate its application in gap filling of multiple time series of parameters. The third GP modeling that we review here is that of combining real and simulated data. Very often a forward model encoding the well-understood physical relations is available. Inverting the model with GP is a standard practice known as hybrid modeling. In addition, we review a joint GP (JGP) model that combines both in situ measurements and simulated data in a single GP model and allows us to transfer information across spatial, temporal, and land cover modalities such as different crops. In recent years, forced by the data deluge, a plethora of large-scale GP models were introduced. We review recent advances and illustrate their performance for the estimation of surface temperature from infrared sounding data. Finally, we take a reversed pathway and focus on mimicking physical models with GPs. We present an automatic emulation scheme that approximates the forward physical model via interpolation, reducing the number of necessary nodes. Empirical evidence of the performance of these models will be presented through illustrative examples of land and vegetation monitoring.
Sep 2012,
Master thesis: Stochastic Modeling in Finance Browian Motion and Stochastic Calculus
Gonzalo Mateo-García
Master Thesis
URL🔗
BibTeX▼
@mastersthesis{mateo-garcia_master_2012,
title = {Master thesis: {Stochastic} {Modeling} in {Finance} {Browian} {Motion} and {Stochastic} {Calculus}},
url = {https://www.dropbox.com/s/jrkwu0853xeleap/TFM_Definitivo.pdf},
1. The work constitutes an introduction to Stochastic Calculus and Continuous time stochastic processes. We went rigorously through the theory of Brownian motion proving some of its most important theorems. In addition, we have developed with high precision the theory of Itô integral pointing out their possible generalizations when we change the integrand process.
2. We presented, as a motivation for the theory, some of the most important applications of stochastic Calculus to Finance.
3. In the last chapter, fractional Brownian motion has been introduced. As it is explained, this process is gaining importance to model situations where the assumptions of Brownian motion are unrealistic.
4. We went through some of the most famous books in stochastic calculus, Brownian motion and martingale theory highlighting several passages of those books that contains the proofs of the theorems or a further development of the exposed ideas. This knowledge will be very useful since this work could serve as a guide through some of this texts.},
school = {Universidad Complutense de Madrid},
author = {Mateo-García, Gonzalo},
month = sep,
year = {2012},
}
Abstract▼
The motivation of this work was to be introduced into the mathematical theory of Stochastic Processes with the purpose of getting a solid background. The following list remarks some of the most important achievements of the project: 1. The work constitutes an introduction to Stochastic Calculus and Continuous time stochastic processes. We went rigorously through the theory of Brownian motion proving some of its most important theorems. In addition, we have developed with high precision the theory of Itô integral pointing out their possible generalizations when we change the integrand process. 2. We presented, as a motivation for the theory, some of the most important applications of stochastic Calculus to Finance. 3. In the last chapter, fractional Brownian motion has been introduced. As it is explained, this process is gaining importance to model situations where the assumptions of Brownian motion are unrealistic. 4. We went through some of the most famous books in stochastic calculus, Brownian motion and martingale theory highlighting several passages of those books that contains the proofs of the theorems or a further development of the exposed ideas. This knowledge will be very useful since this work could serve as a guide through some of this texts.