An ESA-funded project is showing the benefits of using raw Copernicus Sentinel-2 data and artificial intelligence to improve response times in early warning systems for catastrophic events.

An abundance of useful satellite imagery is now available, for a realm of applications. The Sentinel-2 mission the European Copernicus programme provides high-resolution optical imagery with good re-visit times over large areas, that are invaluable for land monitoring.

However, for applications that require rapid response, such as the monitoring of catastrophic events or the detection of illegal vessels, traditional data handling and processing schemes usually exhibit excessive latency. The processing is typically encompassed within the ground segment, which causes delay.

Traditional data handling and processing chain for generating a Level-1C product of Sentinel-2.

Copyright:Del Prete et al

The traditional data handling paradigm is optimised for ground processing, but is not well suited for early warning applications where latency is critical. While high-level products such as Level-1C are meticulously calibrated, ortho-rectified and include fine band co-registration, these features are not always necessary for specific applications that prioritise rapid response.

Directly identify forest wildfires with onboard AI processing

A team of researchers, working at ESA Φ-lab in ESRIN, Italy, have been investigating the benefits of using raw Copernicus Sentinel-2 data directly, instead of these traditional processed satellite products.

Their novel approach uses a low latency co-registration technique followed by an Artificial Intelligence (AI) model to identify forest wildfires faster.  

“In early warning applications, minimising response latency is a priority. By using raw satellite data directly onboard with AI, we bypass the extensive steps of traditional ground processing. This acceleration enables the provision of near real-time, actionable information that is crucial for disaster response teams on the ground,’’ says Roberto Del Prete, from the University of Naples, a visiting researcher at Φ-lab.  

Typically, AI techniques are trained using high-level satellite imagery, such as Level-1C data. To use AI models onboard, however, these high-level products must be produced on the satellite.

Filling the gap with raw satellite datasets

This replication of ground-based processing is computationally intensive and difficult to perform with limited onboard resources. To circumvent this, the team of researchers from Φ-lab developed a specialised technique to analyse raw data using deep learning methods.

“By analysing directly the raw data, the method can provide a low-latency response for use in near real-time detection, such as early wildfire alert systems and other applications where rapid response is required,” says Del Prete.

One prerequisite for using AI methodologies on raw data is the availability of raw satellite datasets. To fill this gap, the research team created a dataset of raw Sentinel-2 data for the detection of specific events, such as the thermal hotspots created by bushfires.

Wildfire detected in raw Copernicus Sentinel-2 raw imagery using the co-registration technique.

Copyright:ESA/EU

This represents the first release of raw Sentinel-2 products to the remote sensing community. Moreover, the team’s energy-efficient co-registration technique for on-board applications is accessible through a freely available API.

The co-registration technique is based on a statistical analysis of the band-to-band misalignments, calculated with a deep learning-based feature-matching architecture.

Del Prete adds, "The coarse co-registration technique is energy-efficient and substantially quicker than other benchmarked techniques." This technique is implemented without additional data corrections before its ingestion into the AI model.

Processing scheme of the raw wildfire detection application from Sentinel-2 imagery.

Copyright:Del Prete et al

Del Prete explains, "The AI model is directly trained on raw Sentinel-2 data and is capable of handling small residual band misalignments. This capability facilitates the low-latency responses needed for near real-time applications, such as early wildfire alert systems."

The AI algorithm runs on an onboard specialised component, known as the VPU (Vision Processing Unit), which is energy-efficient and designed for integration into future satellites. While deploying AI onboard presents challenges related to power and hardware limitations, the reduced latency makes it valuable for rapid response applications.

In the second phase of the project, a collaboration with Ubotica and TU Delft was established when the model trained to recognise fires on raw data was tested on space-graded hardware. A paper detailing this study will be released shortly.

Coarse co-registration technique.

Copyright:Del Prete et al

About the Copernicus Sentinels

The Copernicus Sentinels are a fleet of dedicated EU-owned satellites, designed to deliver the wealth of data and imagery that are central to the European Union's Copernicus environmental programme.

The European Commission leads and coordinates this programme, to improve the management of the environment, safeguarding lives every day. ESA is in charge of the space component, responsible for developing the family of Copernicus Sentinel satellites on behalf of the European Union and ensuring the flow of data for the Copernicus services, while the operations of the Copernicus Sentinels have been entrusted to ESA and EUMETSAT.

Did you know that?

Earth observation data from the Copernicus Sentinel satellites are fed into the Copernicus Services. First launched in 2012 with the Land Monitoring and Emergency Management services, these services provide free and open support, in six different thematic areas.

The Copernicus Emergency Management Service (Copernicus EMS) provides all operators involved in the management of natural disasters, man-made emergency situations, and humanitarian crises with timely and accurate geo-spatial information derived from satellite remote sensing and completed by available in situ or open data sources.

The Copernicus Land Monitoring Service (CLMS) provides geographical information on land cover and its changes, land use, vegetation state, water cycle and Earth's surface energy variables, to a broad range of users in Europe and across the World in the field of environmental terrestrial applications.

It supports applications in a variety of domains such as spatial and urban planning, forest management, water management, agriculture and food security, nature conservation and restoration, rural development, ecosystem accounting and mitigation/adaptation to climate change.