A team of European remote sensing scientists has introduced a marine benchmark dataset for the detection of marine litter using Copernicus Sentinel-2 multispectral data. This open-access database enables the research community to develop marine debris detection solutions based on artificial intelligence.
It is a benchmark for developing and evaluating machine learning algorithms and the first dataset based on multispectral Copernicus Sentinel-2 data, which distinguish marine litter from co-existing marine features by exploring the spectral behaviour of certain floating materials, sea state features and water types.
In 2017, when the Motagua river in Guatemala flooded the Honduras bay with a tide of plastic trash, the photographs from Caroline Power triggered a viral media response. These shocking photographs inspired Greek researchers to test Copernicus Sentinel-2’s sensor capabilities for tracking dense marine debris patches. The results of this research have now been published, showing how multispectral satellite data were successfully leveraged to automate marine debris monitoring [1].
Images of plastic floods inspired the research
An estimated 4.8 to 12.7 million tonnes of plastic enter the ocean each year [2]. Plastic threatens marine life, human health and maritime safety. Remote sensing of marine debris is one way to track this problem, but detecting floating plastic using satellite images can be challenging. It is hard to distinguish plastic from macroalgae or other bright oceanic features such as waves, foam or sun glints.
Researchers from the Remote Sensing Laboratory of the National Technical University of Athens (NTUA) and the National and Kapodistrian University of Athens (NKUA), recently developed new marine litter monitoring solutions using multispectral data from the Sentinel-2 mission of the European Union’s Copernicus Programme. The researchers participate in the community of Remote Sensing of Marine Litter where a plethora of ESA-funded projects based on Copernicus Sentinel-2 data are on-going.
The study aimed at distinguishing dense macro marine debris accumulation from various co-existing marine features and to automate this method using machine learning. The results have been published in an open-access benchmark dataset, named MARIDA—MARIne Debris Archive, which enables remote sensing of many floating materials globally [1].
Copernicus Sentinel-2 shows dense plastic patches
Previous marine litter studies often relied on drones or in-situ portable spectroradiometers to measure the spectral signatures of various polymers [3] [4]. Investigating marine debris characteristics using multispectral satellite data has also been attempted. However, discrimination of plastics from features such as ships, waves, clouds and organic material was challenging. Notwithstanding those floating plastics are themselves diverse, having different colours, polymer types and water submersion levels. This study’s success depended on the use of open-access multispectral data from the Copernicus Sentinel-2 satellites.
Marine debris and their different spectral signatures
“Sentinel-2 was chosen since it has potential to detect relatively large accumulations of floating debris with a suitable spectral resolution of between 10 and 20 m, despite potential limitations to the spatial availability. The revisit time of 2-5 days was also sufficiently frequent,” says Katerina Kikaki, lead researcher from NTUA. The mission supports monitoring of Earth’s surface changes with two polar-orbiting satellites equipped with filter-based push-broom imager multispectral (MSI) sensors. Detecting various classes of floating material was possible thanks to the many bands of the sensors – at 10 m resolution the bands are blue (458 to 523 nm), green (543 to 578 nm), red (650 to 680 nm) and near-infrared (NIR) (785 to 900 nm). “While we cannot identify the specific polymer type, seeing only large accumulations of macro and micro plastics mixed together, with Copernicus Sentinel-2 multispectral images we can distinguish dense accumulations of plastic debris from other floating material or marine features,” explains Katerina Kikaki.
Examples of Sentinel-2 images and annotated features in MARIDA
Unlike existing landcover datasets, which provide extensive classification of various land types, marine floating material datasets often focus on particular classes. A dataset might target specific organic material such as the macroalgae Sargassum – classifying only dense Sargassum, sparse Sargassum or water. Other datasets are simply designed for binary classification (plastic/no plastic). This is where the MARIDA dataset is an outlier, providing in total 15 classes co-existing with plastic based on annotated multispectral data from Copernicus Sentinel-2. The classes in MARIDA range from plastic and organic debris, to clouds, wakes, foam on breaking waves or various water types, such as clear and turbid water.
Following an initial study in the Honduras Gulf, the researchers then chose 17 Copernicus Sentinel-2 Tiles at locations known for oceanic plastic pollution [5]. Copernicus Sentinel-2 data were chosen as they are open access and available for all coastal areas around the globe. During a seven-year period (2015-2021), the researchers gathered in-situ reports from citizen scientists, social media and ocean clean-up activities, to corroborate so-called marine debris events at these globally distributed locations. The MARIDA dataset was subsequently developed through careful annotation of multispectral images for the marine debris events at pixel level. Weather data, well established spectral indices and further verification using very high resolution (VHS) data from PlanetScope, were also used. The dataset structure is based on 63 Copernicus Sentinel-2 unique dates [6].
“Given dense plastic has a different spectral signature than dense Sargassum, for instance, they can be distinguished. However, the challenge arose when water was dominant while the floating material was sparse. Therefore, along with annotation, we also provided confidence levels for each pixel,” states Katerina Kikaki. Expert annotators recorded the thematic class and their confidence level for each digitised polygon. Annotated polygons were labelled with three confidence levels: high, moderate or low. After the annotation step, an inter-annotators agreement protocol was established. Important insights were gained also about the spectral behaviour of various classes of ocean features, using spectral signatures analysis and statistical analysis with the t-SNE algorithm. “The dataset is based on realistic cases, for example when clouds exist. So, the user can examine how the spectral signatures of features on the surface of our ocean change under different conditions,” concludes Katerina Kikaki.
Machine learning is a powerful tool, which in recent years has been harnessed by the remote sensing community. For the MARIDA dataset to be of use with machine learning algorithms and provide automated solutions, the final step in the study was to provide machine learning benchmarks. Looking to the future, the researchers hope the dataset will be expanded with other classes of floating materials such as macroalgae species or with even more verified marine debris events.
Read more on marine debris and Copernicus Sentinel-2, on the Sentinel Success story about the Earth Observation Tracking of Marine Debris in the Mediterranean Sea project.
About the Copernicus Sentinels
The Copernicus Sentinels are a fleet of dedicated EU-owned satellites, designed to deliver the wealth of data and imagery that are central to the European Union's Copernicus environmental programme. The European Commission leads and coordinates this programme, to improve the management of the environment, safeguarding lives every day. ESA is in charge of the space component, responsible for developing the family of Copernicus Sentinel satellites on behalf of the European Union and ensuring the flow of data for the Copernicus services, while the operations of the Copernicus Sentinels have been entrusted to ESA and EUMETSAT.
Did you know that?
Earth observation data from the Copernicus Sentinel satellites are fed into the Copernicus Services. First launched in 2012 with the Land Monitoring and Emergency Management services, these services provide free and open support, in six different thematic areas.
The Copernicus Marine Environment Monitoring Service (CMEMS) provides regular and systematic reference information on the physical and biogeochemical state, variability and dynamics of the ocean and marine ecosystems for the global ocean and the European regional seas.
References
[1] Kikaki, K., Kakogeorgiou, I., Mikeli, P., Raitsos, D. E., & Karantzalos, K. (2022). “MARIDA: A benchmark for Marine Debris detection from Sentinel-2 remote sensing data.” PloS one, 17(1), e0262247.
[2] “Plastic waste inputs from land into the ocean” Jambeck et al,SCIENCE Volume 347 | Issue 6223, 13 February 2015.
[3] Jakovljevic, G., Govedarica, M., Alvarez-Taboada, F. “A Deep Learning Model for Automatic Plastic Mapping Using Unmanned Aerial Vehicle (UAV) Data.” Remote Sens. 2020, 12, 1515.
[4] Shungudzemwoyo P. Garaba, Heidi M. Dierssen,. “An airborne remote sensing case study of synthetic hydrocarbon detection using short wave infrared absorption features identified from marine-harvested macro- and microplastics.” Remote Sensing of Environment, Volume 205, 2018, Pages 224-235, ISSN 0034-4257.
[5] Kikaki, A., Karantzalos, K., Power, C. A., & Raitsos, D. E. (2020). Remotely sensing the source and transport of marine plastic debris in Bay Islands of Honduras (Caribbean Sea). Remote Sensing, 12(11), 1727.
[6] MARIDA dataset can be downloaded from the repository Zenodo: https://zenodo.org/record/5151941#.Yfz5wPixV3g
[7] A quick start guide and detailed overview of the dataset can be found here: https://marine-debris.github.io/