OLCI Processing

Processing Baseline

This page summarises the evolutions of the processing baseline used to process OLCI-A data since July 2016, end of commissioning phase.

Related OLCI Processing Baseline documents:

• Sentinel-3 Product Notice - OLCI Level-1B NRT and NTC

• Sentinel-3 Product Notice - OLCI Level-2 NRT and NTC

• Sentinel-3 OLCI Data Hub Availability

• Sentinel-3 OLCI Product Data Format Specification - Level 1 products

• Sentinel-3 OLCI Product Data Format Specification - Level 2 products Land products

Table 1:

Summary of last outcomes of Product Notices

• Past anomalies corrected

• Known limitations

 Table 2:

Sentinel-3A OLCI Processing Baseline Timeline

Sentinel-3B OLCI Processing Baseline Timeline

L0 Production

Level-0 processing results in the generation of Level-0 products, i.e. time sorted and annotated data from Instrument Source Packet (ISP).

The first part of the process involves unpacking the ISPs, performing a quality check and appending annotation data to them. Once the input raw data files are read, all necessary data are extracted and parsed. The ISPs are then sorted and checked, including missing and duplicated packet numbering.

The final part of the processing is the Level-0 product generation. Several quality flags are computed and included in the associated metadata. Raw data, time sorted and annotated are included in the Level-0 package.

Level-0 processing should ensure:

• leap second management is handled

• product quality flags provide information concerning the nominal processing, satellite manoeuvre, contingency processing and degraded processing

• the total number of missing ISPs is reported

• no duplicated packets are included in Level-0 products

NOTE: OLCI Level-0 products are internal products only. They are not distributed to Sentinel-3 users.

L1 Algorithms

OLCI Level-1 processing is divided into three processing modes, each with its own Level-1B product:

• Earth Observation (EO) processing, aims to compute Level-1B products from Level-0 products related to raw data acquired by the instrument on one unique orbit in Earth observation configuration.

• Radiometric Calibration (RC), aims to compute calibration coefficients.

• Spectral Calibration (SC), aims to compute absolute wavelengths at specific positions on the sensor's arrays.

Earth Observation (EO) processing

Earth Observation (EO) processing inputs are Level-0 products, the orbit scenario file and several auxiliary data files providing calibration coefficient, surface classification or threshold for bright and glint classification. EO processing output is Level-1B data, i.e. radiometrically calibrated, geo-referenced and annotated radiances.

EO processing involves the calibrating of the the numerical counts contained in ISPs into radiances, geo-locating the acquired pixels on the Earth's surface and re-sampling the image onto an orthogonal product grid, representing the instrument's ideal swath. The final steps involve quality flags, meteorological annotations and pixel classification flags, appended with computed variables to the outputted Level-1B products.

EO processing is divided into seven steps:

1. Data extraction and quality checks from ISP products.

2. Radiometric scaling: derivation of calibrated TOA radiance values from the numerical counts previously extracted. This section is itself divided into six sub-sections: initialisation, non-linearity correction, dark signal correction, smear correction, absolute gain calibration and cosmetic pixel filling.

3. Stray light correction: a two-step process that estimates and corrects stray light contamination.

4. Geo-referencing: Computation, for every pixel, of the first intersection between the pixel line-of-sight and the Earth's surface (assumed to be perfectly represented by the WGS84 Reference Ellipsoid completed by a Digital Elevation Model).

5. Pixel classification: characterisation of the pixels according to classes of underlying surface, whatever the atmospheric conditions, to provide preliminary detection of cloudy pixels and to detect pixels showing a risk of contamination by sun glint.

6. Spatial re-sampling: definition and filling of the output products grids, taking into account the full Resolution and the Reduced Resolution grids.

7. Product formatting: production of the OL_1_EFR and OL_1_ERR products.

Note that data extraction, quality checks, instrument count corrections (included in the radiometric scaling step) and product formatting sub-sections are common, or share commonalities with the three processing levels.

Data Extraction and Quality Checks

Data extraction and quality check processing aims to extract useful data from Instrument Source Packets (ISPs). Science data is extracted from the ISP's data field as well as information necessary for further processing and detection of anomalies in the data stream such as:

• transmission error

• format error

• sequence error.

The OLCI measurement data are ordered and packaged, with additional information about the instrument status, into a sequence of strings of bits referred to as the OLCI ISPs.

The packets are input to the processing in the form of Level-0 product.
Level-0 products contain packets pertaining to a single instrument mode, nominally either Earth observation or calibration, with or without spectral relaxation. If a Level-0 product containing calibration data is submitted for Earth observation Level-1 processing, all its ISPs will be rejected by the data extraction step and processing will stop.
The Level-0 product may contain gaps (missing packets) of any size. Duplicates are assumed to have been removed by Level-0 processing.

Information in the packet header allows identification of:

• OLCI operational mode: Earth observation or calibration modes (calibration mode packets are ignored)

• events and exceptions in the operation of OLCI including disruptions in the clock or counter sequence, and instrument configuration changes.

Figure 1: Logical Flow of OLCI Data Extraction and Quality Checks

Radiometric Scaling

Radiometric processing, applied to the OLCI raw counts, aims to derive calibrated Top Of Atmosphere (TOA) radiance values. The incoming OLCI samples are processed one by one into radiance at TOA. Radiometric processing includes:

• initialisation

• non-linearity correction

• dark signal correction

• smear correction

• absolute gain calibration

• cosmetic pixel filling.

Initialisation

Long term trend corrections are applied to reference radiometric gain to match product time. Smear factors are computed. Non-linearity correction LUTs are built using a cubic interpolation and quality flags are initialised.

Non-Linearity Correction

All instrument counts are corrected for non-linearity, including the smear band.

Dark Signal Correction

All instrument counts are corrected for dark offset, including the smear band. Reference dark offset is corrected for short time temperature effect.

Smear Correction

The smear signal is estimated at each image sample from the smear band values and corrected for.

Absolute Gain Calibration

Instrument counts corrected of all the above instrumental effects are scaled to radiance using absolute gain coefficients corrected for long-term effects.

Cosmetic Pixel Filling

Some missing samples are filled with cosmetic radiance values and flagged as "cosmetic". Radiances of pixels listed in the "dead pixels" map are replaced by an interpolation of their valid neighbours. Empty samples generated during extraction, because of missing packets, are filled if the packet gap is small enough, by values from the previous valid frame.

Figure 2: Logical Flow of Radiometric Scaling

Stray Light Correction

The signal of a given sample is polluted by stray light coming into the instrument from other samples by means of either specular reflections (ghost images) or scatter. Stray light may be an significant contributor to the measured signal, particularly in the infra-red for ocean pixels close to clouds or land covered by vegetation.

The OLCI stray light correction algorithm is defined below. It uses characterisation of stray light contamination to estimate the degradation and correct it.

Stray light contribution to signal is evaluated from the already contaminated signal on the assumption that, since it is a small contribution, the fundamental signal structure is preserved and it can be considered as an epsilon (ε) in the approximation:

Straylight contamination can be split into the following two steps

1. A first contamination taking place in the ground imager, i.e. the imaging part of the instrument optics, with mixing of energy from the whole field-of-view, including both spatial dimensions, but without spectral mixing.

2. A second contamination occurring inside the spectrometer, with one spatial dimension (the along track direction) filtered out by the spectrometer entrance slit but including spectral mixing through scattering and reflections during or after spectral dispersion.

Therefore, stray light correction is implemented in two steps, following the instrument signal generation but in the backward direction. Since the output from radiometric scaling corresponds to the signal sensed at CCD surface, it includes both stray light contributions. It must therefore be corrected, firstly for the spectrometer contribution, and secondly for the ground imager contribution.

The spectrometer stray light term can be expressed as a two-dimensional convolution of the two-dimensional weighted radiance field. Note that the spectral dimension of the CCD shall be reconstructed from the 21 available samples (the OLCI channels) using linear interpolation on normalised radiance, avoiding the use of saturated samples.

The ground imager stray light contribution term can also be expressed as a two-dimensional convolution of the incoming radiance field, independently for each OLCI channel. It can be estimated from the radiance field corrected for the spectrometer contribution.

Georeferencing

Georeferencing of OLCI pixels involves computing for every pixel, the first intersection between the pixel line-of-sight and the Earth's surface. The Earth's surface is assumed to be perfectly represented by the WGS-84 reference ellipsoid completed by a Digital Elevation Model (DEM).

This algorithm includes four main steps:

1. Retrieve satellite location and attitude at the time of each OLCI acquisition (mid-exposure). This step is realised through calls to mission CFI functions, on the basis of NAVATT packets information. Direct handling of NAVATT packets by the mission CFI is not available, so the NAVATT data are used to generate orbit and attitude files consistent with their use by the mission CFI routines.

2. Retrieve instrument to satellite alignment matrix (accounting for along-orbit thermoelastic effects). This is achieved by a dedicated function providing the alignment matrix as a function of Position On Orbit (POO) (short term effect) and day-of-year (long-term seasonal effect).

3. Compute the intersection of each pixel's line-of-sight with the Earth's surface, including a high resolution elevation model. Compute the corresponding sun and view azimuth and zenith angles. This is implemented by the corresponding mission CFI functions, accounting for satellite location and attitude and instrument/satellite alignment derived in previous steps and to Earth/Sun ephemeris embedded in the CFI functionalities.

4. On the basis of the GeoReferencing information, pixels are further classified through calls to the Pixels Classification Functions described in next section

In addition to the four algorithm steps described above, implementation requires an additional upstream step for tools initialisation.

Figure 4: Logical Flow of Georeferencing

Pixel Classifications Functions

Several functions aimed at partitioning the pixels are identified and defined below:

• Classes of underlying surface, whatever the atmospheric conditions, on the basis of a priori knowledge from static or quasi-static atlas (currently land/saline water, fresh inland water (yes/no), high tidal activity areas (yes/no) and coastline (yes/no), that may evolve in the future).

• Pixels showing a risk of contamination by sun glint.

• Preliminary detection of cloudy pixels using simple brightness criteria.

The first type of classification is retrieved solely from the pixel geolocation information, while the second and third, being of a dynamic nature, are estimated from observation/illumination geometry and local wind conditions for the second, and from the measured radiance, taking account of the acquisition geometry, for the third. It is meaningful to split the classification process into three independent functions according to their required input data:

• APrioriClasses returns the a priori surface classes at the provided location.

• GlintRiskMask returns the possibility of sun glint (on water surfaces) according to current surface wind conditions and acquisition geometry.

• BrightMask returns the radiometrically based bright cloud classification from the input radiance at a given wavelength and the sun and view directions.

Spatial Re-Sampling

Spatial re-sampling processing defines and fills the output products grids. There are two such grids:

• Full Resolution (FR) product grid

• Reduced Resolution (RR) product grid (four times coarser than FR).

Both product grids are built on the basis of equally spaced time samples for the along track dimension and of samples equally spaced in on-ground distance for the across track dimension. The on-ground distance is computed at the surface of the reference ellipsoid (WGS84). The across track dimension is defined as the intersection between the YZ plane in the satellite actual reference frame and the reference ellipsoid.

Time sampling for the FR product is the OLCI instrument time sampling (44 ms) and the middle sample from four consecutive OLCI time samples for the RR grid. The across track distance sampling step for FR corresponds to the best instrument spatial sampling step (~300 m) and is specified through auxiliary data. The across track distance sampling step for RR is four times that of FR. Product pixels are filled with instrument pixels for FR and averages of surrounding instrument pixels for RR on the basis of the nearest neighbour re-sampling method.

The re-sampling scheme for FR pixels involves parsing the output product grid and, for every product pixel, knowing its across track pointing angle (the angle between the satellite nadir and the projection of the viewing direction onto the YZ plane, assigned with the sign of Y). The array of actual pointing directions of the OLCI pixels (converted into satellite actual reference frame) is searched for the closest one. Once found, the along track depointing of the corresponding OLCI pixel, available from characterisation (expressed in units of along track pixels and referred to as the frame offset) is used to address the appropriate OLCI frame, defining a unique pixel in the instrument 2D spatial grid. All the values affected by the OLCI instrument source pixel are then assigned to the current product pixel. The OLCI pixel is "marked" as used and if selected again, the corresponding product pixel is flagged as "duplicate". This multiple selection occurs often since instrument spatial resolution degrades at FOV edges while the product's spatial resolution remains constant. Processing continues with the next product pixel. As a consequence of the (potential) along track misalignment of OLCI pixels with respect to the theoretical viewing plane, alignment with the YZ plane requires the availability of several frames simultaneously during the re-sampling step.

The resampling scheme for RR follows a very similar scheme but takes advantage of the availability of the highest spatial resolution of the source data to minimise the spatial resolution degradation toward the FOV edges. This can be achieved by averaging a variable number of pixels in the across track direction according to the in-FOV position (or to the across track pointing angle). The number of pixels to be averaged in the along track direction remains constant (four). The number of pixels to be averaged in the across track direction is retrieved as a function of the across track pointing angle from the RR product pixel centre. The RR resampling is otherwise identical to FR resampling.

The regularity of the adopted grid as well as the relatively low accuracy requirements on their associated geo-reference data (accurate geo-reference is provided independently for each OLCI sample) allows the use of a sub-grid called the tie-point grid and bilinear interpolation in between. The tie-point grid has the same ground spacing in the along-track direction than the pixels grid, i.e. it is not sub-sampled, in order to allow maximum flexibility in products start and stop times. The tie-point grid has the same ground spacing, in the across-track direction, for the FR and RR product. The corresponding sub- sampling factor in the across-track direction, with respect to the FR product grid, is defined by a configuration parameter, and is currently set to 64 FR pixels (and consequently 16 RR pixels).

It is also only at tie-points that meteorological data are appended. Meteorological data are retrieved from global data grids provided by a weather forecast centre (ECMWF) and interpolated in time and space at tie-point locations. It is assumed that the closest forecasts/analysis files bracketing the OLCI product are available at the time of processing. It is also assumed that the products are provided in one of the ECMWF "reduced Gaussian grids".

Figure 5: Logical Flow of Spatial Re-Sampling

Radiometric Calibration (RC)

OLCI Radiometric Calibration (RC) processing is based on the same inputs as EO processing and outputs a set of calibration Look-Up-Tables (LUTs). The associated steps are:

1. Acquisition geometry.

2. Diffuser radiance computation, determining the radiance at instrument entrance, from knowledge of the acquisition geometry, sun flux and the diffuser bi-directional reflectance.

3. Stray light computation, similar to stray light correction in EO processing.

4. Radiometric LUT computation.

Figure 6: OLCI Radiometric Calibration Processing Top Level Breakdown

Spectral Calibration (SC)

OLCI Spectral Calibration (SC) processing accurately determines the central wavelengths of specific rows of the detector arrays and contributes to the overall accuracy and reliability of the instrument spectral model used during RC and EO processing.

SC processing uses two Level-0 products, pertaining to two successive orbits. The associated steps are:

1. Relative spectral calibration containing a data extraction and quality check sub-process (once for each input Level-0 product) and an instrument count correction sub-process (once for each input Level-0 product). A final sub-process derives the spectral diffuser's relative spectral BRDF from corrected counts of both diffusers.

2. Wavelength calibration analysing the above and deriving absolute wavelength characterisation of the instrument, over the whole field of view, by comparison of measured spectral BRDF to a reference.

Figure 7: OLCI Spectral Calibration Processing Top Level Breakdown

L2 Algorithms

OLCI Level-2 processing is divided into two main processes:

• Ocean processing, providing OL_2_WFR and OL_2_WRR products

• Land processing, providing OL_2_LFR and OL_2_LRR products.

In addition a common pre-processing and product formatting process aims to read and check the input, and to define and write the outputs.

It is important to note that, for each geophysical parameter included in the OLCI Level-2 product, a switch has been defined in the OLCI Level-2 configuration file. Each parameter and its associated flags are produced only if the associated switch is set to '1'. As a consequence, the module generating each parameter is triggered only if the appropriate switch is set to '1'.

Figure 8: OLCI Level-2 Processing Top Level Breakdown

Pre-Processing

The pre-processing module, starting from the Level-1B TOA radiances, derives reflectances corrected for gaseous absorption. The consolidation of pixel classifications from Level-1B and the definition of water vapour retrieval are included in this module.

The algorithm is divided into five successive steps:

1. The conversion from radiances to reflectances step-checks the Level-1B products and converts radiances into reflectances (also known as first instrumental correction).

2. To be correctly taken into account or to be rejected from the algorithm, pixels have to be differentiated according to four criteria: cloud, land, water and invalid pixels. The first pixel classification focuses on identification of cloudy pixels. This Cloud masking will be improved in the frame of coming evolutions, including the improvement tested and validated during the MERIS 4^th reprocessing.

3. Gaseous correction: correcting reflectances for gaseous absorption (i.e. O₂, H₂O and O₃). Five OLCI bands are dedicated to this correction and are not used after this step: Oa13 to Oa15, Oa19 and Oa20.

4. The second pixel classification estimates glint reflectance and completes pixel classification starting at the second step by consolidating the classification land and water pixels.

5. The water vapour process retrieves atmospheric water vapour content from clear sky pixels.

Radiance to Reflectance Conversion

This process aims to extract the input Level-1B data and convert the included radiance into reflectance through three main steps.

1. Pre-processing for geometry and meteorological parameters: This step is carried out to derive geometry and meteorological parameters (pressure, wind) at each pixel, including invalid ones, from those provided at each tie-point of the Level-1B annotation product.

2. Level-1B pixel classification screening. Level-2 pixel identification starts with reading of the INVALID flag of the Level-1B product. If it is set to TRUE, no further processing of the current pixel is performed, the Level-2 product shall contain fixed values for all geophysical products, with the INVALID flag set, and the next pixel is examined. Otherwise, processing of the current pixel is pursued.

3. Pixel extraction and reflectance conversion. If the Level-1B pixel is not flagged INVALID, the other Level-1B flags and the top of atmosphere radiances at all bands are extracted from the Level-1B product. Radiances are converted to reflectances using the sun zenith angle cosine interpolated at the pixel and the sun spectral flux, read from the Level-1B product annotations.

Note that this process also includes preliminary processing applied to annotation and measurements data sets (interpolation of annotations from the tie-points grid to the pixel grid and radiance to reflectance conversion respectively).

Figure 9: Functional Block Diagram of Radiances Into Reflectances Conversion

Gas Correction

The radiative transfer processes in the atmosphere and OLCI measurements are mainly affected by:

• air molecular scattering

• ozone absorption

• water vapour absorption

• oxygen absorption

• nitrogen-dioxide absorption

• chlorine-dioxide absorption

• aerosols scattering and absorption

• cloud scattering and absorption.

As a consequence, all the atmospheric constituents have to be investigated to estimate the impact of atmospheric gases on OLCI measurements. The amount of absorption as well as the vertical distribution of each atmospheric gas determines the impact and strategy for correction of its impact.

Before being able to fully identify land and water pixels and to perform smile correction, the TOA reflectances need to be corrected for absorption by atmospheric gases (NO₂, O₂, H₂ O and O₃). If absorption by nitrogen-dioxide, ozone and oxygen are based on meteorological or climatological data, specific OLCI channels are used to first estimate the atmosphere total water vapour content along the pixel light path to better correct for H₂ O absorption.

Figure 10: Functional Block Diagram of Gas Correction

Ocean Processing

The processing of MARINE products is performed by EUMESAT. Information can be found here https://www.eumetsat.int/ocean-colour-services

Land Processing

The land processing module consists of two independent sections (one for each product):

• The Green Instantaneous Fraction of Absorbed Photosynthetically Active Radiation (GI-FAPAR) section combines the information contained in the blue band with that contained in the bands at 681 and 865 nm to generate "rectified channels" at these latter two wavelengths.

• The OLCI Terrestrial Chlorophyll Index (OTCI) section uses Rayleigh correction to produce the necessary index. This step accounts for the spectral smile i.e. the in-FOV variation of the central wavelengths of OLCI channels.