Project

General

Profile

Bug #33291

[PDGS-ANOM-8804] Processing error on ALH

Added by Maarten Sneep about 2 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
Start date:
10/05/2021
Due date:
% Done:

100%


Description

Processing ALH with PyCAMA failed for 2021-07-14, 2021-07-15, and 2021-07-16 failed. There may be a correlation with the S-NPP input data.

History

#1 Updated by Maarten Sneep about 2 months ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

The NRTI reports are available for these days, the OFFL are not.

I downloaded one of these orbits(19461) and inspected it. I find the attribute number_of_successfully_processed_pixels in /METADATA/QA_STATISTICS with the value set to 0. If that is the case for all of these three days, then the PyCAMA failure is entirely expected. If there is no valid data, then there is nothing to analyse, and PyCAMA is expected to exit with an error.

So this question becomes: why where there no valid retrievals at all in these three days? From the metadata I see that there is an input file for the S-NPP data: S5P_OFFL_L2__NP_BD6_20210716T020830_20210716T035000_19461_02_010300_20210717T155913.nc. Looking at the PyCAMA results for that day for NP_BD6, I see no obvious problems.

Orbit 19461 ("bad")

  • number_of_groundpixels = 1869504 ;
  • number_of_processed_pixels = 1869504 ;
  • number_of_failed_retrievals = 1869504 ;
  • number_of_cloud_fraction_viirs_filter_occurrences = 908305 ;
  • number_of_sza_range_error_occurrences = 531244 ;
  • number_of_cirrus_reflectance_viirs_filter_occurrences = 233960 ;
  • number_of_snow_ice_filter_occurrences = 108618 ;
  • number_of_sun_glint_filter_occurrences = 65794 ;
  • number_of_mixed_surface_type_filter_occurrences = 18182 ;
  • number_of_altitude_roughness_filter_occurrences = 3362 ;
  • number_of_aai_filter_occurrences = 39 ;
  • number_of_ground_pixels_with_warnings = 211209 ;
  • number_of_cloud_warning_occurrences = 128946 ;
  • number_of_sun_glint_warning_occurrences = 65833 ;
  • number_of_pixel_level_input_data_missing_occurrences = 18374 ;
  • number_of_input_spectrum_warning_occurrences = 91 ;

Orbit 19380 ("good")

  • number_of_groundpixels = 1869056 ;
  • number_of_processed_pixels = 1869056 ;
  • number_of_successfully_processed_pixels = 109112 ;
  • number_of_failed_retrievals = 1759944 ;
  • number_of_cloud_fraction_viirs_filter_occurrences = 801086 ;
  • number_of_sza_range_error_occurrences = 530564 ;
  • number_of_convergence_error_occurrences = 140224 ;
  • number_of_snow_ice_filter_occurrences = 136136 ;
  • number_of_model_error_occurrences = 84028 ;
  • number_of_mixed_surface_type_filter_occurrences = 23982 ;
  • number_of_ler_range_error_occurrences = 13480 ;
  • number_of_altitude_roughness_filter_occurrences = 12054 ;
  • number_of_aai_scene_albedo_filter_occurrences = 11133 ;
  • number_of_sun_glint_filter_occurrences = 7248 ;
  • number_of_rejected_pixels_not_enough_spectrum = 7 ;
  • number_of_input_spectrum_missing_occurrences = 7 ;
  • number_of_wavelength_calibration_error_occurrences = 2 ;
  • number_of_ground_pixels_with_warnings = 547941 ;
  • number_of_cloud_warning_occurrences = 327164 ;
  • number_of_extrapolation_warning_occurrences = 231100 ;
  • number_of_pixel_level_input_data_missing_occurrences = 30359 ;
  • number_of_sun_glint_warning_occurrences = 7248 ;
  • number_of_input_spectrum_warning_occurrences = 2 ;

The errors add up to the total number of pixels, or rather the filters add up to the total number of pixels. Everything is filtered by the NPP-VIIRS data, the rest is rather normal. While the average cloud fraction is comparable to the days before and after the anomaly, the number of number_of_cirrus_reflectance_viirs_filter_occurrences is very high. This is likely an NPP-VIIRS processing issue propagating into the ALH product.

With that cause I'm closing this issue.

Also available in: Atom PDF