Support #12131
PyCAMA test aborted when processing ALH
100%
Description
Anomalous behaviour of PyCAMA was detected during the validation test of PyCAMA in the PDGS release 2.2.0 update. This crash happened while extracting data from ALH.
... 2018-10-11T14:34:57.350796 pc170012.knmi.nl PyCAMA 0.8.0 [0000015655]: [W] Variable 'wavelength_calibration_offset' contains no valid data in orbit 01192 (removing granule from all analysis) 2018-10-11T14:34:59.867584 pc170012.knmi.nl PyCAMA 0.8.0 [0000015655]: [W] Orbit 1192 contains no valid data 2018-10-11T14:34:59.867838 pc170012.knmi.nl PyCAMA 0.8.0 [0000015655]: [W] Caught an unexpected exception 2018-10-11T14:34:59.867887 pc170012.knmi.nl PyCAMA 0.8.0 [0000015655]: [E] zero-size array to reduction operation minimum which has no identity Traceback (most recent call last): File "/usr/people/sneep/PyCAMA/pycama/src/pycama/CAMA_call.py", line 159, in main data.read() File "/usr/people/sneep/PyCAMA/pycama/src/pycama/Reader.py", line 1157, in read self.collect_aggregate_data() File "/usr/people/sneep/PyCAMA/pycama/src/pycama/Reader.py", line 990, in collect_aggregate_data if np.min(v.aggregate_data) == np.max(v.aggregate_data): File "/usr/people/sneep/.local/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 2417, in amin return amin(axis=axis, out=out, **kwargs) File "/usr/people/sneep/.local/lib/python3.6/site-packages/numpy/ma/core.py", line 5605, in min axis=axis, out=out, **kwargs).view(type(self)) File "/usr/people/sneep/.local/lib/python3.6/site-packages/numpy/core/_methods.py", line 29, in _amin return umr_minimum(a, axis, None, out, keepdims) ValueError: zero-size array to reduction operation minimum which has no identity 2018-10-11T14:34:59.941415 pc170012.knmi.nl PyCAMA 0.8.0 [0000015655]: [I] Processing time: 38.4 seconds 2018-10-11T14:34:59.941599 pc170012.knmi.nl PyCAMA 0.8.0 [0000015655]: [I] Errors: 1, warnings: 25 2018-10-11T14:34:59.941737 pc170012.knmi.nl PyCAMA 0.8.0 [0000015655]: [I] Stopping with exit code: 255
The issue could be reproduced, also in the current development version of PyCAMA. While the handling of the error can be improved, the cause is extracting data that is supposed to be empty (the wavelength calibration parameters). The in turn causes all data to be discarded, leading ultimately to the crash.
The solution is simple: a new configuration that does not try to extract empty parameters. This configuration file is attached here. I can make a configuration maintenance request for this file alone, or collect all updates - including the one for methane - and submit that later. Let me know how you want to proceed.
History
#1 Updated by PDGS Operations over 3 years ago
The modified configuration file was tested successfully for OFFL and NRTI processing in the PDGS VAL environment (PDGS-internal tracking: S5P_SCR-180).
#2 Updated by Maarten Sneep over 3 years ago
- Status changed from In Progress to Resolved
Thanks for the feedback. Resolving issue here.
#3 Updated by Maarten Sneep over 3 years ago
- Project changed from X-Phase E1 to PyCAMA
- Category deleted (
PyCAMA)
#4 Updated by Maarten Sneep over 3 years ago
- Status changed from Resolved to Closed
- % Done changed from 0 to 100