PyCAMA

Prepared release 0.10.5 to fix an issue when processing NPP-VIIRS data.

2020-11-26T17:30:47.252852 s5p-val2-off-pn00 PyCAMA 0.10.2 [0000026634]: [E] [Errno -101] NetCDF: HDF error: b'/mnt/data1/storage_offl_l1b/pp_pycama/NPP/S5P_OFFL_L2__NP_BD3_20190510T081608_20190510T095738_08143_01_010100_20201125T083454.nc'
Traceback (most recent call last):
  File "/data/sw/MPC/MPC_PYCAMA_V00.10.02/src/pycama/CAMA_call.py", line 169, in main
    data.read()
  File "/data/sw/MPC/MPC_PYCAMA_V00.10.02/src/pycama/Reader.py", line 1220, in read
    self.read_file(files[p][i], p, scanline_selection=selected_scanlines[p])
  File "/data/sw/MPC/MPC_PYCAMA_V00.10.02/src/pycama/Reader.py", line 574, in read_file
    dim_names = f.dimension_names(v.primary_var)
  File "/data/sw/MPC/MPC_PYCAMA_V00.10.02/src/pycama/File.py", line 246, in dimension_names
    with netCDF4.Dataset(self.path, 'r') as ncref:
  File "netCDF4/_netCDF4.pyx", line 2321, in netCDF4._netCDF4.Dataset.__init__
  File "netCDF4/_netCDF4.pyx", line 1885, in netCDF4._netCDF4._ensure_nc_success
OSError: [Errno -101] NetCDF: HDF error: b'/mnt/data1/storage_offl_l1b/pp_pycama/NPP/S5P_OFFL_L2__NP_BD3_20190510T081608_20190510T095738_08143_01_010100_20201125T083454.nc'

This could be traced back to the separation between dimensions and the corresponding variables, as introduced by the L1B team. In the crashing code, I used a fallback to find the dimension names using the netCDF4 interface, while most of the reading is done via the h5py interface. I have now figured out how to achieve the same result using the h5py interface only.

Fixed in 0.10.5 (released today).