Project

General

Profile

Statistics
| Branch: | Tag: | Revision:

pycama / src / pycama / Reader.py @ 838:750fb4349dae

History | View | Annotate | Download (68.4 KB)

# Date Author Comment
828:bd4380a19a5b 08/26/2020 11:00 PM Maarten Sneep

Remove warning for empty (NRTI) granules

823:809a0ab08949 08/19/2020 10:10 PM Maarten Sneep

Add metadata filter

818:8316d57368b6 07/29/2020 03:33 PM Maarten Sneep

Explicitly add the actual filename to the extracted metadata (issue #27361)

809:e12ac8c88c25 07/22/2020 02:46 PM Maarten Sneep

Capture empty input granules before they cause damage

766:c0412d2006c6 05/18/2020 08:58 AM Maarten Sneep

Silence some known warnings, or reduce them to info messages

746:702f246aae49 03/05/2020 06:35 PM Maarten Sneep

Add job-order file name and configuration file name(s) to netCDF4 output as attributes

740:9809f94372a2 02/21/2020 02:38 PM Maarten Sneep

Add check on fraction of "good" pixels before reading a granule

731:f6ef15292199 06/18/2019 04:57 PM Maarten Sneep

Add robustness against missing groups

714:188c6ff8e8df 03/28/2019 11:26 AM Maarten Sneep

Report time difference at info level

713:8daffbaa450a 03/27/2019 07:55 PM Maarten Sneep

Add difference configuration maker

709:a6312b49f9cf 02/22/2019 11:28 AM Maarten Sneep

Improve handling of empty granules in NRT processing

703:adfe2d8b896b 02/15/2019 05:16 PM Maarten Sneep

Update zonal average methods, is now a separate routine.
Add hemispheric and land/sea statistics to Histogram.
Update report creation.

696:6b307a7a5c9f 01/24/2019 05:02 PM Maarten Sneep

Avoid taking the minimum of a zero-length array

650:b57b21ee127e 10/15/2018 12:44 PM Maarten Sneep

Catch error for empty data

612:c81c6f56ee0a 08/21/2018 02:01 PM Maarten Sneep

Report range after applying filter

605:fed73990a11b 06/27/2018 06:25 PM Maarten Sneep

Add data selection based on valid_min/valid_max

600:109d30735633 06/27/2018 10:14 AM Maarten Sneep

Handle incorrect tracking_id attribute in NPP-Cloud product
Change time handling in time dependent plotting
Handle missing support variables in NPP-Cloud
Handle row count mismatch for cloud comparison

593:7966fe3d8d51 06/05/2018 06:44 PM Maarten Sneep

Match scanlines by time

579:bb1adc7e50ee 04/25/2018 12:01 PM Maarten Sneep

Change handling of 'empty' variables.

We now check for the solar zenith angles in the file.
If these are all > 88 degrees, we do not expect to find
valid data, and the granule is skipped. Before the variable
was removed.

510:62a51b9192df 08/11/2017 05:57 PM Maarten Sneep

Support map_scale key, improve error handling when reading scaling

507:4cbda88aa7e3 08/10/2017 03:49 PM Maarten Sneep

Allow for selective exclusion of variables from L3 creation

498:82f6d2bb96aa 08/09/2017 12:30 PM Maarten Sneep

Updated parameters for CH4 and CO

476:9988c9258094 08/02/2017 10:42 AM Maarten Sneep

Skip files where the scanline dimension has length zero

447:8a97afc61d27 07/11/2017 03:29 PM Maarten Sneep

Add input granule counter variable

444:8002dd721e12 07/11/2017 02:11 PM Maarten Sneep

Change input file count storage

442:c549c386f4ea 07/11/2017 01:15 PM Maarten Sneep

Add the number of input granules as a global attribute.

434:6ed64bbfce1d 07/07/2017 01:53 PM Maarten Sneep

Exclude variables from scatterplot if range is 0

429:6ca4343ccda8 07/07/2017 12:05 PM Maarten Sneep

Do not cast irradiance data to a masked array.

425:a4be601c2f4e 07/05/2017 02:56 PM Maarten Sneep

Improved handling of unconfigured variables; clarifying error/warning messages; improved product detection; better full_synchronize detection.

415:cc39f1ae18db 06/26/2017 02:20 PM Maarten Sneep

Remove variable when an empty granule is encountered, when "full_synchronize" is not in effect

409:8d560c380ba6 06/23/2017 12:00 PM Maarten Sneep

Add full_synchronize key, cleanu up reporting

397:3eca33b188c8 06/22/2017 01:36 PM Maarten Sneep

Update syncing rules to avoid masking all data by an empty variable

364:8e7db7095e13 05/01/2017 04:14 PM Maarten Sneep

Ensure that transformer object is only used when not None

350:642c4148ac35 04/04/2017 06:03 PM Maarten Sneep

Update version number to 0.3.2

344:9629a074e094 03/27/2017 12:38 PM Maarten Sneep

Write and read version number of PyCAMA to/from output to avoid version mismatches

329:292f9397b530 03/22/2017 12:34 PM Maarten Sneep

Transformation I/F change, real product name mapping support, count events per granule.

Transformer filters now store keyword parameters per filter, not per variable.

314:30108679f700 03/13/2017 03:32 PM Maarten Sneep

Properly deal with empty variables.

Clarify error message in case of length mismatch (see above).
Allow of internal and external product types (for comparing different versions of the same product).
Ensure that fill values are correctly dealt with.

303:29488ac061f4 02/23/2017 05:08 PM Maarten Sneep

Correct custom mapping of products.

300:e678812f6351 02/23/2017 09:39 AM Maarten Sneep

Merged

299:f937ef698427 10/24/2016 09:18 PM Maarten Sneep

Merged

294:6c82ebd94114 02/21/2017 05:26 PM Maarten Sneep

Handle special products; get variable in read_irradiance_data() to avoid stale references; close file references asap.

273:5f0f50951ce2 01/09/2017 11:22 AM Maarten Sneep

Add include_scatter option to allow exclusion from scatter density plots

267:66d125675305 12/08/2016 02:46 PM Maarten Sneep

Fixes for NP_BDx and OMI L2 files combined

257:75d3b5e9046c 12/01/2016 03:07 PM Maarten Sneep

Cleanup of the temporary QA4ECV hack

251:f288ad1f57b7 12/01/2016 01:16 PM Maarten Sneep

Add processing mode dependent variables

240:8b5720e11172 11/23/2016 06:05 PM Maarten Sneep

Periodically check the warning counter; make sure # ground_pixel is lowest encountered (filter will deal with rest)

232:46c8d10dffd7 11/21/2016 05:21 PM Maarten Sneep

Changes for reading OMI L2 data as well, update to version 0.1.3

230:b308d45dd26a 11/09/2016 03:41 PM Maarten Sneep

Capture that other error as well

225:5f89a74fc5d4 11/03/2016 06:08 PM Maarten Sneep

Extract processing performance figures

218:fab615e14746 10/25/2016 11:04 AM Maarten Sneep

Correctly capture transformation errors

212:1abfcdd69f6e 10/24/2016 04:09 PM Maarten Sneep

Clarify error message for easier debugging

202:4a35c3b896a3 10/24/2016 09:08 AM Maarten Sneep

Merged

199:d3dae3346085 10/23/2016 02:09 PM Maarten Sneep

Show progress to 0.1 % (NRTI contains 144 files, 1% is insufficient).

195:64ff0f57125a 10/21/2016 06:11 PM Maarten Sneep

Print correct orbit number (not orbit data). Capture more errors.

190:97fabc90aca5 10/20/2016 02:19 PM Maarten Sneep

Merged.

188:0af14483edd5 10/20/2016 02:17 PM Maarten Sneep

Improve robustness agains missing or empty attributes.

162:44c1e555d298 10/18/2016 04:14 PM Maarten Sneep

Remove variables that are not available in the input, no longer stop on missign variables; Clean up logging, add progress messages

143:07d7fb245706 10/13/2016 04:19 PM Maarten Sneep

switch to h5py

134:f57d792d43db 09/26/2016 04:11 PM Maarten Sneep

Add input pointer extraction; detect variables without scanline dimension; deal with non-synchonization (incomplete)

95:deed8124a1d4 09/09/2016 01:10 PM Maarten Sneep

Add option to configure (build) variables from configuration input.

Add collecting of variables that do not depend on the 'scanline' dimension.

81:536a5c23a7a1 08/23/2016 05:54 PM Maarten Sneep

Add event counting

74:b66004a0f8d8 08/17/2016 04:54 PM Maarten Sneep

PyCAMA updates, removal of vlen data (as much as possible), others

40:7b36ff0a1291 03/03/2016 04:06 PM Maarten Sneep

Check that first dimension is a time dimension of length 1 before cutting it off

32:132a60278bf8 03/03/2016 09:09 AM Maarten Sneep

Filter out NaN values from synthetic variables

23:3a400de68fde 02/25/2016 06:25 PM Maarten Sneep

Worldplot & histogram now working

22:cfaf8c58ebcb 02/22/2016 05:33 PM Maarten Sneep

Further progress

21:9689b14e0618 02/21/2016 10:44 PM Maarten Sneep

More analysis types & plot variants.

20:3d6468175881 02/18/2016 05:36 PM Maarten Sneep

Progressing

17:41d2382c94ba 02/16/2016 05:18 PM Maarten Sneep

Add combined data for global analysis

9:cb386e6e2a85 02/15/2016 04:59 PM Maarten Sneep

Include loop over time steps

5:3bfdb29d0ed0 02/14/2016 10:09 PM Maarten Sneep

Turn PyCAMA into a real package, including load magic for plugins.