Project

General

Profile

Adding GOES-16 data to ADAGUC » History » Version 1

Ernst de Vreede, 11/21/2018 11:35 AM

1 1 Ernst de Vreede
h1. Adding GOES-16 data to ADAGUC
2 1 Ernst de Vreede
3 1 Ernst de Vreede
ADAGUC can also handle GOES-16 NetCDF data.
4 1 Ernst de Vreede
GOES-16 NetCDF data is distributed by NOAA through an public dataset on Amazon S3. See https://registry.opendata.aws/ for more information on the public datasets available there.
5 1 Ernst de Vreede
6 1 Ernst de Vreede
The data is stored in a S3 bucket named noaa-goes16. The AWS S3 bucket can be accessed through the AWS command-line client, which you invoke with the command *aws*.
7 1 Ernst de Vreede
8 1 Ernst de Vreede
The AWS client by default wants to use AWS credentials; these can be entered  by running *aws config*. Open datasets can usually be accessed by anonymous users, by adding the *--no-sign-request* switch to the command.
9 1 Ernst de Vreede
10 1 Ernst de Vreede
Listing the primary folders in the noaa-goes16 bucket:
11 1 Ernst de Vreede
<pre>
12 1 Ernst de Vreede
aws s3 ls noaa-goes16/
13 1 Ernst de Vreede
</pre>
14 1 Ernst de Vreede
15 1 Ernst de Vreede
This generates a list of GOES-16 products (the PRE string indicates a S# bucket (pseudo)folder name):
16 1 Ernst de Vreede
<pre>
17 1 Ernst de Vreede
                           PRE ABI-L1b-RadC/
18 1 Ernst de Vreede
                           PRE ABI-L1b-RadF/
19 1 Ernst de Vreede
                           PRE ABI-L1b-RadM/
20 1 Ernst de Vreede
                           PRE ABI-L2-CMIPC/
21 1 Ernst de Vreede
                           PRE ABI-L2-CMIPF/
22 1 Ernst de Vreede
                           PRE ABI-L2-CMIPM/
23 1 Ernst de Vreede
                           PRE ABI-L2-MCMIPC/
24 1 Ernst de Vreede
                           PRE ABI-L2-MCMIPF/
25 1 Ernst de Vreede
                           PRE ABI-L2-MCMIPM/
26 1 Ernst de Vreede
                           PRE GLM-L2-LCFA/
27 1 Ernst de Vreede
</pre>
28 1 Ernst de Vreede
The folders with names ending in F contain the full-disc data.
29 1 Ernst de Vreede
For example the ABI-L2-MCMIPF/ folder contains multi-channel (16 channels) data for the full earth disc. In these folders, files are stored in folders per year and per day of the year. For
30 1 Ernst de Vreede
 example a file could be stored in the folder noaa-goes16/ABI-L2-MCMIPF/2018/324.
31 1 Ernst de Vreede
The contents of such a directory can be listed by the command:
32 1 Ernst de Vreede
<pre>
33 1 Ernst de Vreede
aws s3 ls noaa-goes16/ABI-L2-MCMIPF/2018/324/12/
34 1 Ernst de Vreede
</pre>
35 1 Ernst de Vreede
This command yields a list of filenames, which are rather difficult, so you might want to copy/paste them:
36 1 Ernst de Vreede
<pre>
37 1 Ernst de Vreede
2018-11-20 12:12:59  360805563 OR_ABI-L2-MCMIPF-M3_G16_s20183241200344_e20183241211122_c20183241211199.nc
38 1 Ernst de Vreede
2018-11-20 12:28:56  365359389 OR_ABI-L2-MCMIPF-M3_G16_s20183241215343_e20183241226122_c20183241227173.nc
39 1 Ernst de Vreede
2018-11-20 12:42:59  370656716 OR_ABI-L2-MCMIPF-M3_G16_s20183241230343_e20183241241110_c20183241241199.nc
40 1 Ernst de Vreede
2018-11-20 12:58:01  375511280 OR_ABI-L2-MCMIPF-M3_G16_s20183241245343_e20183241256110_c20183241256212.nc
41 1 Ernst de Vreede
</pre>
42 1 Ernst de Vreede
43 1 Ernst de Vreede
As an example let's look at one of these files in the ADAGUC AutoWMS:
44 1 Ernst de Vreede
Copy the required file from the S3 bucket with a command like:
45 1 Ernst de Vreede
<pre>
46 1 Ernst de Vreede
aws s3 cp s3://noaa-goes16/ABI-L2-MCMIPF/2018/324/12/OR_ABI-L2-MCMIPF-M3_G16_s20183241245343_e20183241256110_c20183241256212.nc  $HOME/data/adaguc-autowms
47 1 Ernst de Vreede
</pre>
48 1 Ernst de Vreede
49 1 Ernst de Vreede
(Don't forget to add s3:// before the filename!)
50 1 Ernst de Vreede
 
51 1 Ernst de Vreede
Refreshing the autowms list should show the copied file in the directory listing of the AutoWMS panel. The file can then be selected and viewed in Adaguc.
52 1 Ernst de Vreede
53 1 Ernst de Vreede
The selected variable from satellite file will now be shown with a default color table. Better suited colr tables could be configured manually.
54 1 Ernst de Vreede
55 1 Ernst de Vreede
There are also some quality indicator parameters in the files; these are usually not very nice to visualize.