Commandline access to secured ESGF NetCDF files¶
First install a myproxy client to obtain a certificate from one of the ESGF index nodes. The python tool MyProxyClient is suitable to do this: https://pypi.python.org/pypi/MyProxyClient/
(tip: create a python virtualenv if you have no root rights)
easy_install MyProxyClient
When installed, you can use the commandline tool myproxyclient to obtain the certificate, in this case from the pcmdi9 server:
myproxyclient logon -b -T -s pcmdi9.llnl.gov -l yourusername -o ~/creds.pem
You can check the obtained certificate with the following command:
openssl x509 -in creds.pem -text
The CA certificates can be retrieved from wget --no-check-certificate https://raw.githubusercontent.com/ESGF/esgf-dist/master/installer/certs/esg_trusted_certificates.tar
If you want to use the NetCDF tools directly on NetCDF files stored in ESGF datanodes, you need a small config file for NetCDF:
cat > .dodsrc << EOF HTTP.SSL.VALIDATE=0 HTTP.COOKIEJAR=${HOME}/.dods_cookies HTTP.SSL.CERTIFICATE=${HOME}/creds.pem HTTP.SSL.KEY=${HOME}/creds.pem HTTP.SSL.CAPATH=${HOME}/.globus/certificates EOF
Ncdump should now work:
ncdump -h http://albedo2.dkrz.de/thredds/dodsC/cmip5/output1/ICHEC/EC-EARTH/rcp26/day/atmos/day/r8i1p1/v20120213/tasmax/tasmax_day_EC-EARTH_rcp26_r8i1p1_20760101-21001231.nc
You can also download the file directly: (note that dodsC has changed to fileServer in the URL)
wget --no-check-certificate --private-key ${HOME}/creds.pem --certificate ${HOME}/creds.pem http://albedo2.dkrz.de/thredds/fileServer/cmip5/output1/ICHEC/EC-EARTH/rcp26/day/atmos/day/r8i1p1/v20120213/tasmax/tasmax_day_EC-EARTH_rcp26_r8i1p1_20760101-21001231.nc
Or with CURL:
curl -k -L -O "https://albedo2.dkrz.de/thredds/fileServer/cmip5/output1/ICHEC/EC-EARTH/rcp26/day/atmos/day/r8i1p1/v20120213/tasmax/tasmax_day_EC-EARTH_rcp26_r8i1p1_20760101-21001231.nc" --cert "./creds.pem" --cacert "./creds.pem" --cert-type PEM