Commandline access to secured ESGF NetCDF files¶

First install a myproxy client to obtain a certificate from one of the ESGF index nodes. The python tool MyProxyClient is suitable to do this: https://pypi.python.org/pypi/MyProxyClient/
(tip: create a python virtualenv if you have no root rights)

easy_install MyProxyClient

When installed, you can use the commandline tool myproxyclient to obtain the certificate, in this case from the pcmdi9 server:

myproxyclient logon -b -T -s pcmdi9.llnl.gov -l yourusername -o ~/creds.pem

You can check the obtained certificate with the following command:

openssl x509 -in creds.pem -text

The CA certificates can be retrieved from wget --no-check-certificate https://raw.githubusercontent.com/ESGF/esgf-dist/master/installer/certs/esg_trusted_certificates.tar

If you want to use the NetCDF tools directly on NetCDF files stored in ESGF datanodes, you need a small config file for NetCDF:

cat > .dodsrc << EOF
HTTP.SSL.VALIDATE=0
HTTP.COOKIEJAR=${HOME}/.dods_cookies
HTTP.SSL.CERTIFICATE=${HOME}/creds.pem
HTTP.SSL.KEY=${HOME}/creds.pem
HTTP.SSL.CAPATH=${HOME}/.globus/certificates
EOF

Ncdump should now work:

ncdump -h http://albedo2.dkrz.de/thredds/dodsC/cmip5/output1/ICHEC/EC-EARTH/rcp26/day/atmos/day/r8i1p1/v20120213/tasmax/tasmax_day_EC-EARTH_rcp26_r8i1p1_20760101-21001231.nc

You can also download the file directly: (note that dodsC has changed to fileServer in the URL)

wget --no-check-certificate --private-key ${HOME}/creds.pem --certificate ${HOME}/creds.pem http://albedo2.dkrz.de/thredds/fileServer/cmip5/output1/ICHEC/EC-EARTH/rcp26/day/atmos/day/r8i1p1/v20120213/tasmax/tasmax_day_EC-EARTH_rcp26_r8i1p1_20760101-21001231.nc

Or with CURL:

curl  -k -L -O "https://albedo2.dkrz.de/thredds/fileServer/cmip5/output1/ICHEC/EC-EARTH/rcp26/day/atmos/day/r8i1p1/v20120213/tasmax/tasmax_day_EC-EARTH_rcp26_r8i1p1_20760101-21001231.nc" --cert "./creds.pem" --cacert "./creds.pem" --cert-type PEM