Project

General

Profile

OPeNDAP - Notes on OPeNDAP

Some facts about opendap:

An OPeNDAP server can serve an arbitrarily large collection of data. Data on the server is often in HDF or NetCDF format, but can be in any format including a user-defined format. Compared to ordinary file transfer protocols (e.g. FTP), a major advantage using OPeNDAP is the ability to retrieve subsets of files, and also the ability to aggregate data from several files in one transfer operation.

  • OPeNDAP is the name of the organization and the name of the protocol: Open-source Project for a Network Data Access Protocol
  • Data is stored at remote server
  • Data model is similar to NetCDF’s data model (with differences) : N-dimensional array container, with variables, dimensions and attributes
  • Only requested pieces of data are sent
    • Accessing small pieces of large files on a remote server can still be quick
    • Data is requested based on sub-setting along dimensions
  • OPeNDAP resources can be opened locally on your computer as if it were local files using the NetCDF library
  • Local files versus remote files is transparent
  • The concept of a file is gone, an OPeNDAP endpoint can represent thousands of files aggregated along a dimension; E.g. Usually concatenate a large time series observation to one endpoint using the time dimension
  • OPeNDAP does not necessarily need to be served from NetCDF files!!

Presentations about Climate4Impact and CLIPC: Details on OPeNDAP, NetCDF and WMS :

  1. https://dev.knmi.nl/attachments/669/EGU2015_ESSI2.1-MaartenPlieger-Climate4Impact-IS-ENES2.pdf
  2. https://dev.knmi.nl/attachments/672/KNMI_ADAGUC_WMS_WCS_NetCDF_CF_OpenDAP_DataFormats_20150616.pdf
  3. http://www.clipc.eu/media/clipc/org/documents/other/plieger_climate4impact_wms_opendap_clipc_portal_design_knmi.pdf
  4. https://dev.knmi.nl/attachments/719/KNMI_EGU2013_ESSI31_Plieger_generic_visualization_of_opendap_using_WMS_20130403.pdf
  5. https://dev.knmi.nl/attachments/718/KNMI_EGU2013_ESSI213_Plieger_climate4impact_20130403.pdf
  6. https://publicwiki.deltares.nl/download/attachments/69959684/present.pdf <-- OpenDAP vs WCS

Detailed info on usage of OpenDAP/WMS in Climate4Impact/CLIPC and OpenLayers3

CLIPC - Visualize NetCDF files using OpenDAP and WMS

Datasets for testing

OpenDAP URL for KNMI datasets:
  1. http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc
  2. http://opendap.knmi.nl/knmi/thredds/dodsC/ADAGUC/testsets/projectedgrids/testkdc_KNW-1.0_H37-ERA_NL_20131231.nc

When you click on this link you will get an error 400, because you are not making a valid opendap request.

You can make the following valid requests :

  1. DAS: The DAS service provides access to the 'semantic' metadata (aka domain metadata) for the data resource:
    http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc.das
  2. DDS: The DDS service provides access to the 'syntactic' metadata (aka use or structural metadata) for the data resource:
    http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc.dds
  3. DDX: The DDX is an attempt to combine the information in the both the DDS and DAS into one response:
    http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc.ddx
  4. DODS Binary data is received when the client sends a DODS request.
    http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc.dods?precipitation
  5. You can make a variable selection with the netcdf library:
    ncdump -h http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc?precipitation

For details on these see http://docs.opendap.org/index.php/DDX_versus_DDS_and_DAS

You can do ncdump -h or ncview on this file. You can pass it to ADAGUC via
http://climate4impact.eu/impactportal/ImpactService?source=&lt;opendap url>

You can enter this URL in a WMS viewer like the ADAGUC viewer by doing:
  1. Go to http://geoservices.knmi.nl/viewer2.0/
  2. Click "Add layers..." and "Add custom WMS service"
  3. Fill out: http://climate4impact.eu/impactportal/ImpactService?source=http://opendap.knmi.nl/knmi/thredds/dodsC/ADAGUC/testsets/projectedgrids/testkdc_KNW-1.0_H37-ERA_NL_20131231.nc and klik "Add"
  4. Choose a layer

Minimal OpenDAP implementation which only works with the NetCDF library as client:

https://dev.knmi.nl/projects/impactportal/repository/show/src/opendap/

References:

  1. https://en.wikipedia.org/wiki/OPeNDAP
  2. http://docs.opendap.org/

KNMI_EGU2013_ESSI213_Plieger_climate4impact_20130403.pdf (6.2 MB) Maarten Plieger, 08/26/2015 04:21 PM

KNMI_EGU2013_ESSI31_Plieger_generic_visualization_of_opendap_using_WMS_20130403.pdf (4.78 MB) Maarten Plieger, 08/26/2015 04:21 PM