OPeNDAP - Notes on OPeNDAP¶
Some facts about opendap:¶
An OPeNDAP server can serve an arbitrarily large collection of data. Data on the server is often in HDF or NetCDF format, but can be in any format including a user-defined format. Compared to ordinary file transfer protocols (e.g. FTP), a major advantage using OPeNDAP is the ability to retrieve subsets of files, and also the ability to aggregate data from several files in one transfer operation.
- OPeNDAP is the name of the organization and the name of the protocol: Open-source Project for a Network Data Access Protocol
- Data is stored at remote server
- Data model is similar to NetCDF’s data model (with differences) : N-dimensional array container, with variables, dimensions and attributes
- Only requested pieces of data are sent
- Accessing small pieces of large files on a remote server can still be quick
- Data is requested based on sub-setting along dimensions
- OPeNDAP resources can be opened locally on your computer as if it were local files using the NetCDF library
- Local files versus remote files is transparent
- The concept of a file is gone, an OPeNDAP endpoint can represent thousands of files aggregated along a dimension; E.g. Usually concatenate a large time series observation to one endpoint using the time dimension
- OPeNDAP does not necessarily need to be served from NetCDF files!!
Presentations about Climate4Impact and CLIPC: Details on OPeNDAP, NetCDF and WMS :¶
- https://dev.knmi.nl/attachments/669/EGU2015_ESSI2.1-MaartenPlieger-Climate4Impact-IS-ENES2.pdf
- https://dev.knmi.nl/attachments/672/KNMI_ADAGUC_WMS_WCS_NetCDF_CF_OpenDAP_DataFormats_20150616.pdf
- http://www.clipc.eu/media/clipc/org/documents/other/plieger_climate4impact_wms_opendap_clipc_portal_design_knmi.pdf
- https://dev.knmi.nl/attachments/719/KNMI_EGU2013_ESSI31_Plieger_generic_visualization_of_opendap_using_WMS_20130403.pdf
- https://dev.knmi.nl/attachments/718/KNMI_EGU2013_ESSI213_Plieger_climate4impact_20130403.pdf
- https://publicwiki.deltares.nl/download/attachments/69959684/present.pdf <-- OpenDAP vs WCS
Detailed info on usage of OpenDAP/WMS in Climate4Impact/CLIPC and OpenLayers3¶
CLIPC - Visualize NetCDF files using OpenDAP and WMS
Datasets for testing¶
OpenDAP URL for KNMI datasets:- http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc
- http://opendap.knmi.nl/knmi/thredds/dodsC/ADAGUC/testsets/projectedgrids/testkdc_KNW-1.0_H37-ERA_NL_20131231.nc
When you click on this link you will get an error 400, because you are not making a valid opendap request.
You can make the following valid requests :
- DAS: The DAS service provides access to the 'semantic' metadata (aka domain metadata) for the data resource:
http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc.das - DDS: The DDS service provides access to the 'syntactic' metadata (aka use or structural metadata) for the data resource:
http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc.dds - DDX: The DDX is an attempt to combine the information in the both the DDS and DAS into one response:
http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc.ddx - DODS Binary data is received when the client sends a DODS request.
http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc.dods?precipitation - You can make a variable selection with the netcdf library:
ncdump -h http://opendap.knmi.nl/knmi/thredds/dodsC/CLIPC/storyline_urbanheat/clipcstorylinedata/test/RADNL_OPER_R___25PCPRR_L3__20120827T171000_20120827T171500_0001.nc?precipitation
For details on these see http://docs.opendap.org/index.php/DDX_versus_DDS_and_DAS
You can do ncdump -h or ncview on this file. You can pass it to ADAGUC via
http://climate4impact.eu/impactportal/ImpactService?source=<opendap url>
- Go to http://geoservices.knmi.nl/viewer2.0/
- Click "Add layers..." and "Add custom WMS service"
- Fill out: http://climate4impact.eu/impactportal/ImpactService?source=http://opendap.knmi.nl/knmi/thredds/dodsC/ADAGUC/testsets/projectedgrids/testkdc_KNW-1.0_H37-ERA_NL_20131231.nc and klik "Add"
- Choose a layer
Minimal OpenDAP implementation which only works with the NetCDF library as client:¶
https://dev.knmi.nl/projects/impactportal/repository/show/src/opendap/