Climate4Impact (C4I) Coding sprint KNMI-march-2014


Focus is on preparing the operational phase of climate4impact by integration of the prepared components (Search, Downscaling and WPS indices processing).

Monday 24 March 12:00 - 17:00
  • Tour de table
  • Status reports on activities
    • Natalia: ICCLIM and OCGIS
    • Maarten: WPS
  • Actions/planning for the next days
Tuesday 25 March 9:00-17:00
  • Antonio: downscaling portal services
  • Coding sprint
Wednesday 26 March 9:00-17:00
  • Coding sprint & wrap-up


  • Christian Page (CERFACS)
  • Maarten Plieger (KNMI)
  • Natalia Tatarinova (CERFACS)
  • Antonio Cofiño (UC Cantabria)
  • Michael Kolax (SMHI)
  • Wim Som de Cerff (KNMI)
  • Nils Hempelmann (DKRZ/HZG)
  • Nicolas Carenton (IPSL)
  • Renate Wilcke (SMHI)
  • Ernst de Vreede (KNMI)

Notes and actions from the sprint:

Actions and planning, notes day 1:
  • Development of new generic WPS Process with new OpenClimateGIS and ICCLIM
    • Summer Days first, use as template for other indices
    • Integrate in climate4impact portal
  • Search:
    • Integrate Processing in search page
      • How will this be integrated?
      • Define usecases (multimodel, multivariable and multiperiod)
    • Feedback in search on what items are selected
    • Show all possible facets, e.g. Cordex has different facets than CMIP5 (for example the domain)
    • Add help buttons to search page
  • Connection to downscaling portal of Cantabria
    • Authentication is working!
    • Do we make a special downscaling page? (yes, we will move Map&Plot and put Downscaling)
    • Build the downscaling web page on climate4impact
  • How do we organize the processing near the data (IS-ENES2/CLIPC/EUDAT)
    • WPS on the ESGF data node?
    • Which processes? Data reduction: Averaging, cumulative?
    • How to access the data, create a virtual (postprocessed) dataset?
    • How do we collaborate/connect with DKRZ Processing?
  • Develop wizard for Processing indices
  • Update Drupal
    • Test restoration CMS by using backup files
    • Test EUPORIAS glossary plugin
  • Make ESGF registration/Login page floating on climate4impact to indicate your are in climate4impact
    • Showing the “You are signed in” screen after signing is is not necessary, redirect to the original page is more useful (keep URL)
    • Use fancybox/iframe or window popup login
    • Will there an iframe problem with future browsers, preventing cross origin iframes?
Coding sprint notes day 2:
  • How do we collaborate/connect with DKRZ Processing?
    • Possibility to incorporate CDO routines in OCG for interpolation
      • Data handling is done by OCG
      • Interpolation and algorithms is done by CDO using Python bindings to CDO routines
      • Eventually CDO/OCGIS can come available in WPS
    • DKRZ will investigate whether they can use OCG in their WPS
      • When OCG is working, CDO can possibly be integrated.
      • Take advantage of OCG data handling in combination of CDO’s algorithms.
  • Connect from climate4impact to DKRZ’s WPS service
    • First attempts show some promising results
  • Connect from DKRZ to climate4impact’s WPS service
    • Climate4impact’s WPS is secured by X509 client authentication, adjustments to DKRZ routines need to be made first
  • How do we organize the processing near the data?
  • PID’s of the data and datasets (Persistent Identifiers)
    • Has been pushed in the 2013 December ESGF meeting, but it is not sure yet whether ESGF will pick this up.
    • In EUDAT all data will be referenced with PID’s
    • Referencing data by unique ID’s becomes more important (scientific publications, etc.)
  • Connection to downscaling portal of Cantabria
    • Use case and (web)form interface is defined
    • Technical implementation to support the use case has been identified
  • Develop new generic WPS Process with new OpenClimateGIS and ICCLIM
    • Response from OCG on callback function is promising, but not yet fully integrated
      • OCG callback is not fully integrated yet: icclim’s indices has no callback functionality yet
      • OCG is working on the callback functionality, when finished this will be implemented in the indice calculation as well.
      • Percentage feedback is provided, but feedback messages are still basic
Coding sprint notes day 3:
  • Work on email notification, we like to send users updates about climate4impact. E.g. when the downscaling is available. We would like to send all current registered users an email once with the question whether they want to receive updates on climate4impact. On their account page we will add a checkbox where you can choose whether you want to receive notifications.
    • Add a check box at account page to enable or disable email notifications
    • Derive email addresses from OpenId's
  • Search interface
    • Needs to be modular to support processing. We would like to be able to start processing indices from the search page.
    • Files within catalogues need to be accessible from the search page, the search page should already open the catalogues. Currently catalogues are opened in another page.
    • The catalogues need to be grouped when replicas are encountered, and should take care of older versions
    • It should be possible that collections of files in a catalog can be transferred at once to the basket, processing or download as zip-file.
  • It needs to be possible to group files in catalogs and pass them to processing/OCGIS


  • Connection between climate4impact WPS GUI and WPS from DKRZ was established
  • A start has been made with the floating ESGF login window in climate4impact
  • ICCLIM/OCGIS was succesfully installed by the participants
  • A new WPS based on ICCLIM/OCGIS is made
  • The climate4impact virtual machine development environment and links to the repository were distributed
  • Use case and (web)form interface is defined, use cases on how processing can be integrated in the search page were made