Coding sprint KNMI-April-2015


Wednesday April 1 - 13:00 till 17:00
  • C4I-Coding-Sprint-Welcome.pptx
  • Summary of recent developments
    • Summary of climate4impact reviews from Carter and Klump C4I-Coding-Sprint-reviews-clipc-results.pptx (Wim, Fokke)
    • Discuss and show latest developments on climate4impact
      • improved login, faceted search, vocabularies, etc...) (Maarten)
      • basket multidownload, wget script (Ernst)
      • Connection to UC downscaling portal (Manuel)
    • Documentation update C4I-Coding-Sprint-Documentation.pptx (Wim, Fokke)
    • WPS/ICCLIM developments and demonstration (Christian/Maarten)
  • Discussion
    • Implementation of new use cases, which and how (Christian)?
    • Discuss processing wizard
    • Discuss faceted search
    • Integration of jobs view from climate4impact WPS and downscaling portal
    • Prioritized wish list and new functionality
    • Plan actions for the next days
Thursday April 2 - 09:00 till 17:00
  • Getting work done
Friday April 3 - 09:00 till 17:00
  • Update prioritized wish-list
  • Getting work done


  • Manuel Vega Saldarriaga (UC Cantabria)
  • Christian Page (CERFACS)
  • Fokke de Jong (Alterra Wageningen UR)
  • Maarten Plieger (KNMI)
  • Wim Som de Cerff (KNMI)
  • Ernst de Vreede (KNMI)


Things to improve

  • Search
    • Improve information about projects (CORDEX/CMIP5), experiments, variables. Explain that hovering provides more information via tooltips. Use same styling as CMS for tooltips.
    • Wire the basic search to the faceted search, use the same search method/module in the future
    • Only first 30 results are displayed, use lazy loading to increase results.
    • Is it possible to sort results in the ESGF query? e.g. sortby=model,experiment,variable
      • Or when the search result only contains 100 results we can enable sorting.
    • Maybe it is possible to abstract the concept of files from the users. Only use datasets. The results of a search is a file, maybe created by processing many datasets.
    • When free text searching facets, why click search again? Enter!
    • Rename search to apply
  • Multiselect facets is not clear, ctrl button does not work on all devices.
    • Always use apply button?
    • Order facets which are used mostly
  • Vizualation
    • Big potential but needs more guidance
    • ADAGUC Viewer time panel does not show time resolution clearly
    • Would be very nice if simpe statistics could be calculation directly
    • Users do not know how to use it effectively
    • Color ramps should have pre-defined standardized suggestions but still be customizable
  • Downscaling
    • Only provides statistical downscaling
  • Datasetviewer
    • Free text search is not clear
    • ES-DOC link is broken
    • Datasets names/id's are too cryptic for novice users
    • Metadata about the dataset like geographic region and timestamp could be displayed
  • Too many browser tabs open all the time
    • Make the dataset viewer as overlay
    • Use this consistently through the portal
  • Login/signin
    • It is not clear that only a ESGF openid works. Google openid is not working for example
    • Group registration is not clear
    • OpenID is deprecated
    • From the login overlay, The howto opens in a new tab Confusing! --> Move to overlay
    • From the login overlay, The request help opens in a window but does not close. Confusing! --> Move to overlay
  • WCS
    • Direct links to QGIS do not work because of security reasons, x509 certificates are needed for client authentication
    • Download files directly to your computer and importing them in QGIS will work
    • We could make basic authentication at C4I which works with C4I.
    • AAIGRID, CSV/ GeoTIFF output is needed
  • Processing
    • Statistics, multiensemble output is desired
    • Reduced ensembles/information
    • Flexibility to put together various datasets/steps (time and space)
  • Documentation
    • Data quality vs uncertainty --> need for clear terminology, dfinition and ethics
    • Drupal CMS needs basic authentication in front to prevent spam
  • Processing
    • Subset by bounding box (almost working)
    • Subset by polygon/nuts region or uploaded set
  • Basket
    • Upload file validation. Do not upload images/movies etc. Check file headers.
    • Possibility to share files with others, make public. (Always need to be signed, but share with others who are signed in).
  • Downscaling
    • Should results be transferred to the climate4impact basket or should the results stay at UC with OpenDap access enabled?
  • Social media
    • Merge linked in on the climate4impact portal (icon and link to LinkedIn)
  • The "Basket Get script multifile download option" needs documentation
  • Basket needs icons for file types, can be opendap, http, opendap+http and catalog. Guide the user what is possible with these file types (e.g. processing on opendap, list of files on catalog, download on http)
  • Faceted search needs result pagination to browse through more than 30 results
  • Faceted search needs icons to show that catalogs are listed
  • Faceted search needs a wizard to preset important facets like project, frequency, variable, experiment. Simple search can be replaced by this wizard.
  • Faceted search needs extra information/tooltips about abbreviations from variables, experiments, projects, etc...
  • Dataset viewer needs icons to display the file types, same as basket
  • Dataset viewer needs functionality (by two buttons?) for processing and script download.
  • The dataset viewer and/or the faceted search need to show extra file information like timestamp and boundingbox. The file information is already available in the catalog. This can be implemented by hovering over the line where the file is displayed.
  • The dataset viewer needs to check files on errors, the same way as is checked at faceted search
  • From the dataset viewer, the fileviewer should open as overlay, not as new page
  • Modular design
    • Portal components (e.g. faceted search, or dataset viewer) need to have their own JavaScript, HTML and CSS files, to allow them to be used separate from the climate4imapct portal
    • Portal components should be embeddable by assigning an element ID and including the required Javascript and CSS files in the page
    • Using JSONP allows for cross origin ajax calls. This enables portal components to be re-used on other sites and they be developed on other machines than where the server is hosted.
    • The fileviewer needs to be reimplemented in JQuery and using JSONP.
  • Replicas need to be shown in the search/datasetviewer as well.

Use cases Christian

Expert simple use case

Select daily tas and psl for RCP45 in the time period 1950-2099 for all the models from CMIP5, for the Northern Hemisphere

Anomaly use case

Calculate the anomaly and provide me a GIS compatible map for tas from model CNRM CMIP5 r1i1p1 for RCP85, for time periods 2071-2100 and 1971-2000 over Western Europe

Developments since last sprint

  • Downscaling integration
  • Opendap server for every user
  • Upload your own files to the basket *


Day 1:

  • Improve search: Make use case 1 possible by allowing to filter and add all the search results to the basket at once.
    • filter on timestamp and variable. Take latest version
  • Replica handling
    • When a file gives an error, try to search for alternatives (hidden for the user)
  • Processing: Enable simple statistics calculation: average and substract using ICCLIM
  • Improve documentation
    • Get content from Ronald regarding EUPORIAS user stories (Fokke)
    • SRS scenario page; move below the RCPs and SSPs text (Fokke)
    • Cleanup of old pages (FAQ, Howto’s, AskBot, etc.) (Fokke, Wim)
    • Use case scenario’s: check if the data acquisition steps are valid (Fokke, Wim)
    • Implement reveiwers’ comments (Fokke, Wim)
    • Add IPCC glossary (Wim, workshop)
    • Cleanup C4I glossary (Wim, workshop)
    • Mapping table for short, longnames, usefull descriptions for use in mouse overs (e.g. CMOR descriptions?) (Ernst, Maarten, Wim)
    • Linkedin: make more use of it to build up community (all)
    • Check if we can do a statistical analysis of the usage of the site using PIWIK (Maarten)
    • Improve SPECS integration as it is now. We can improve it by editing the text of Paco into the portal itself: and have a link to EUPORIAS there as wel.
    • Adding SMHI use cases (from. EUPORIAS)
  • Make downscaling work on development version of C4I

C4I-Coding-Sprint-Welcome.pptx (186 KB) Maarten Plieger, 04/01/2015 07:37 AM

C4I-Coding-Sprint-Documentation.pptx (245 KB) Maarten Plieger, 04/01/2015 07:41 AM

C4I-Coding-Sprint-reviews-clipc-results.pptx (272 KB) Maarten Plieger, 04/01/2015 07:41 AM