Connection with downscaling portal of University of Cantabria

The University of Cantabria downscaling portal (UC) will setup a password protected REST based API webservice which can be used by climate4impact. The REST service from UC will be run over HTTPS/SSL and is password protected, a username and password will be provided for the climate4impact portal. UC applies statistical downscaling and provides a set of pre-calculated downscaling methods and predictands which can be offered at climate4impact. Climate4impact will offer a list of predictands, downscaling methods and downscaling reports to a user. A user is then able to apply a downscaling method on his selected dataset (GCM). When the user starts the downscale process, his x509 certificate is sent to UC with his user id. When the downscale finishes successfully, the data is sent to the climate4impact portal and stored in the corresponding users space.

We identified some vocabulary:
  • predictand: Which observation to use for downscaling, usually point data from weather observations, but can be gridded observations as well. Always daily data.
  • predictor: GCM Model fields output to use for downscaling, can be daily or subdaily data
  • downscaling method: Relationship between reanalysis data or GCM model with observations (predictor <--> predictant). A list of pre-calculated downscaling methods is provided by UC.
  • Downscale: downscale a GCM with a downscaling method, the goal is that this can be done by a user at the climate4impact portal.
The University of Cantabria downscaling portal will provide the following operations:
  1. Get a list of available predictands. Probably using GET, climate4impact provides the OpenID identifier o thef current user.
  2. Get a list of downscaling methods for the corresponding predictand at 1
  3. Get a downscaling report for a downscaling method selected at 2, can be in PDF form
  4. Start downscale, by choosing GCM and downscaling method. The users openID identifier and his x509 certificate will be sent to UC by using POST. A check is done at UC to see whether the user has provided correct settings and will return a statusLocation identifier on success. The statuslocation can be polled once in a while for progress monitoring. When finished, the data is copied by climate4impact to the user space.
Some remarks:
  • We currently use x509 certificates which have no delegation mechanism, this will work differently when Oath2 is common technology.
  • The uploaded short lived credential (x509 cert) is valid for only 7 days, what happens if a downscale process takes longer? This issue will probably be solved when we migrate to Oath2.
    • For the short lived x509 cert, we could at least let the user know (suspend the job, waiting for the user to re-authenticate), and then continue the job AND/OR download all needed data first.
  • What is the status of Oath2 integration into ESGF?
  • Climate4impact currently interfaces with processing tools using the Web Processing Standard, is this standard also suitable for the link between UC and climate4impact? We think that the service of UC can be based on the WPS standard, but what about security?
  • The CMCC dashboard monitoring tool needs to check the status of the UC downscaling portal, this allows climate4impact to provide status information about UC to users.
  • The UC portal always needs a user ID for every operation.
  • In order to provide a list of predictands and downscaling methods, the UC portal always needs a kind of user id.
  • only a limited set of GCMs will be compatible with downscaling method chosen at step 2, we need to discuss how we implement this.
  • The predictand could be location-specific (or country-specific). We need to inform the user about it at step 1.
    • The first approach is to provide a pre-defined predictand-downscaling-method. Therefore the predictand will be defined for a collection of point location (like grid points) defined at a geographical region (contry, river basin, ...)
  • The predictand could be sub-daily, but in practice, for data volume concerns, we should keep it daily.
  • It would be great if the user can: a) rerun the same later, with little modifications: this would imply sending back to the user an XML file containing all the information to resubmit, but letting the user change some parameters; b) linked with a), let the user rerun with another downscaling method, another predictand, if applicable.
    • For this first prototype, a predictand will have different pre-defined downscaling methods.
  • How do the user deal with the training period? Because it depends on the reanalysis we want to use, as well as the observation dataset (predictand) and the region of interest.
    • The training period will not be selectable, all downscaling methods will be pre-fixed including reanalysis dataset and training/tes/validation periods. This means, that a predictand could have 2 different downscaling methods just selecting 2 different reanalysis datasets. We need to discuss how to let users change these options. Our idea is just to provide users direct access to the Downscaling Portal, where they can generate their downscaling methods, and upload their own predictands.
  • We normally can downscale all GCMs (and also RCMs with large enough grid): how do we have an ergonomic interface for the user to select? The user could want to compare GCM downscaled with different RCPs, an ensemble, etc.
    • Requires discussion with VALUE and CORDEX communities?