is-enes » impactportal

h1. API

h2. Clipc combine process

The WPS to combine two netcdf files: [[Clipc Combine process]]

h2. Access token API - service "account"

*1). Generate a token for the current user which can be used to access C4I services by using "generatetoken":*

* https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=generatetoken

The token has the following structure:

<pre><code class="json">

{ 

  "creationdate" : "2015-10-23T09:50:05Z",

  "notafter" : "2015-10-30T09:50:05Z",

  "notbefore" : "2015-10-23T09:50:05Z",

  "openid" : "https://esg-dn1.nsc.liu.se/esgf-idp/openid/<identifier>",

  "token" : "daad2944-1def-4bee-b032-b08048f88302", 

  "userid" : "esg-dn1.nsc.liu.se.esgf-idp.openid.<identifier>"

}

</code></pre>

*2). List all tokens available for the current user by using "listtokens":*

* https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=listtokens

*Service endpoints which can be used with a token:*

* WMS can be used via <pre>https://dev.climate4impact.eu/impactportal/adagucserver/<accesstoken>/?</pre>

* WCS can be used via <pre>https://dev.climate4impact.eu/impactportal/adagucserver/<accesstoken>/?</pre>

* WPS can be used via <pre>https://dev.climate4impact.eu/impactportal/WPS/<accesstoken>/?</pre>

* File download requests and OpenDAP requests from the basket can be done via <pre>https://dev.climate4impact.eu/impactportal/DAP/<accesstoken>/<userid>/<file>"</pre>

* Basket requests can be done via <pre>https://dev.climate4impact.eu/impactportal/basket/<accesstoken>?&service=basket&request=getoverview&</pre>

*3). Crash course token generation:*

# Login with a google account at https://dev.climate4impact.eu/impactportal/account/OAuth2.jsp 

# Generate a new token via https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=generatetoken

# Copy paste the accesstoken value in your URL

h2. CLIP-C Access token API - service "clipcaccount"

This is not a public API, but a machine to machine service end point, which the CLIPC server can use to request tokens onbehalfof the logged-in user.

This token then enables theCLIPC server to

*1). Generate a token for the current user which can be used to access C4I services by using "generatetoken":*

* https://dev.climate4impact.eu/impactportal/tokenapi?service=clipcaccount&request=generatetoken&userid=<userid>

The token has the following structure:

<pre><code class="json">

{ 

  "creationdate" : "2015-10-23T09:50:05Z",

  "notafter" : "2015-10-30T09:50:05Z",

  "notbefore" : "2015-10-23T09:50:05Z",

  "openid" : "https://esg-dn1.nsc.liu.se/esgf-idp/openid/<identifier>",

  "token" : "daad2944-1def-4bee-b032-b08048f88302", 

  "userid" : "esg-dn1.nsc.liu.se.esgf-idp.openid.<identifier>"

}

</code></pre>

h2. Basket requests

* Get overview of all files in the users basket: service=basket&request=getoverview, 

** e.g. <pre>https://dev.climate4impact.eu/impactportal/basket?&service=basket&request=getoverview&</pre>

** For commandline access with an access token: <pre>https://dev.climate4impact.eu/impactportal/basket/<accesstoken>/?&service=basket&request=getoverview&</pre>

The result is a json file with nested childs representing te directory structure at C4I

<pre><code class="json">

{ "browser" : "/impactportal/data/catalogbrowser.jsp?",

  "children" : [ { "children" : [  ],

        "date" : "",

        "expanded" : true,

        "iconCls" : "typeFolder",

        "leaf" : false,

        "text" : "Remote data"

      },

      { "children" : [ { "dapurl" : "https://localhost/impactportal/DAP/<userid>/INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",

              "date" : "2014-06-06 14:31:23Z",

              "filesize" : "611.573K",

              "hasdap" : true,

              "hashttp" : true,

              "httpurl" : "https://localhost/impactportal/DAP/<userid>/INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",

              "iconCls" : "typeOF",

              "id" : "INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",

              "index" : 1,

              "leaf" : true,

              "text" : "INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",

              "type" : "file"

            },

            { "dapurl" : "https://localhost/impactportal/DAP/<userid>/tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",

              "date" : "2015-10-23 09:37:33Z",

              "filesize" : "47.867M",

              "hasdap" : true,

              "hashttp" : true,

              "httpurl" : "https://localhost/impactportal/DAP/<userid>/tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",

              "iconCls" : "typeOF",

              "id" : "tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",

              "index" : 2,

              "leaf" : true,

              "text" : "tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",

              "type" : "file"

            }

          ],

        "date" : "",

        "expanded" : true,

        "iconCls" : "typeFolder",

        "leaf" : false,

        "text" : "My data"

      }

    ],

  "leaf" : false,

  "text" : "https://esg-dn1.nsc.liu.se/esgf-idp/openid/<openid>",

  "viewer" : "/impactportal/data/datasetviewer.jsp?"

}

</code>

</pre>

h2. WPS Requests

Climate4impact uses PyWPS, see http://pywps.wald.intevation.org/documentation/course/ for more information

* WPS can be used via <pre>https://dev.climate4impact.eu/impactportal/WPS/<accesstoken>/?</pre>

*Example Python WPS client on WPS Process "binaryoperatoronnumbers"*

The following Python example behaves like a minimal WPS client, using C4I services to perform analysis. The example below makes use of the binaryoperatorfornumbers process, which enables operations on two numbers. The process needs five seconds to run, listing progress in percentages. This is done to ease progress monitoring developments. The process is executed asynchronously and is executed with the GET method. PyWPS also supports POST, this can be used for posting bigger inputs. This is described here: [[WPS]] 

The process has three inputs:

# inputa - a float number

# inputb - a float number

# operation -  Can be "add", "substract", "divide" or "multiply"

<pre><code class="python">

#!/bin/python

"""

Obtain an access token:

- Login with a google account at https://dev.climate4impact.eu/impactportal/account/OAuth2.jsp 

- Generate via https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=generatetoken

- Copy paste the accesstoken value in the "token" variable below.

"""

token="<access_token>" 

wpsendpoint="https://dev.climate4impact.eu/impactportal/WPS/"+token+"/?"

inputa=11;

inputb=22;

operator="divide" # Any of add, substract, multiply, divide

print "Calculating ["+str(inputa)+" "+str(operator)+" "+str(inputb)+"]"

import xml.etree.ElementTree as et

import urllib2

import time

""" Parses URL to XML tree """

def getandparsexml(url):

  response = urllib2.urlopen(url)

  html = response.read()

  #print html

  return et.fromstring(html)

# Execute the ultimatequestionprocess aynchronousely and obtain the statusLocation for progress

datainputs = "inputa="+str(inputa)+";inputb="+str(inputb)+";operator="+str(operator)

root = getandparsexml(wpsendpoint+"service=WPS&version=1.0.0&request=execute&storeExecuteResponse=true&status=true&identifier=binaryoperatorfornumbers&datainputs="+datainputs)    

monitorURL = root.attrib['statusLocation']

print "Process executed checking " +monitorURL

# Get starttime of process, sometimes the monitorURL is not immediately available. Iterate until a valid creationtime is available would be good practice.

numberOfTries=10

while numberOfTries>0:

  numberOfTries = numberOfTries-1 

  try:

    root = getandparsexml(monitorURL)

    numberOfTries = -1

    print "Process started at "+str(root[1].attrib['creationTime'])

  except:

    time.sleep(1)

    pass

# Monitor progress, polling the statuslocation frequently

percentage=0

while percentage<100:

  root = getandparsexml(monitorURL)

  try:

    percentage = float(root[1][0].attrib['percentCompleted'])

  except:

    percentage=100.0

    pass

  print "Progress: "+str(percentage)+ " complete."

  time.sleep(1)

# Obtain the answer  

root = getandparsexml(monitorURL)

try:

  print "The answer for ["+str(inputa)+" "+str(operator)+" "+str(inputb)+"] is "+(root[2][0][2][0].text)

except:

  print "Processing failed: "+root[1][0][0][0][0].text

  pass

</code>

</pre>

The result is:

<pre>

(pyvirtenv)c4m@climate4impact:~$ python c4i_testwps_binaryoperator.py 

Calculating [11 divide 22]

Process executed checking http://climate4impact.eu/impactportal/WPS?OUTPUT=/pywps-144559994232.xml

Process started at 2015-10-23T11:32:23Z

Progress: 20.0 complete.

Progress: 40.0 complete.

Progress: 60.0 complete.

Progress: 80.0 complete.

Progress: 100.0 complete.

The answer for [11 divide 22] is 0.5

(pyvirtenv)c4m@climate4impact:~$ 

</pre>

h3. WPS JobList requests

All submitted jobs are captured and stored in a per user joblist. The joblist can be obtained via:

* https://dev.climate4impact.eu/impactportal/ImpactService?service=processor&request=getProcessorStatusOverview

A json with array called jobs is returned:

<pre>

{

  jobs:[{

    "wpspostdata": ...,

    "creationdate": "2016-03-30 16:57:03Z",

    "wpsid": "clipc_combine_execute",

    "processid": "pywps-a9840c38-f687-11e5-98b2-901b0e19e05f.xml",

    "statuslocation": "http://bhw485.knmi.nl:8280/impactportal/WPS?OUTPUT=/pywps-a9840c38-f687-11e5-98b2-901b0e19e05f.xml",

    "progress": "ready", // Ready indicated completed job

    "error": false,

    "status": "ready" // Ready, running, failed

  },{

    "wpspostdata":  ...,

    "creationdate": "2016-03-30 17:01:36Z",

    "wpsid": "wps_simple_indice",

    "processid": "pywps-4c323e96-f688-11e5-817a-901b0e19e05f.xml",

    "statuslocation": "http://bhw485.knmi.nl:8280/impactportal/WPS?OUTPUT=/pywps-4c323e96-f688-11e5-817a-901b0e19e05f.xml",

    "progress": "0 %", //Percentage completed

    "error": false,

    "status": "running" // Ready, running, failed

  }]

}

</pre>

A job can be removed from the list by doing:

* https://dev.climate4impact.eu/impactportal/ImpactService?service=processor&request=removeFromList&id=pywps-4c323e96-f688-11e5-817a-901b0e19e05f.xml