Project

General

Profile

API » History » Version 50

Ernst de Vreede, 03/02/2016 10:59 AM

1 1 Maarten Plieger
h1. API
2 1 Maarten Plieger
3 48 Maarten Plieger
h2. Clipc combine process
4 48 Maarten Plieger
5 49 Maarten Plieger
The WPS to combine two netcdf files: [[Clipc Combine process]]
6 48 Maarten Plieger
7 28 Maarten Plieger
h2. Access token API - service "account"
8 1 Maarten Plieger
9 28 Maarten Plieger
*1). Generate a token for the current user which can be used to access C4I services by using "generatetoken":*
10 33 Maarten Plieger
* https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=generatetoken
11 1 Maarten Plieger
12 17 Maarten Plieger
The token has the following structure:
13 17 Maarten Plieger
<pre><code class="json">
14 18 Maarten Plieger
{ 
15 18 Maarten Plieger
  "creationdate" : "2015-10-23T09:50:05Z",
16 17 Maarten Plieger
  "notafter" : "2015-10-30T09:50:05Z",
17 17 Maarten Plieger
  "notbefore" : "2015-10-23T09:50:05Z",
18 22 Maarten Plieger
  "openid" : "https://esg-dn1.nsc.liu.se/esgf-idp/openid/<identifier>",
19 24 Maarten Plieger
  "token" : "daad2944-1def-4bee-b032-b08048f88302", 
20 22 Maarten Plieger
  "userid" : "esg-dn1.nsc.liu.se.esgf-idp.openid.<identifier>"
21 17 Maarten Plieger
}
22 17 Maarten Plieger
</code></pre>
23 17 Maarten Plieger
24 28 Maarten Plieger
*2). List all tokens available for the current user by using "listtokens":*
25 33 Maarten Plieger
* https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=listtokens
26 1 Maarten Plieger
27 1 Maarten Plieger
*Service endpoints which can be used with a token:*
28 1 Maarten Plieger
* WMS can be used via <pre>https://dev.climate4impact.eu/impactportal/adagucserver/<accesstoken>/?</pre>
29 1 Maarten Plieger
* WCS can be used via <pre>https://dev.climate4impact.eu/impactportal/adagucserver/<accesstoken>/?</pre>
30 1 Maarten Plieger
* WPS can be used via <pre>https://dev.climate4impact.eu/impactportal/WPS/<accesstoken>/?</pre>
31 1 Maarten Plieger
* File download requests and OpenDAP requests from the basket can be done via <pre>https://dev.climate4impact.eu/impactportal/DAP/<accesstoken>/<userid>/<file>"</pre>
32 29 Maarten Plieger
* Basket requests can be done via <pre>https://dev.climate4impact.eu/impactportal/basket/<accesstoken>?&service=basket&request=getoverview&</pre>
33 39 Maarten Plieger
34 41 Maarten Plieger
*3). Crash course token generation:*
35 39 Maarten Plieger
# Login with a google account at https://dev.climate4impact.eu/impactportal/account/OAuth2.jsp 
36 40 Maarten Plieger
# Generate a new token via https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=generatetoken
37 39 Maarten Plieger
# Copy paste the accesstoken value in your URL
38 1 Maarten Plieger
39 50 Ernst de Vreede
h2. CLIP-C Access token API - service "clipcaccount"
40 50 Ernst de Vreede
41 50 Ernst de Vreede
*1). Generate a token for the current user which can be used to access C4I services by using "generatetoken":*
42 50 Ernst de Vreede
* https://dev.climate4impact.eu/impactportal/tokenapi?service=clipcaccount&request=generatetoken&userid=<userid>
43 50 Ernst de Vreede
44 50 Ernst de Vreede
The token has the following structure:
45 50 Ernst de Vreede
<pre><code class="json">
46 50 Ernst de Vreede
{ 
47 50 Ernst de Vreede
  "creationdate" : "2015-10-23T09:50:05Z",
48 50 Ernst de Vreede
  "notafter" : "2015-10-30T09:50:05Z",
49 50 Ernst de Vreede
  "notbefore" : "2015-10-23T09:50:05Z",
50 50 Ernst de Vreede
  "openid" : "https://esg-dn1.nsc.liu.se/esgf-idp/openid/<identifier>",
51 50 Ernst de Vreede
  "token" : "daad2944-1def-4bee-b032-b08048f88302", 
52 50 Ernst de Vreede
  "userid" : "esg-dn1.nsc.liu.se.esgf-idp.openid.<identifier>"
53 50 Ernst de Vreede
}
54 50 Ernst de Vreede
</code></pre>
55 50 Ernst de Vreede
56 5 Maarten Plieger
h2. Basket requests
57 1 Maarten Plieger
58 2 Maarten Plieger
* Get overview of all files in the users basket: service=basket&request=getoverview, 
59 33 Maarten Plieger
** e.g. <pre>https://dev.climate4impact.eu/impactportal/basket?&service=basket&request=getoverview&</pre>
60 33 Maarten Plieger
** For commandline access with an access token: <pre>https://dev.climate4impact.eu/impactportal/basket/<accesstoken>/?&service=basket&request=getoverview&</pre>
61 7 Maarten Plieger
62 14 Maarten Plieger
The result is a json file with nested childs representing te directory structure at C4I
63 10 Maarten Plieger
<pre><code class="json">
64 12 Maarten Plieger
{ "browser" : "/impactportal/data/catalogbrowser.jsp?",
65 12 Maarten Plieger
  "children" : [ { "children" : [  ],
66 12 Maarten Plieger
        "date" : "",
67 12 Maarten Plieger
        "expanded" : true,
68 12 Maarten Plieger
        "iconCls" : "typeFolder",
69 12 Maarten Plieger
        "leaf" : false,
70 12 Maarten Plieger
        "text" : "Remote data"
71 1 Maarten Plieger
      },
72 13 Maarten Plieger
      { "children" : [ { "dapurl" : "https://localhost/impactportal/DAP/<userid>/INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",
73 12 Maarten Plieger
              "date" : "2014-06-06 14:31:23Z",
74 12 Maarten Plieger
              "filesize" : "611.573K",
75 1 Maarten Plieger
              "hasdap" : true,
76 12 Maarten Plieger
              "hashttp" : true,
77 13 Maarten Plieger
              "httpurl" : "https://localhost/impactportal/DAP/<userid>/INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",
78 12 Maarten Plieger
              "iconCls" : "typeOF",
79 12 Maarten Plieger
              "id" : "INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",
80 12 Maarten Plieger
              "index" : 1,
81 12 Maarten Plieger
              "leaf" : true,
82 1 Maarten Plieger
              "text" : "INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",
83 12 Maarten Plieger
              "type" : "file"
84 12 Maarten Plieger
            },
85 13 Maarten Plieger
            { "dapurl" : "https://localhost/impactportal/DAP/<userid>/tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",
86 1 Maarten Plieger
              "date" : "2015-10-23 09:37:33Z",
87 12 Maarten Plieger
              "filesize" : "47.867M",
88 12 Maarten Plieger
              "hasdap" : true,
89 12 Maarten Plieger
              "hashttp" : true,
90 13 Maarten Plieger
              "httpurl" : "https://localhost/impactportal/DAP/<userid>/tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",
91 12 Maarten Plieger
              "iconCls" : "typeOF",
92 12 Maarten Plieger
              "id" : "tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",
93 12 Maarten Plieger
              "index" : 2,
94 12 Maarten Plieger
              "leaf" : true,
95 12 Maarten Plieger
              "text" : "tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",
96 12 Maarten Plieger
              "type" : "file"
97 12 Maarten Plieger
            }
98 12 Maarten Plieger
          ],
99 12 Maarten Plieger
        "date" : "",
100 12 Maarten Plieger
        "expanded" : true,
101 12 Maarten Plieger
        "iconCls" : "typeFolder",
102 1 Maarten Plieger
        "leaf" : false,
103 12 Maarten Plieger
        "text" : "My data"
104 12 Maarten Plieger
      }
105 12 Maarten Plieger
    ],
106 12 Maarten Plieger
  "leaf" : false,
107 13 Maarten Plieger
  "text" : "https://esg-dn1.nsc.liu.se/esgf-idp/openid/<openid>",
108 11 Maarten Plieger
  "viewer" : "/impactportal/data/datasetviewer.jsp?"
109 11 Maarten Plieger
}
110 10 Maarten Plieger
</code>
111 9 Maarten Plieger
</pre>
112 30 Maarten Plieger
113 30 Maarten Plieger
h2. WPS Requests
114 30 Maarten Plieger
115 36 Maarten Plieger
Climate4impact uses PyWPS, see http://pywps.wald.intevation.org/documentation/course/ for more information
116 36 Maarten Plieger
117 36 Maarten Plieger
* WPS can be used via <pre>https://dev.climate4impact.eu/impactportal/WPS/<accesstoken>/?</pre>
118 36 Maarten Plieger
119 44 Maarten Plieger
120 44 Maarten Plieger
121 44 Maarten Plieger
*Example Python WPS client on WPS Process "binaryoperatoronnumbers"*
122 36 Maarten Plieger
123 46 Maarten Plieger
The following Python example behaves like a minimal WPS client, using C4I services to perform analysis. The example below makes use of the binaryoperatorfornumbers process, which enables operations on two numbers. The process needs five seconds to run, listing progress in percentages. This is done to ease progress monitoring developments. The process is executed asynchronously and is executed with the GET method. PyWPS also supports POST, this can be used for posting bigger inputs. This is described here: [[WPS]] 
124 46 Maarten Plieger
125 46 Maarten Plieger
The process has three inputs:
126 34 Maarten Plieger
# inputa - a float number
127 34 Maarten Plieger
# inputb - a float number
128 34 Maarten Plieger
# operation -  Can be "add", "substract", "divide" or "multiply"
129 35 Maarten Plieger
130 34 Maarten Plieger
131 30 Maarten Plieger
<pre><code class="python">
132 30 Maarten Plieger
#!/bin/python
133 30 Maarten Plieger
134 30 Maarten Plieger
"""
135 30 Maarten Plieger
Obtain an access token:
136 33 Maarten Plieger
- Login with a google account at https://dev.climate4impact.eu/impactportal/account/OAuth2.jsp 
137 30 Maarten Plieger
- Generate via https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=generatetoken
138 32 Maarten Plieger
- Copy paste the accesstoken value in the "token" variable below.
139 31 Maarten Plieger
"""
140 30 Maarten Plieger
141 30 Maarten Plieger
token="<access_token>" 
142 30 Maarten Plieger
 
143 30 Maarten Plieger
144 33 Maarten Plieger
wpsendpoint="https://dev.climate4impact.eu/impactportal/WPS/"+token+"/?"
145 30 Maarten Plieger
146 30 Maarten Plieger
inputa=11;
147 30 Maarten Plieger
inputb=22;
148 30 Maarten Plieger
operator="divide" # Any of add, substract, multiply, divide
149 30 Maarten Plieger
150 30 Maarten Plieger
print "Calculating ["+str(inputa)+" "+str(operator)+" "+str(inputb)+"]"
151 30 Maarten Plieger
152 30 Maarten Plieger
import xml.etree.ElementTree as et
153 30 Maarten Plieger
import urllib2
154 30 Maarten Plieger
import time
155 30 Maarten Plieger
156 30 Maarten Plieger
""" Parses URL to XML tree """
157 30 Maarten Plieger
def getandparsexml(url):
158 30 Maarten Plieger
  response = urllib2.urlopen(url)
159 30 Maarten Plieger
  html = response.read()
160 30 Maarten Plieger
  #print html
161 30 Maarten Plieger
  return et.fromstring(html)
162 30 Maarten Plieger
  
163 30 Maarten Plieger
# Execute the ultimatequestionprocess aynchronousely and obtain the statusLocation for progress
164 30 Maarten Plieger
datainputs = "inputa="+str(inputa)+";inputb="+str(inputb)+";operator="+str(operator)
165 30 Maarten Plieger
root = getandparsexml(wpsendpoint+"service=WPS&version=1.0.0&request=execute&storeExecuteResponse=true&status=true&identifier=binaryoperatorfornumbers&datainputs="+datainputs)    
166 30 Maarten Plieger
monitorURL = root.attrib['statusLocation']
167 30 Maarten Plieger
168 30 Maarten Plieger
print "Process executed checking " +monitorURL
169 30 Maarten Plieger
170 30 Maarten Plieger
# Get starttime of process, sometimes the monitorURL is not immediately available. Iterate until a valid creationtime is available would be good practice.
171 30 Maarten Plieger
numberOfTries=10
172 30 Maarten Plieger
while numberOfTries>0:
173 30 Maarten Plieger
  numberOfTries = numberOfTries-1 
174 30 Maarten Plieger
  try:
175 30 Maarten Plieger
    root = getandparsexml(monitorURL)
176 30 Maarten Plieger
    numberOfTries = -1
177 30 Maarten Plieger
    print "Process started at "+str(root[1].attrib['creationTime'])
178 30 Maarten Plieger
  except:
179 30 Maarten Plieger
    time.sleep(1)
180 30 Maarten Plieger
    pass
181 30 Maarten Plieger
182 30 Maarten Plieger
# Monitor progress, polling the statuslocation frequently
183 30 Maarten Plieger
percentage=0
184 30 Maarten Plieger
while percentage<100:
185 30 Maarten Plieger
  root = getandparsexml(monitorURL)
186 30 Maarten Plieger
  try:
187 30 Maarten Plieger
    percentage = float(root[1][0].attrib['percentCompleted'])
188 30 Maarten Plieger
  except:
189 30 Maarten Plieger
    percentage=100.0
190 30 Maarten Plieger
    pass
191 30 Maarten Plieger
  print "Progress: "+str(percentage)+ " complete."
192 30 Maarten Plieger
  time.sleep(1)
193 30 Maarten Plieger
194 30 Maarten Plieger
# Obtain the answer  
195 30 Maarten Plieger
root = getandparsexml(monitorURL)
196 30 Maarten Plieger
try:
197 30 Maarten Plieger
  print "The answer for ["+str(inputa)+" "+str(operator)+" "+str(inputb)+"] is "+(root[2][0][2][0].text)
198 30 Maarten Plieger
except:
199 30 Maarten Plieger
  print "Processing failed: "+root[1][0][0][0][0].text
200 30 Maarten Plieger
  pass
201 30 Maarten Plieger
</code>
202 30 Maarten Plieger
</pre>
203 42 Maarten Plieger
204 42 Maarten Plieger
The result is:
205 42 Maarten Plieger
<pre>
206 42 Maarten Plieger
(pyvirtenv)c4m@climate4impact:~$ python c4i_testwps_binaryoperator.py 
207 42 Maarten Plieger
Calculating [11 divide 22]
208 42 Maarten Plieger
Process executed checking http://climate4impact.eu/impactportal/WPS?OUTPUT=/pywps-144559994232.xml
209 42 Maarten Plieger
Process started at 2015-10-23T11:32:23Z
210 42 Maarten Plieger
Progress: 20.0 complete.
211 42 Maarten Plieger
Progress: 40.0 complete.
212 42 Maarten Plieger
Progress: 60.0 complete.
213 42 Maarten Plieger
Progress: 80.0 complete.
214 47 Maarten Plieger
Progress: 100.0 complete.
215 47 Maarten Plieger
The answer for [11 divide 22] is 0.5
216 47 Maarten Plieger
(pyvirtenv)c4m@climate4impact:~$ 
217 47 Maarten Plieger
</pre>