Project

General

Profile

API » History » Version 48

Maarten Plieger, 11/16/2015 12:51 PM

1 1 Maarten Plieger
h1. API
2 1 Maarten Plieger
3 48 Maarten Plieger
h2. Clipc combine process
4 48 Maarten Plieger
5 48 Maarten Plieger
[[Clipc Combine process]]
6 48 Maarten Plieger
7 28 Maarten Plieger
h2. Access token API - service "account"
8 1 Maarten Plieger
9 28 Maarten Plieger
*1). Generate a token for the current user which can be used to access C4I services by using "generatetoken":*
10 33 Maarten Plieger
* https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=generatetoken
11 1 Maarten Plieger
12 17 Maarten Plieger
The token has the following structure:
13 17 Maarten Plieger
<pre><code class="json">
14 18 Maarten Plieger
{ 
15 18 Maarten Plieger
  "creationdate" : "2015-10-23T09:50:05Z",
16 17 Maarten Plieger
  "notafter" : "2015-10-30T09:50:05Z",
17 17 Maarten Plieger
  "notbefore" : "2015-10-23T09:50:05Z",
18 22 Maarten Plieger
  "openid" : "https://esg-dn1.nsc.liu.se/esgf-idp/openid/<identifier>",
19 24 Maarten Plieger
  "token" : "daad2944-1def-4bee-b032-b08048f88302", 
20 22 Maarten Plieger
  "userid" : "esg-dn1.nsc.liu.se.esgf-idp.openid.<identifier>"
21 17 Maarten Plieger
}
22 17 Maarten Plieger
</code></pre>
23 17 Maarten Plieger
24 28 Maarten Plieger
*2). List all tokens available for the current user by using "listtokens":*
25 33 Maarten Plieger
* https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=listtokens
26 1 Maarten Plieger
27 1 Maarten Plieger
*Service endpoints which can be used with a token:*
28 1 Maarten Plieger
* WMS can be used via <pre>https://dev.climate4impact.eu/impactportal/adagucserver/<accesstoken>/?</pre>
29 1 Maarten Plieger
* WCS can be used via <pre>https://dev.climate4impact.eu/impactportal/adagucserver/<accesstoken>/?</pre>
30 1 Maarten Plieger
* WPS can be used via <pre>https://dev.climate4impact.eu/impactportal/WPS/<accesstoken>/?</pre>
31 1 Maarten Plieger
* File download requests and OpenDAP requests from the basket can be done via <pre>https://dev.climate4impact.eu/impactportal/DAP/<accesstoken>/<userid>/<file>"</pre>
32 29 Maarten Plieger
* Basket requests can be done via <pre>https://dev.climate4impact.eu/impactportal/basket/<accesstoken>?&service=basket&request=getoverview&</pre>
33 39 Maarten Plieger
34 41 Maarten Plieger
*3). Crash course token generation:*
35 39 Maarten Plieger
# Login with a google account at https://dev.climate4impact.eu/impactportal/account/OAuth2.jsp 
36 40 Maarten Plieger
# Generate a new token via https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=generatetoken
37 39 Maarten Plieger
# Copy paste the accesstoken value in your URL
38 1 Maarten Plieger
39 5 Maarten Plieger
h2. Basket requests
40 1 Maarten Plieger
41 2 Maarten Plieger
* Get overview of all files in the users basket: service=basket&request=getoverview, 
42 33 Maarten Plieger
** e.g. <pre>https://dev.climate4impact.eu/impactportal/basket?&service=basket&request=getoverview&</pre>
43 33 Maarten Plieger
** For commandline access with an access token: <pre>https://dev.climate4impact.eu/impactportal/basket/<accesstoken>/?&service=basket&request=getoverview&</pre>
44 7 Maarten Plieger
45 14 Maarten Plieger
The result is a json file with nested childs representing te directory structure at C4I
46 10 Maarten Plieger
<pre><code class="json">
47 12 Maarten Plieger
{ "browser" : "/impactportal/data/catalogbrowser.jsp?",
48 12 Maarten Plieger
  "children" : [ { "children" : [  ],
49 12 Maarten Plieger
        "date" : "",
50 12 Maarten Plieger
        "expanded" : true,
51 12 Maarten Plieger
        "iconCls" : "typeFolder",
52 12 Maarten Plieger
        "leaf" : false,
53 12 Maarten Plieger
        "text" : "Remote data"
54 1 Maarten Plieger
      },
55 13 Maarten Plieger
      { "children" : [ { "dapurl" : "https://localhost/impactportal/DAP/<userid>/INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",
56 12 Maarten Plieger
              "date" : "2014-06-06 14:31:23Z",
57 12 Maarten Plieger
              "filesize" : "611.573K",
58 1 Maarten Plieger
              "hasdap" : true,
59 12 Maarten Plieger
              "hashttp" : true,
60 13 Maarten Plieger
              "httpurl" : "https://localhost/impactportal/DAP/<userid>/INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",
61 12 Maarten Plieger
              "iconCls" : "typeOF",
62 12 Maarten Plieger
              "id" : "INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",
63 12 Maarten Plieger
              "index" : 1,
64 12 Maarten Plieger
              "leaf" : true,
65 1 Maarten Plieger
              "text" : "INTER_OPER_R___RD3_____L3__20100101T000000_20100201T000000_0004.nc",
66 12 Maarten Plieger
              "type" : "file"
67 12 Maarten Plieger
            },
68 13 Maarten Plieger
            { "dapurl" : "https://localhost/impactportal/DAP/<userid>/tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",
69 1 Maarten Plieger
              "date" : "2015-10-23 09:37:33Z",
70 12 Maarten Plieger
              "filesize" : "47.867M",
71 12 Maarten Plieger
              "hasdap" : true,
72 12 Maarten Plieger
              "hashttp" : true,
73 13 Maarten Plieger
              "httpurl" : "https://localhost/impactportal/DAP/<userid>/tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",
74 12 Maarten Plieger
              "iconCls" : "typeOF",
75 12 Maarten Plieger
              "id" : "tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",
76 12 Maarten Plieger
              "index" : 2,
77 12 Maarten Plieger
              "leaf" : true,
78 12 Maarten Plieger
              "text" : "tasmax_day_CNRM-CM5_historical_r1i1p1_1890.nc",
79 12 Maarten Plieger
              "type" : "file"
80 12 Maarten Plieger
            }
81 12 Maarten Plieger
          ],
82 12 Maarten Plieger
        "date" : "",
83 12 Maarten Plieger
        "expanded" : true,
84 12 Maarten Plieger
        "iconCls" : "typeFolder",
85 1 Maarten Plieger
        "leaf" : false,
86 12 Maarten Plieger
        "text" : "My data"
87 12 Maarten Plieger
      }
88 12 Maarten Plieger
    ],
89 12 Maarten Plieger
  "leaf" : false,
90 13 Maarten Plieger
  "text" : "https://esg-dn1.nsc.liu.se/esgf-idp/openid/<openid>",
91 11 Maarten Plieger
  "viewer" : "/impactportal/data/datasetviewer.jsp?"
92 11 Maarten Plieger
}
93 10 Maarten Plieger
</code>
94 9 Maarten Plieger
</pre>
95 30 Maarten Plieger
96 30 Maarten Plieger
h2. WPS Requests
97 30 Maarten Plieger
98 36 Maarten Plieger
Climate4impact uses PyWPS, see http://pywps.wald.intevation.org/documentation/course/ for more information
99 36 Maarten Plieger
100 36 Maarten Plieger
* WPS can be used via <pre>https://dev.climate4impact.eu/impactportal/WPS/<accesstoken>/?</pre>
101 36 Maarten Plieger
102 44 Maarten Plieger
103 44 Maarten Plieger
104 44 Maarten Plieger
*Example Python WPS client on WPS Process "binaryoperatoronnumbers"*
105 36 Maarten Plieger
106 46 Maarten Plieger
The following Python example behaves like a minimal WPS client, using C4I services to perform analysis. The example below makes use of the binaryoperatorfornumbers process, which enables operations on two numbers. The process needs five seconds to run, listing progress in percentages. This is done to ease progress monitoring developments. The process is executed asynchronously and is executed with the GET method. PyWPS also supports POST, this can be used for posting bigger inputs. This is described here: [[WPS]] 
107 46 Maarten Plieger
108 46 Maarten Plieger
The process has three inputs:
109 34 Maarten Plieger
# inputa - a float number
110 34 Maarten Plieger
# inputb - a float number
111 34 Maarten Plieger
# operation -  Can be "add", "substract", "divide" or "multiply"
112 35 Maarten Plieger
113 34 Maarten Plieger
114 30 Maarten Plieger
<pre><code class="python">
115 30 Maarten Plieger
#!/bin/python
116 30 Maarten Plieger
117 30 Maarten Plieger
"""
118 30 Maarten Plieger
Obtain an access token:
119 33 Maarten Plieger
- Login with a google account at https://dev.climate4impact.eu/impactportal/account/OAuth2.jsp 
120 30 Maarten Plieger
- Generate via https://dev.climate4impact.eu/impactportal/tokenapi?service=account&request=generatetoken
121 32 Maarten Plieger
- Copy paste the accesstoken value in the "token" variable below.
122 31 Maarten Plieger
"""
123 30 Maarten Plieger
124 30 Maarten Plieger
token="<access_token>" 
125 30 Maarten Plieger
 
126 30 Maarten Plieger
127 33 Maarten Plieger
wpsendpoint="https://dev.climate4impact.eu/impactportal/WPS/"+token+"/?"
128 30 Maarten Plieger
129 30 Maarten Plieger
inputa=11;
130 30 Maarten Plieger
inputb=22;
131 30 Maarten Plieger
operator="divide" # Any of add, substract, multiply, divide
132 30 Maarten Plieger
133 30 Maarten Plieger
print "Calculating ["+str(inputa)+" "+str(operator)+" "+str(inputb)+"]"
134 30 Maarten Plieger
135 30 Maarten Plieger
import xml.etree.ElementTree as et
136 30 Maarten Plieger
import urllib2
137 30 Maarten Plieger
import time
138 30 Maarten Plieger
139 30 Maarten Plieger
""" Parses URL to XML tree """
140 30 Maarten Plieger
def getandparsexml(url):
141 30 Maarten Plieger
  response = urllib2.urlopen(url)
142 30 Maarten Plieger
  html = response.read()
143 30 Maarten Plieger
  #print html
144 30 Maarten Plieger
  return et.fromstring(html)
145 30 Maarten Plieger
  
146 30 Maarten Plieger
# Execute the ultimatequestionprocess aynchronousely and obtain the statusLocation for progress
147 30 Maarten Plieger
datainputs = "inputa="+str(inputa)+";inputb="+str(inputb)+";operator="+str(operator)
148 30 Maarten Plieger
root = getandparsexml(wpsendpoint+"service=WPS&version=1.0.0&request=execute&storeExecuteResponse=true&status=true&identifier=binaryoperatorfornumbers&datainputs="+datainputs)    
149 30 Maarten Plieger
monitorURL = root.attrib['statusLocation']
150 30 Maarten Plieger
151 30 Maarten Plieger
print "Process executed checking " +monitorURL
152 30 Maarten Plieger
153 30 Maarten Plieger
# Get starttime of process, sometimes the monitorURL is not immediately available. Iterate until a valid creationtime is available would be good practice.
154 30 Maarten Plieger
numberOfTries=10
155 30 Maarten Plieger
while numberOfTries>0:
156 30 Maarten Plieger
  numberOfTries = numberOfTries-1 
157 30 Maarten Plieger
  try:
158 30 Maarten Plieger
    root = getandparsexml(monitorURL)
159 30 Maarten Plieger
    numberOfTries = -1
160 30 Maarten Plieger
    print "Process started at "+str(root[1].attrib['creationTime'])
161 30 Maarten Plieger
  except:
162 30 Maarten Plieger
    time.sleep(1)
163 30 Maarten Plieger
    pass
164 30 Maarten Plieger
165 30 Maarten Plieger
# Monitor progress, polling the statuslocation frequently
166 30 Maarten Plieger
percentage=0
167 30 Maarten Plieger
while percentage<100:
168 30 Maarten Plieger
  root = getandparsexml(monitorURL)
169 30 Maarten Plieger
  try:
170 30 Maarten Plieger
    percentage = float(root[1][0].attrib['percentCompleted'])
171 30 Maarten Plieger
  except:
172 30 Maarten Plieger
    percentage=100.0
173 30 Maarten Plieger
    pass
174 30 Maarten Plieger
  print "Progress: "+str(percentage)+ " complete."
175 30 Maarten Plieger
  time.sleep(1)
176 30 Maarten Plieger
177 30 Maarten Plieger
# Obtain the answer  
178 30 Maarten Plieger
root = getandparsexml(monitorURL)
179 30 Maarten Plieger
try:
180 30 Maarten Plieger
  print "The answer for ["+str(inputa)+" "+str(operator)+" "+str(inputb)+"] is "+(root[2][0][2][0].text)
181 30 Maarten Plieger
except:
182 30 Maarten Plieger
  print "Processing failed: "+root[1][0][0][0][0].text
183 30 Maarten Plieger
  pass
184 30 Maarten Plieger
</code>
185 30 Maarten Plieger
</pre>
186 42 Maarten Plieger
187 42 Maarten Plieger
The result is:
188 42 Maarten Plieger
<pre>
189 42 Maarten Plieger
(pyvirtenv)c4m@climate4impact:~$ python c4i_testwps_binaryoperator.py 
190 42 Maarten Plieger
Calculating [11 divide 22]
191 42 Maarten Plieger
Process executed checking http://climate4impact.eu/impactportal/WPS?OUTPUT=/pywps-144559994232.xml
192 42 Maarten Plieger
Process started at 2015-10-23T11:32:23Z
193 42 Maarten Plieger
Progress: 20.0 complete.
194 42 Maarten Plieger
Progress: 40.0 complete.
195 42 Maarten Plieger
Progress: 60.0 complete.
196 42 Maarten Plieger
Progress: 80.0 complete.
197 47 Maarten Plieger
Progress: 100.0 complete.
198 47 Maarten Plieger
The answer for [11 divide 22] is 0.5
199 47 Maarten Plieger
(pyvirtenv)c4m@climate4impact:~$ 
200 47 Maarten Plieger
</pre>