.. _tutorial:
Tutorial
========
WPS Client Examples
*******************
The examples for the WPS clients are using a demo WPS serivce on the bovec test machine at DKRZ.
https://bovec.dkrz.de/ows/proxy/copernicus?Service=WPS&Request=GetCapabilities&Version=1.0.0
There is currently not much data attached (especially for observation data).
Please use the default data selection parameters, otherwise no result might be returned.
CURL with HTTP Get requests
---------------------------
GetCapabilities
+++++++++++++++
Run ``GetCapabilities`` request to see which processes are available::
$ curl -s -o caps.xml \
"https://bovec.dkrz.de/ows/proxy/copernicus?Service=WPS&Request=GetCapabilities&Version=1.0.0"
DescribeProcess
+++++++++++++++
Run ``DescribeProcess`` request to see input/output parameters of the ``rainfarm`` process::
$ curl -s -o describe.xml \
"https://bovec.dkrz.de/ows/proxy/copernicus?Service=WPS&Request=DescribeProcess&Version=1.0.0&identifier=rainfarm"
Execute (sync mode)
+++++++++++++++++++
Run ``Exceute`` in synchronous mode for ``rainfarm`` with default input parameters:
.. code-block:: bash
$ curl -s -o execute.xml \
"https://bovec.dkrz.de/ows/proxy/copernicus?Service=WPS&Request=Execute&Version=1.0.0&identifier=rainfarm&DataInputs=regridding=0;slope=0"
.. warning::
The execution request my have a time-out. Please use the *asynchronous* mode for real testing.
A status document is returned. Open the URL with the reference to the output plot:
.. code-block:: xml
output
Execute (async mode)
++++++++++++++++++++
Run ``Exceute`` in asynchronous mode for ``rainfarm``:
.. code-block:: bash
$ curl -s -o execute.xml \
"https://bovec.dkrz.de/ows/proxy/copernicus?Service=WPS&Request=Execute&Version=1.0.0&identifier=rainfarm&DataInputs=regridding=0;slope=0&storeExecuteResponse=true&status=true"
A status document is returned.
.. code-block:: xml
rainfarm
RainFARM stochastic downscaling
PyWPS Process rainfarm accepted
Check the status document given by the ``statusLoction`` URL until the job has finished:
.. code-block:: bash
$ curl -s -o status.xml \
"https://bovec.dkrz.de/download/wpsoutputs/copernicus/8faddd84-8046-11e8-814d-109836a7cf3a.xml"
The final status document should similar to this one:
.. code-block:: xml
rainfarm
RainFARM stochastic downscaling
PyWPS Process RainFARM stochastic downscaling finished
output
Output plot
Open the URL pointing to the plot output.
OWSLib Python module
--------------------
`OWSLib`_ is a Python library to interact with OWS/OGC services like WPS, WMS, etc.
It is using the Python `requests`_ library.
.. todo::
Add IPython notebook.
http://birdhouse-workshop.readthedocs.io/en/latest/advanced/owslib.html
Birdy Command line client
-------------------------
`Birdy`_ is a WPS command line client.
Install birdy (Linux, macOS)::
$ conda install -c birdhouse -c conda-forge birdhouse-birdy
Set WPS service::
$ export WPS_SERVICE=https://bovec.dkrz.de/ows/proxy/copernicus # demo service on bovec
# OR
$ export WPS_SERVICE=http://localhost:5000/wps # your local WPS service
See which processes are available::
$ birdy -h
Run *rainfarm*::
$ birdy rainfarm -h
$ birdy rainfarm --regridding 0 --slope 0
Check the process status. The processes should finish after 10 seconds with a response simliar to this one::
ProcessSucceeded [####################################] 100%
Output:
output=https://bovec.dkrz.de/download/wpsoutputs/copernicus/ecac32a0-8047-11e8-ad8f-109836a7cf3a/RainFARM_example_64x64_pe72ysqs.png
Open the ouptut URL in Browser to see the plot.
Phoenix Web Client
------------------
You can run the demo processes directly without log-in on Phoenix.
* GetCapabilites: https://bovec.dkrz.de/processes/list?wps=copernicus
* DescribeProcess: https://bovec.dkrz.de/processes/execute?wps=copernicus&process=rainfarm
* Execute: press ``Submit`` button.
Job status is monitored. When job has finished you can either show the output directly or show the output details:
https://bovec.dkrz.de/monitor/details/3cd6b18e-81ce-431d-990a-6fca36cae052/outputs
WPS Client Examples with x509 Certificate
*****************************************
A WPS service can be secured with x509 certificates by using the `Twitcher`_ OWS security proxy.
A WPS ``Execute`` request can only be run when the WPS client provides a valid x509 proxy certificate.
In the following examples we will use a CP4CDS WPS demo service which is protected by a Twitcher security proxy.
It will only accept x509 proxy certificates from `ESGF`_ to execute a process. The ``GetCapabilites`` and ``DescribeProcess``
requests are public.
CURL with HTTP Get requests
---------------------------
The following examples are using ``curl``. You may also like to use the Firefox `RestClient`_ plugin.
GetCapabilities
+++++++++++++++
Run ``GetCapabilities`` request to see which processes are available::
$ curl -s -o caps.xml \
"https://bovec.dkrz.de:5000/ows/proxy/copernicus?Service=WPS&Request=GetCapabilities&Version=1.0.0"
DescribeProcess
+++++++++++++++
Run ``DescribeProcess`` request to see input/output parameters of the ``rainfarm`` process::
$ curl -s -o describe.xml \
"https://bovec.dkrz.de:5000/ows/proxy/copernicus?Service=WPS&Request=DescribeProcess&Version=1.0.0&identifier=rainfarm"
Execute (sync mode)
+++++++++++++++++++
Run ``Exceute`` in synchronous mode for ``rainfarm`` with default input parameters:
.. code-block:: bash
$ curl -s -o execute.xml \
"https://bovec.dkrz.de:5000/ows/proxy/copernicus?Service=WPS&Request=Execute&Version=1.0.0&identifier=rainfarm&DataInputs=regridding=0;slope=0"
You should get an exception report asking you to provide a x509 certificate:
.. code-block:: xml
A valid X.509 client certificate is needed.
Get a valid x509 certifcate from `ESGF`_, for example using the `esgf-pyclient`_.
See the `logon example`_.
Let's say your proxy certificate is in the file ``cert.pem``.
Run the curl example above with this certificate:
.. code-block:: bash
$ curl -s -o execute.xml --cert cert.pem --key cert.pem \
"https://bovec.dkrz.de:5000/ows/proxy/copernicus?Service=WPS&Request=Execute&Version=1.0.0&identifier=rainfarm&DataInputs=regridding=0;slope=0"
If your certificate is valid then your process will be executed (sync mode) and you will get an XML result document
providing you with URL references to a generated plot:
.. code-block:: xml
output
Try more examples as shown in the examples above using a x509 certificate.
Using Python requests library
-----------------------------
In this example we show how you can use the Python `requests`_ library to run WPS requests.
.. code-block:: python
import requests
# GetCapabilites
url = "https://bovec.dkrz.de:5000/ows/proxy/copernicus?request=GetCapabilities&service=WPS"
requests.get(url, verify=True).text
# DescribeProcess
url = "https://bovec.dkrz.de:5000/ows/proxy/copernicus?request=DescribeProcess&service=WPS&version=1.0.0&identifier=sleep"
requests.get(url, verify=True).text
# Execute with client certifcate cert.pem
url = "https://bovec.dkrz.de:5000/ows/proxy/copernicus?request=Execute&service=WPS&version=1.0.0&identifier=sleep&DataInputs=delay=1"
requests.get(url, cert="cert.pem" verify=True).text
See the `requests documentation`_ for details.
Using OWSLib Python library
---------------------------
An example with `OWSLib`_.
Make sure you have the latest version from the conda birdhouse channel:
.. code-block:: sh
$ conda install -c birdhouse -c conda-forge owslib
Run the *hello* process with a client certificate:
.. code-block:: python
from owslib.wps import WebProcessingService
wps = WebProcessingService(url="https://bovec.dkrz.de:5000/ows/proxy/copernicus",
verify=True, cert="cert.pem")
exc = wps.execute(identifier='sleep', inputs=[('delay', '1')])
exc.checkStatus()
exc.getStatus()
exc.isSucceded()
exc.processOutputs[0].data
Using Birdy
-----------
An example with `Birdy`_
Install latest birdy (Linux, macOS) from conda birdhouse channel:
.. code-block:: sh
$ conda install -c birdhouse -c conda-forge birdhouse-birdy owslib
.. code-block:: sh
$ export WPS_SERVICE=https://bovec.dkrz.de:5000/ows/proxy/copernicus
$ birdy -h
$ birdy sleep -h
$ birdy --cert cert.pem sleep --delay 1
Using Docker
************
Get docker images using docker-compose::
$ docker-compose pull
Start the demo with docker-compose::
$ docker-compose up -d # runs with -d in the background
$ docker-compose logs -f # check the logs if running in background
By default the WPS service should be available on port 5000::
$ firefox "http://localhost:5000/wps?service=wps&request=GetCapabilities"
Run docker exec to watch logs::
$ docker ps # find container name
copernicus-wps-demo_copernicus_1
$ docker exec copernicus-wps-demo_copernicus_1 tail -f /opt/wps/pywps.log
Use docker-compose to stop the containers::
$ docker-compose down
Testdata
********
For the demo processes you can fetch CMIP5 test-data from the ESGF archive.
You need a valid ESGF credentials which you can fetch for example with `esgf-pyclient`_.
For the examples you need CMIP5 data with the following facets:
* project=CMIP5
* experiment=historical
* ensemble=r1i1p1
* variable=ta, tas, or pr
* model=MPI-ESM-LR
* time_frequency=mon
You can use wget to download ESGF NetCDF files (``-x`` option to create directories)::
$ wget --certificate cert.pem --private-key cert.pem --ca-certificate cert.pem -N -x -P /path/to/esgf/cmip5/archive
.. _Copernicus: http://climate.copernicus.eu/
.. _ESGF: https://esgf.llnl.gov/
.. _PyWPS: http://pywps.org/
.. _NCL: http://www.ncl.ucar.edu/
.. _esgf-pyclient: http://esgf-pyclient.readthedocs.io/en/latest/index.html
.. _Twitcher: http://twitcher.readthedocs.io/en/latest/
.. _RestClient: http://birdhouse-workshop.readthedocs.io/en/latest/pywps/testing.html?highlight=rest#restclient-firefox-only
.. _logon example: http://esgf-pyclient.readthedocs.io/en/latest/examples.html
.. _requests: http://docs.python-requests.org/en/master/
.. _requests documentation: http://docs.python-requests.org/en/master/user/advanced/?highlight=ssl#client-side-certificates
.. _OWSLib: http://geopython.github.io/OWSLib/
.. _Birdy: http://birdy.readthedocs.io/en/latest/