- Create runtime - Create function - Invoke function(i.e. create execution)changes/17/465917/1
@ -1,12 +1,175 @@ | |||
============ | |||
Qinling Installation Guide | |||
========================== | |||
Prerequisites | |||
------------- | |||
It is necessary to install some specific system libs for installing Qinling. | |||
They can be installed on most popular operating system using their package | |||
manager (for Ubuntu - *apt*, for Fedora - *dnf*, CentOS - *yum*, for Mac OS - | |||
*brew* or *macports*). | |||
The list of needed packages is shown below: | |||
1. **python-dev** | |||
2. **python-setuptools** | |||
3. **python-pip** | |||
4. **libffi-dev** | |||
5. **libxslt1-dev (or libxslt-dev)** | |||
6. **libxml2-dev** | |||
7. **libyaml-dev** | |||
8. **libssl-dev** | |||
In case of Ubuntu, just run:: | |||
$ apt-get install -y python-dev python-setuptools python-pip libffi-dev libxslt1-dev \ | |||
libxml2-dev libyaml-dev libssl-dev | |||
**NOTE:** **Qinling can be used without authentication at all or it can work | |||
with OpenStack.** In case of OpenStack, it works **only on Keystone v3**, make | |||
sure **Keystone v3** is installed. | |||
Installation | |||
============ | |||
------------ | |||
First of all, clone the repo and go to the repo directory:: | |||
$ git clone https://github.com/openstack/qinling.git | |||
$ cd qinling | |||
Generate config:: | |||
$ tox -egenconfig | |||
Configure Qinling as needed. The configuration file is located in | |||
``etc/qinling.conf.sample``. You will need to modify the configuration options | |||
and then copy it into ``/etc/qinling/qinling.conf``. | |||
For details see :doc:`Qinling Configuration Guide </guides/configuration_guide>` | |||
**Virtualenv installation**:: | |||
$ tox | |||
This will install necessary virtual environments and run all the project tests. | |||
Installing virtual environments may take significant time (~10-15 mins). | |||
**Local installation**:: | |||
$ pip install -e . | |||
or:: | |||
$ pip install -r requirements.txt | |||
$ python setup.py install | |||
**NOTE**: Differences *pip install -e* and *setup.py install*. **pip install -e** | |||
works very similarly to **setup.py install** or the EasyInstall tool, except | |||
that it doesnโt actually install anything. Instead, it creates a special | |||
.egg-link file in the deployment directory, that links to your projectโs | |||
source code. | |||
Before the first run | |||
-------------------- | |||
After installation you will see **qinling-server** and **qinling-db-manage** commands | |||
in your environment, either in system or virtual environment. | |||
**NOTE**: In case of using **virtualenv**, all Qinling related commands available via | |||
**tox -evenv --**. For example, *qinling-server* is available via | |||
*tox -evenv -- qinling-server*. | |||
**qinling-db-manage** command can be used for migrations. | |||
For updating the database to the latest revision type:: | |||
$ qinling-db-manage --config-file <path-to-qinling.conf> upgrade head | |||
Before starting Qinling server, run *qinling-db-manage populate* command. | |||
It prepares the DB, creates in it with all standard actions and standard | |||
workflows which Qinling provides for all Qinling users.:: | |||
$ qinling-db-manage --config-file <path-to-qinling.conf> populate | |||
For more detailed information about *qinling-db-manage* script please see :doc:`Qinling Upgrade Guide </guides/upgrade_guide>`. | |||
**NOTE**: For users who want a dry run with **SQLite** database backend(not | |||
used in production), *qinling-db-manage* is not recommended for database | |||
initialization because of `SQLite limitations <http://www.sqlite.org/omitted.html>`_. | |||
Please use sync_db script described below instead for database initialization. | |||
**If you use virtualenv**:: | |||
$ tools/sync_db.sh --config-file <path-to-qinling.conf> | |||
**Or run sync_db directly**:: | |||
$ python tools/sync_db.py --config-file <path-to-qinling.conf> | |||
Running Qinling API server | |||
-------------------------- | |||
To run Qinling API server perform the following command in a shell:: | |||
$ qinling-server --server api --config-file <path-to-qinling.conf> | |||
Running Qinling Engines | |||
----------------------- | |||
To run Qinling Engine perform the following command in a shell:: | |||
$ qinling-server --server engine --config-file <path-to-qinling.conf> | |||
Running Qinling Task Executors | |||
------------------------------ | |||
To run Qinling Task Executor instance perform the following command in a shell:: | |||
$ qinling-server --server executor --config-file <path-to-qinling.conf> | |||
Note that at least one Engine instance and one Executor instance should be | |||
running so that workflow tasks are processed by Qinling. | |||
Running Multiple Qinling Servers Under the Same Process | |||
------------------------------------------------------- | |||
To run more than one server (API, Engine, or Task Executor) on the same process, | |||
perform the following command in a shell:: | |||
$ qinling-server --server api,engine --config-file <path-to-qinling.conf> | |||
The --server command line option can be a comma delimited list. The valid | |||
options are "all" (by default if not specified) or any combination of "api", | |||
"engine", and "executor". It's important to note that the "fake" transport for | |||
the rpc_backend defined in the config file should only be used if "all" the | |||
Qinling servers are launched on the same process. Otherwise, messages do not | |||
get delivered if the Qinling servers are launched on different processes | |||
because the "fake" transport is using an in process queue. | |||
Qinling And Docker | |||
------------------ | |||
Please first refer `installation steps for docker <https://docs.docker.com/installation/>`_. | |||
To build the image from the qinling source, change directory to the root | |||
directory of the Qinling git repository and run:: | |||
$ docker build -t <Name of image> . | |||
In case you want pre-built image, you can download it from `openstack tarballs source <https://tarballs.openstack.org/qinling/images/qinling-docker.tar.gz>`_. | |||
To load this image to docker registry, please run following command:: | |||
$ docker load -i '<path of qinling-docker.tar.gz>' | |||
The Qinling Docker image is configured to store the database in the user's home | |||
directory. For persistence of these data, you may want to keep this directory | |||
outside of the container. This may be done by the following steps:: | |||
$ sudo mkdir '<user-defined-directory>' | |||
$ docker run -it -v '<user-defined-directory>':/home/qinling <Name of image> | |||
At the command line:: | |||
More about docker: https://www.docker.com/ | |||
$ pip install qinling | |||
**NOTE:** This docker image uses **SQLite** database. So, it cannot be used for | |||
production environment. If you want to use this for production environment, | |||
then put customized qinling.conf to '<user-defined-directory>'. | |||
Or, if you have virtualenvwrapper installed:: | |||
Qinling Client Installation | |||
--------------------------- | |||
$ mkvirtualenv qinling | |||
$ pip install qinling | |||
Please refer to :doc:`Qinling Client / CLI Guide </guides/mistralclient_guide>` |
@ -0,0 +1,106 @@ | |||
# Copyright 2017 Catalyst IT Limited | |||
# | |||
# Licensed under the Apache License, Version 2.0 (the "License"); | |||
# you may not use this file except in compliance with the License. | |||
# You may obtain a copy of the License at | |||
# | |||
# http://www.apache.org/licenses/LICENSE-2.0 | |||
# | |||
# Unless required by applicable law or agreed to in writing, software | |||
# distributed under the License is distributed on an "AS IS" BASIS, | |||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |||
# See the License for the specific language governing permissions and | |||
# limitations under the License. | |||
from oslo_log import log as logging | |||
from pecan import rest | |||
import requests | |||
import wsmeext.pecan as wsme_pecan | |||
from qinling.api.controllers.v1 import resources | |||
from qinling.api.controllers.v1 import types | |||
from qinling.db import api as db_api | |||
from qinling import exceptions as exc | |||
from qinling import rpc | |||
from qinling.utils import rest_utils | |||
LOG = logging.getLogger(__name__) | |||
class ExecutionsController(rest.RestController): | |||
def __init__(self, *args, **kwargs): | |||
self.engine_client = rpc.get_engine_client() | |||
super(ExecutionsController, self).__init__(*args, **kwargs) | |||
@rest_utils.wrap_wsme_controller_exception | |||
@wsme_pecan.wsexpose( | |||
resources.Execution, | |||
body=resources.Execution, | |||
status_code=201 | |||
) | |||
def post(self, execution): | |||
params = execution.to_dict() | |||
LOG.info("Creating execution. [execution=%s]", params) | |||
function_id = params['function_id'] | |||
# Check if the service url is existing. | |||
try: | |||
mapping = db_api.get_function_service_mapping(function_id) | |||
LOG.debug('Found Service url for function: %s', function_id) | |||
func_url = '%s/execute' % mapping.service_url | |||
LOG.info('Invoke function %s, url: %s', function_id, func_url) | |||
r = requests.post(func_url, data=params.get('input')) | |||
params.update( | |||
{'status': 'success', 'output': {'result': r.json()}} | |||
) | |||
db_model = db_api.create_execution(params) | |||
return resources.Execution.from_dict(db_model.to_dict()) | |||
except exc.DBEntityNotFoundError: | |||
pass | |||
func = db_api.get_function(function_id) | |||
runtime_id = func.runtime_id | |||
params.update({'status': 'running'}) | |||
db_model = db_api.create_execution(params) | |||
self.engine_client.create_execution( | |||
db_model.id, function_id, runtime_id, input=params.get('input') | |||
) | |||
updated_db = db_api.get_execution(db_model.id) | |||
return resources.Execution.from_dict(updated_db.to_dict()) | |||
@rest_utils.wrap_wsme_controller_exception | |||
@wsme_pecan.wsexpose(resources.Executions) | |||
def get_all(self): | |||
LOG.info("Get all executions.") | |||
executions = [resources.Execution.from_dict(db_model.to_dict()) | |||
for db_model in db_api.get_executions()] | |||
return resources.Executions(executions=executions) | |||
@rest_utils.wrap_wsme_controller_exception | |||
@wsme_pecan.wsexpose(resources.Execution, types.uuid) | |||
def get(self, id): | |||
LOG.info("Fetch execution [id=%s]", id) | |||
execution_db = db_api.get_execution(id) | |||
return resources.Execution.from_dict(execution_db.to_dict()) | |||
@rest_utils.wrap_wsme_controller_exception | |||
@wsme_pecan.wsexpose(None, types.uuid, status_code=204) | |||
def delete(self, id): | |||
"""Delete the specified Execution.""" | |||
LOG.info("Delete execution [id=%s]", id) | |||
return db_api.delete_execution(id) |
@ -0,0 +1,17 @@ | |||
apiVersion: v1 | |||
kind: Service | |||
metadata: | |||
name: {{ service_name }} | |||
labels: | |||
{% for key, value in labels.items() %} | |||
{{ key }}: {{ value }} | |||
{% endfor %} | |||
spec: | |||
type: NodePort | |||
selector: | |||
{% for key, value in selector.items() %} | |||
{{ key}}: "{{ value }}" | |||
{% endfor %} | |||
ports: | |||
- protocol: TCP | |||
port: 9090 |
@ -0,0 +1,13 @@ | |||
FROM alpine:3.5 | |||
RUN apk update | |||
RUN apk add --no-cache python2 python2-dev build-base py2-pip | |||
RUN pip install --upgrade pip | |||
RUN rm -r /root/.cache | |||
COPY . /app | |||
WORKDIR /app | |||
RUN pip install -r requirements.txt | |||
ENTRYPOINT ["python"] | |||
CMD ["server.py"] |
@ -0,0 +1,31 @@ | |||
# Qinling: Python Environment | |||
This is the Python environment for Qinling. | |||
It's a Docker image containing a Python 2.7 runtime, along with a | |||
dynamic loader. A few common dependencies are included in the | |||
requirements.txt file. | |||
## Customizing this image | |||
To add package dependencies, edit requirements.txt to add what you | |||
need, and rebuild this image (instructions below). | |||
You also may want to customize what's available to the function in its | |||
request context. You can do this by editing server.py (see the | |||
comment in that file about customizing request context). | |||
## Rebuilding and pushing the image | |||
You'll need access to a Docker registry to push the image: you can | |||
sign up for Docker hub at hub.docker.com, or use registries from | |||
gcr.io, quay.io, etc. Let's assume you're using a docker hub account | |||
called USER. Build and push the image to the the registry: | |||
``` | |||
docker build -t USER/python-env . && docker push USER/python-env | |||
``` | |||
## Using the image in Qinling | |||
TBD |
@ -0,0 +1,3 @@ | |||
Flask>=0.11.1 | |||
httplib2 | |||
requests>=2.7.0 |
@ -0,0 +1,91 @@ | |||
# Copyright 2017 Catalyst IT Limited | |||
# | |||
# Licensed under the Apache License, Version 2.0 (the "License"); | |||
# you may not use this file except in compliance with the License. | |||
# You may obtain a copy of the License at | |||
# | |||
# http://www.apache.org/licenses/LICENSE-2.0 | |||
# | |||
# Unless required by applicable law or agreed to in writing, software | |||
# distributed under the License is distributed on an "AS IS" BASIS, | |||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |||
# See the License for the specific language governing permissions and | |||
# limitations under the License. | |||
import json | |||