Add some documentation for Qinling
- Add README.rst - Add Vagrantfile for Qinling trial. - Rename python2.7 dir to python2
This commit is contained in:
parent
df32a9f412
commit
6c71504654
233
README.rst
233
README.rst
|
@ -1,19 +1,236 @@
|
||||||
=======
|
=======
|
||||||
qinling
|
Qinling
|
||||||
=======
|
=======
|
||||||
|
|
||||||
Function as a Service (Documentation needs to be added, please stay tuned!)
|
.. note::
|
||||||
|
|
||||||
Please fill here a long description which must be at least 3 lines wrapped on
|
Qinling refers to Qinling Mountains in southern Shaanxi Province in China.
|
||||||
80 cols, so that distribution package maintainers can use it in their packages.
|
The mountains provide a natural boundary between North and South China and
|
||||||
Note that this is a hard requirement.
|
support a huge variety of plant and wildlife, some of which is found
|
||||||
|
nowhere else on Earth.
|
||||||
|
|
||||||
|
Qinling is Function as a Service for OpenStack. This project aims to provide a
|
||||||
|
platform to support serverless functions (like AWS Lambda). Qinling supports
|
||||||
|
different container orchestration platforms (Kubernetes/Swarm, etc.) and
|
||||||
|
different function package storage backends (local/Swift/S3) by nature using
|
||||||
|
plugin mechanism.
|
||||||
|
|
||||||
* Free software: Apache license
|
* Free software: Apache license
|
||||||
* Documentation: http://docs.openstack.org/developer/qinling
|
* Documentation: http://docs.openstack.org/developer/qinling
|
||||||
* Source: http://git.openstack.org/cgit/openstack/qinling
|
* Source: http://git.openstack.org/cgit/openstack/qinling
|
||||||
|
* Features: https://blueprints.launchpad.net/qinling
|
||||||
* Bugs: http://bugs.launchpad.net/qinling
|
* Bugs: http://bugs.launchpad.net/qinling
|
||||||
|
|
||||||
Features
|
Quick Start
|
||||||
--------
|
~~~~~~~~~~~
|
||||||
|
|
||||||
* TODO
|
Installation
|
||||||
|
------------
|
||||||
|
|
||||||
|
A fast and simple way to try Qinling is to create a Vagrant VM including all
|
||||||
|
related components and dependencies of Qinling service. For your convenience,
|
||||||
|
Qinling team already provide a Vagrantfile in ``tools/vagrant`` folder.
|
||||||
|
|
||||||
|
Qinling is a FaaS implemented on top of container orchestration system such as
|
||||||
|
Kubernetes, Swarm, etc. Particularly, Kubernetes is a reference backend
|
||||||
|
considering its popularity. So, you need to setup Kubernetes first before
|
||||||
|
installing Qinling. The easiest way to setup Kubernetes is to use `Minikube
|
||||||
|
<https://kubernetes.io/docs/getting-started-guides/minikube/>`_, it runs a
|
||||||
|
single-node Kubernetes cluster inside a VM alongside Qinling vagrant VM, so
|
||||||
|
they can communicate with each other without any network configuration.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
In order to manage resources on Kubernetes, it is recommended to install
|
||||||
|
`kubectl <https://kubernetes.io/docs/tasks/tools/install-kubectl/>`_
|
||||||
|
command line tool.
|
||||||
|
|
||||||
|
After Kubernetes installation, perform the following commands on your local
|
||||||
|
host.
|
||||||
|
|
||||||
|
#. Setup HTTP proxy to access the Kubernetes API:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ kubectl proxy --accept-hosts='.*' --address='0.0.0.0'
|
||||||
|
|
||||||
|
Starting to serve on [::]:8001
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
#. Clone Qinling repo and go to ``vagrant`` directory:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ git clone https://github.com/LingxianKong/qinling.git
|
||||||
|
$ cd qinling/tools/vagrant
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
#. Modify Qinling sample config file according to your own environment. Suppose
|
||||||
|
your IP address of your local host is ``192.168.200.50``, default Kubernetes
|
||||||
|
API HTTP proxy port is ``8001``, and Qinling vagrant VM IP address is
|
||||||
|
``192.168.33.18`` (the default value in ``Vagrantfile``):
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ sed -i 's/KUBERNETES_API_HOST/192.168.200.50/' qinling.conf.sample
|
||||||
|
$ sed -i 's/KUBERNETES_API_PORT/8001/' qinling.conf.sample
|
||||||
|
$ sed -i 's/QINLING_API_ADDRESS/192.168.33.18/' qinling.conf.sample
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
#. Now, start Qinling vagrant VM:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ vagrant up
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
Getting started with Qinling
|
||||||
|
----------------------------
|
||||||
|
|
||||||
|
**Currently, RESTful API is the only way to interact with Qinling,
|
||||||
|
python-qinlingclient is still under development.**
|
||||||
|
|
||||||
|
``httpie`` is a convenient tool to send HTTP request, make sure you installed
|
||||||
|
``httpie`` via ``pip install httpie`` before playing with Qinling.
|
||||||
|
|
||||||
|
Perform following commands on your local host, the process will create
|
||||||
|
runtime/function/execution in Qinling.
|
||||||
|
|
||||||
|
#. First, build a docker image that is used to create runtime in Qinling and
|
||||||
|
upload to docker hub. Only ``Python 2`` runtime is supported for now, but it
|
||||||
|
is very easy to add another program language support. Run the commands in
|
||||||
|
``qinling`` repo directory, replace ``DOCKER_USER`` with your docker hub
|
||||||
|
username:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ cd runtimes/python2
|
||||||
|
$ docker build -t DOCKER_USER/python-runtime .
|
||||||
|
$ docker push DOCKER_USER/python-runtime
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
#. Create runtime. A runtime in Qinling is running environment for a specific
|
||||||
|
language, this resource is supposed to be created/deleted/updated by cloud
|
||||||
|
operator. After creation, check the runtime status is ``available``:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ http POST http://192.168.33.18:7070/v1/runtimes name=python2.7 \
|
||||||
|
image=DOCKER_USER/python-runtime
|
||||||
|
|
||||||
|
HTTP/1.1 201 Created
|
||||||
|
Connection: keep-alive
|
||||||
|
Content-Length: 194
|
||||||
|
Content-Type: application/json
|
||||||
|
Date: Fri, 12 May 2017 04:37:08 GMT
|
||||||
|
|
||||||
|
{
|
||||||
|
"created_at": "2017-05-12 04:37:08.129860",
|
||||||
|
"id": "c1d78623-56bf-4487-9a72-1299b2c55e65",
|
||||||
|
"image": "DOCKER_USER/python-runtime",
|
||||||
|
"name": "python2.7",
|
||||||
|
"project_id": "default",
|
||||||
|
"status": "creating"
|
||||||
|
}
|
||||||
|
|
||||||
|
$ http GET http://192.168.33.18:7070/v1/runtimes/c1d78623-56bf-4487-9a72-1299b2c55e65
|
||||||
|
|
||||||
|
HTTP/1.1 200 OK
|
||||||
|
Connection: keep-alive
|
||||||
|
Content-Length: 246
|
||||||
|
Content-Type: application/json
|
||||||
|
Date: Fri, 12 May 2017 04:37:50 GMT
|
||||||
|
|
||||||
|
{
|
||||||
|
"created_at": "2017-05-12 04:37:08",
|
||||||
|
"description": null,
|
||||||
|
"id": "c1d78623-56bf-4487-9a72-1299b2c55e65",
|
||||||
|
"image": "DOCKER_USER/python-runtime",
|
||||||
|
"name": "python2.7",
|
||||||
|
"project_id": "default",
|
||||||
|
"status": "available",
|
||||||
|
"updated_at": "2017-05-12 04:37:08"
|
||||||
|
}
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
#. Create a customized function package:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ mkdir ~/qinling_test
|
||||||
|
$ cat <<EOF > ~/qinling_test/main.py
|
||||||
|
import requests
|
||||||
|
def main():
|
||||||
|
r = requests.get('https://api.github.com/events')
|
||||||
|
return len(r.json())
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
|
EOF
|
||||||
|
$ pip install requests -t ~/qinling_test
|
||||||
|
$ zip ~/qinling_test/qinling_test.zip ~/qinling_test/*
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
#. Create function, ``runtime_id`` comes from the output of above command:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ http -f POST http://192.168.33.18:7070/v1/functions name=github_test \
|
||||||
|
runtime_id=c1d78623-56bf-4487-9a72-1299b2c55e65 \
|
||||||
|
code='{"package": "true"}' \
|
||||||
|
package@~/qinling_test/qinling_test.zip
|
||||||
|
|
||||||
|
HTTP/1.1 201 Created
|
||||||
|
Connection: keep-alive
|
||||||
|
Content-Length: 234
|
||||||
|
Content-Type: application/json
|
||||||
|
Date: Fri, 12 May 2017 04:49:59 GMT
|
||||||
|
|
||||||
|
{
|
||||||
|
"code": {
|
||||||
|
"package": "true"
|
||||||
|
},
|
||||||
|
"created_at": "2017-05-12 04:49:59.659345",
|
||||||
|
"description": null,
|
||||||
|
"entry": "main",
|
||||||
|
"id": "352e4c02-3c6b-4860-9b85-f72344b1f986",
|
||||||
|
"name": "github_test",
|
||||||
|
"runtime_id": "c1d78623-56bf-4487-9a72-1299b2c55e65"
|
||||||
|
}
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
#. Invoke the function by specifying ``function_id``:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ http POST http://192.168.33.18:7070/v1/executions \
|
||||||
|
function_id=352e4c02-3c6b-4860-9b85-f72344b1f986
|
||||||
|
|
||||||
|
HTTP/1.1 201 Created
|
||||||
|
Connection: keep-alive
|
||||||
|
Content-Length: 255
|
||||||
|
Content-Type: application/json
|
||||||
|
Date: Thu, 11 May 2017 23:46:12 GMT
|
||||||
|
|
||||||
|
{
|
||||||
|
"created_at": "2017-05-12 04:51:10",
|
||||||
|
"function_id": "352e4c02-3c6b-4860-9b85-f72344b1f986",
|
||||||
|
"id": "80cd55be-d369-49b8-8bd5-e0bfc1d20d25",
|
||||||
|
"input": null,
|
||||||
|
"output": "{\"result\": 30}",
|
||||||
|
"status": "success",
|
||||||
|
"sync": true,
|
||||||
|
"updated_at": "2017-05-12 04:51:23"
|
||||||
|
}
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
If you invoke the same function again, you will find it is much faster
|
||||||
|
thanks to Qinling cache mechanism.
|
||||||
|
|
|
@ -1,31 +0,0 @@
|
||||||
# Qinling: Python Environment
|
|
||||||
|
|
||||||
This is the Python environment for Qinling.
|
|
||||||
|
|
||||||
It's a Docker image containing a Python 2.7 runtime, along with a
|
|
||||||
dynamic loader. A few common dependencies are included in the
|
|
||||||
requirements.txt file.
|
|
||||||
|
|
||||||
## Customizing this image
|
|
||||||
|
|
||||||
To add package dependencies, edit requirements.txt to add what you
|
|
||||||
need, and rebuild this image (instructions below).
|
|
||||||
|
|
||||||
You also may want to customize what's available to the function in its
|
|
||||||
request context. You can do this by editing server.py (see the
|
|
||||||
comment in that file about customizing request context).
|
|
||||||
|
|
||||||
## Rebuilding and pushing the image
|
|
||||||
|
|
||||||
You'll need access to a Docker registry to push the image: you can
|
|
||||||
sign up for Docker hub at hub.docker.com, or use registries from
|
|
||||||
gcr.io, quay.io, etc. Let's assume you're using a docker hub account
|
|
||||||
called USER. Build and push the image to the the registry:
|
|
||||||
|
|
||||||
```
|
|
||||||
docker build -t USER/python-env . && docker push USER/python-env
|
|
||||||
```
|
|
||||||
|
|
||||||
## Using the image in Qinling
|
|
||||||
|
|
||||||
TBD
|
|
|
@ -0,0 +1,22 @@
|
||||||
|
# Qinling: Python Environment
|
||||||
|
|
||||||
|
This is the Python environment for Qinling.
|
||||||
|
|
||||||
|
It's a Docker image containing a Python 2.7 runtime, along with a
|
||||||
|
dynamic loader. A few common dependencies are included in the
|
||||||
|
requirements.txt file. End users need to provide their own dependencies
|
||||||
|
in their function packages through Qinling API or CLI.
|
||||||
|
|
||||||
|
## Rebuilding and pushing the image
|
||||||
|
|
||||||
|
You'll need access to a Docker registry to push the image, by default it's
|
||||||
|
docker hub. After modification, build a new image and upload to docker hub:
|
||||||
|
|
||||||
|
docker build -t USER/python-runtime. && docker push USER/python-runtime
|
||||||
|
|
||||||
|
|
||||||
|
## Using the image in Qinling
|
||||||
|
|
||||||
|
After the image is ready in docker hub, create a runtime in Qinling:
|
||||||
|
|
||||||
|
http POST http://127.0.0.1:7070/v1/runtimes name=python2.7 image=USER/python-runtime
|
|
@ -0,0 +1,71 @@
|
||||||
|
# -*- mode: ruby -*-
|
||||||
|
# vi: set ft=ruby :
|
||||||
|
|
||||||
|
VAGRANTFILE_API_VERSION = "2"
|
||||||
|
|
||||||
|
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
|
||||||
|
config.vm.box = "ubuntu/xenial64"
|
||||||
|
config.vm.hostname = "qinling"
|
||||||
|
|
||||||
|
config.vm.network "private_network", ip: "192.168.33.18"
|
||||||
|
config.vm.network "forwarded_port", guest: 7070, host: 7070
|
||||||
|
|
||||||
|
config.vm.provider "virtualbox" do |vb|
|
||||||
|
vb.customize ["modifyvm", :id, "--memory", "1024"]
|
||||||
|
vb.customize ["modifyvm", :id, "--cpus", "1"]
|
||||||
|
vb.gui = false
|
||||||
|
end
|
||||||
|
|
||||||
|
config.vm.provision "shell", privileged: false, inline: <<-SHELL
|
||||||
|
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get -y upgrade
|
||||||
|
sudo apt-get -y install python-dev python-setuptools libffi-dev \
|
||||||
|
libxslt1-dev libxml2-dev libyaml-dev libssl-dev rabbitmq-server git
|
||||||
|
|
||||||
|
# Install mysql and initialize database.
|
||||||
|
echo mysql-server-5.5 mysql-server/root_password password password | sudo debconf-set-selections
|
||||||
|
echo mysql-server-5.5 mysql-server/root_password_again password password | sudo debconf-set-selections
|
||||||
|
echo mysql-server-5.5 mysql-server/start_on_boot boolean true | sudo debconf-set-selections
|
||||||
|
|
||||||
|
sudo apt-get -y install mysql-server python-mysqldb
|
||||||
|
sudo sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf
|
||||||
|
sudo sed -i '44 i skip-name-resolve' /etc/mysql/my.cnf
|
||||||
|
sudo service mysql restart
|
||||||
|
|
||||||
|
HOSTNAME="127.0.0.1"
|
||||||
|
PORT="3306"
|
||||||
|
USERNAME="root"
|
||||||
|
PASSWORD="password"
|
||||||
|
DBNAME="qinling"
|
||||||
|
create_db_sql="create database IF NOT EXISTS ${DBNAME}"
|
||||||
|
mysql -h${HOSTNAME} -P${PORT} -u${USERNAME} -p${PASSWORD} -e "${create_db_sql}"
|
||||||
|
|
||||||
|
# Change rabbitmq credential.
|
||||||
|
sudo rabbitmqctl change_password guest password
|
||||||
|
|
||||||
|
# Install pip.
|
||||||
|
curl -O https://bootstrap.pypa.io/get-pip.py && sudo python get-pip.py
|
||||||
|
sudo pip install httpie
|
||||||
|
|
||||||
|
# Install Qinling.
|
||||||
|
git clone https://github.com/LingxianKong/qinling.git
|
||||||
|
cd qinling
|
||||||
|
sudo pip install -e .
|
||||||
|
|
||||||
|
# Initialize Qinling configuration.
|
||||||
|
sudo mkdir -p /vagrant/etc/qinling
|
||||||
|
sudo mkdir -p /vagrant/log
|
||||||
|
sudo mkdir -p /opt/qinling/funtion
|
||||||
|
sudo chown ubuntu:ubuntu /opt/qinling/funtion
|
||||||
|
cp /vagrant/qinling.conf.sample /vagrant/etc/qinling/qinling.conf
|
||||||
|
|
||||||
|
# Qinling db migration.
|
||||||
|
qinling-db-manage --config-file /vagrant/etc/qinling/qinling.conf upgrade head
|
||||||
|
|
||||||
|
# Start Qinling service.
|
||||||
|
python qinling/cmd/launch.py --server api,engine --config-file /vagrant/etc/qinling/qinling.conf &
|
||||||
|
|
||||||
|
SHELL
|
||||||
|
end
|
|
@ -0,0 +1,23 @@
|
||||||
|
[DEFAULT]
|
||||||
|
debug=True
|
||||||
|
verbose=False
|
||||||
|
log_file=/vagrant/log/qinling.log
|
||||||
|
logging_default_format_string=%(asctime)s.%(msecs)03d %(process)d %(levelname)s %(instance)s%(message)s (%(name)s) [-]
|
||||||
|
logging_context_format_string=%(asctime)s.%(msecs)03d %(process)d %(levelname)s %(instance)s%(message)s (%(name)s) [%(request_id)s %(user_identity)s]
|
||||||
|
logging_user_identity_format=%(user)s %(tenant)s
|
||||||
|
|
||||||
|
[api]
|
||||||
|
api_workers=1
|
||||||
|
|
||||||
|
[database]
|
||||||
|
connection=mysql://root:password@localhost:3306/qinling
|
||||||
|
|
||||||
|
[oslo_messaging_rabbit]
|
||||||
|
rabbit_password=password
|
||||||
|
|
||||||
|
[pecan]
|
||||||
|
auth_enable = false
|
||||||
|
|
||||||
|
[kubernetes]
|
||||||
|
kube_host = KUBERNETES_API_HOST:KUBERNETES_API_PORT
|
||||||
|
qinling_service_address = QINLING_API_ADDRESS
|
Loading…
Reference in New Issue