RETIRED, Function as a Service for OpenStack
Go to file
Lingxian Kong 35e652f43a Fix job creation with pattern
Change-Id: I0222b853464f6bf3589f11477a0624d72c64a22b
2017-08-06 16:20:11 +12:00
doc/source Complete basic workflow(Finally!) 2017-05-12 12:00:19 +12:00
example/functions/python Support Keystone session in python runtime 2017-07-03 01:16:40 +12:00
qinling Fix job creation with pattern 2017-08-06 16:20:11 +12:00
releasenotes Initial commit for qinling project 2017-04-11 15:13:10 +12:00
runtimes/python2 Do not get openstack session in container 2017-08-06 15:02:49 +12:00
tools Fix relationship of function service mapping 2017-07-27 23:14:27 +12:00
.coveragerc Make api service work 2017-04-14 15:17:05 +12:00
.gitignore Fix file pattern in git ignore 2017-07-05 00:10:09 +12:00
.gitreview Make api service work 2017-04-14 15:17:05 +12:00
.mailmap Make api service work 2017-04-14 15:17:05 +12:00
.testr.conf Make api service work 2017-04-14 15:17:05 +12:00
CONTRIBUTING.rst Initial commit for qinling project 2017-04-11 15:13:10 +12:00
HACKING.rst Initial commit for qinling project 2017-04-11 15:13:10 +12:00
LICENSE Initial commit for qinling project 2017-04-11 15:13:10 +12:00
README.rst Fix a command in README 2017-08-05 23:46:00 +12:00
babel.cfg Initial commit for qinling project 2017-04-11 15:13:10 +12:00
requirements.txt Add jobs api 2017-07-21 22:52:05 +12:00
setup.cfg Complete basic workflow(Finally!) 2017-05-12 12:00:19 +12:00
setup.py Updated from global requirements 2017-07-04 15:00:21 +12:00
test-requirements.txt Updated from global requirements 2017-07-04 15:00:21 +12:00
tox.ini Add basic unit test structure 2017-07-04 14:34:53 +12:00

README.rst

Qinling

Note

Qinling (is pronounced "tʃinliŋ") refers to Qinling Mountains in southern Shaanxi Province in China. The mountains provide a natural boundary between North and South China and support a huge variety of plant and wildlife, some of which is found nowhere else on Earth.

Qinling is Function as a Service for OpenStack. This project aims to provide a platform to support serverless functions (like AWS Lambda). Qinling supports different container orchestration platforms (Kubernetes/Swarm, etc.) and different function package storage backends (local/Swift/S3) by nature using plugin mechanism.

Quick Start

Installation

A fast and simple way to try Qinling is to create a Vagrant VM including all related components and dependencies of Qinling service. For your convenience, Qinling team already provide a Vagrantfile in tools/vagrant folder.

Qinling is a FaaS implemented on top of container orchestration system such as Kubernetes, Swarm, etc. Particularly, Kubernetes is a reference backend considering its popularity. So, you need to setup Kubernetes first before installing Qinling. The easiest way to setup Kubernetes is to use Minikube, it runs a single-node Kubernetes cluster inside a VM alongside Qinling vagrant VM, so they can communicate with each other without any network configuration.

Note

In order to manage resources on Kubernetes, it is recommended to install kubectl command line tool.

Qinling can work with OpenStack Keystone for authentication, or it can work without authentication at all. By default, authentication is disabled, config auth_enable = True to enable authentication.

After Kubernetes installation, perform the following commands on your local host.

  1. Setup HTTP proxy to access the Kubernetes API:

    $ kubectl proxy --accept-hosts='.*' --address='0.0.0.0'
    
    Starting to serve on [::]:8001
  2. Clone Qinling repo and go to vagrant directory:

    $ git clone https://github.com/LingxianKong/qinling.git
    $ cd qinling/tools/vagrant
  3. Modify Qinling sample config file according to your own environment. Suppose your IP address of your local host is 192.168.200.50, default Kubernetes API HTTP proxy port is 8001, and Qinling vagrant VM IP address is 192.168.33.18 (the default value in Vagrantfile):

    $ sed -i 's/KUBERNETES_API_HOST/192.168.200.50/' qinling.conf.sample
    $ sed -i 's/KUBERNETES_API_PORT/8001/' qinling.conf.sample
    $ sed -i 's/QINLING_API_ADDRESS/192.168.33.18/' qinling.conf.sample
  4. Now, start Qinling vagrant VM:

    $ vagrant --version
    Vagrant 1.9.1
    $ vagrant up
    ...
    ==> default: INFO  [alembic.runtime.migration] Context impl MySQLImpl.
    ==> default: INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
    ==> default: INFO  [alembic.runtime.migration] Running upgrade  -> 001, Pike release

    If you see message like the above, congratulations!

Getting started with Qinling

Currently, you can interact with Qinling using python-qinlingclient or sending RESTful API directly. Both ways are described in this guide.

httpie is a convenient tool to send HTTP request, make sure you installed httpie via pip install httpie before playing with Qinling.

If you prefer to use CLI, please make sure python-qinlingclient is installed.

Perform following commands on your local host, the process will create runtime/function/execution in Qinling.

  1. (Optional) Prepare a docker image including development environment for a specific programming language. For your convenience, I already build one (lingxiankong/python-runtime) in my docker hub account that you could directly use to create runtime in Qinling. Only Python 2 runtime is supported for now, but it is very easy to add another program language support. If you indeed want to build a new image, run the following commands in qinling repo directory, replace DOCKER_USER with your own docker hub username:

    $ cd runtimes/python2
    $ docker build -t DOCKER_USER/python-runtime .
    $ docker push DOCKER_USER/python-runtime
  2. Create runtime. A runtime in Qinling is running environment for a specific language, this resource is supposed to be created/deleted/updated by cloud operator. After creation, check the runtime status is available:

    $ http POST http://192.168.33.18:7070/v1/runtimes name=python2.7 \
      image=DOCKER_USER/python-runtime
    
    HTTP/1.1 201 Created
    Connection: keep-alive
    Content-Length: 194
    Content-Type: application/json
    Date: Fri, 12 May 2017 04:37:08 GMT
    
    {
        "created_at": "2017-05-12 04:37:08.129860",
        "id": "c1d78623-56bf-4487-9a72-1299b2c55e65",
        "image": "DOCKER_USER/python-runtime",
        "name": "python2.7",
        "project_id": "default",
        "status": "creating"
    }
    
    $ http GET http://192.168.33.18:7070/v1/runtimes/c1d78623-56bf-4487-9a72-1299b2c55e65
    
    HTTP/1.1 200 OK
    Connection: keep-alive
    Content-Length: 246
    Content-Type: application/json
    Date: Fri, 12 May 2017 04:37:50 GMT
    
    {
        "created_at": "2017-05-12 04:37:08",
        "description": null,
        "id": "c1d78623-56bf-4487-9a72-1299b2c55e65",
        "image": "DOCKER_USER/python-runtime",
        "name": "python2.7",
        "project_id": "default",
        "status": "available",
        "updated_at": "2017-05-12 04:37:08"
    }

    Using CLI:

    $ openstack runtime create python2.7 DOCKER_USER/python-runtime
    +------------+--------------------------------------+
    | Field      | Value                                |
    +------------+--------------------------------------+
    | id         | c1d78623-56bf-4487-9a72-1299b2c55e65 |
    | name       | python2.7                            |
    | image      | DOCKER_USER/python-runtime           |
    | project_id | default                              |
    | status     | available                            |
    | created_at | 2017-05-12 04:37:08.129860           |
    | updated_at |                                      |
    +------------+--------------------------------------+
  3. Create a customized function package:

    $ mkdir ~/qinling_test
    $ cat <<EOF > ~/qinling_test/main.py
      import requests
      def main():
          r = requests.get('https://api.github.com/events')
          return len(r.json())
      if __name__ == '__main__':
          main()
      EOF
    $ pip install requests -t ~/qinling_test
    $ cd ~/qinling_test
    $ zip -r ~/qinling_test/qinling_test.zip ./*
  4. Create function, runtime_id comes from the output of above command:

    $ http -f POST http://192.168.33.18:7070/v1/functions name=github_test \
      runtime_id=c1d78623-56bf-4487-9a72-1299b2c55e65 \
      code='{"package": "true"}' \
      package@~/qinling_test/qinling_test.zip
    
    HTTP/1.1 201 Created
    Connection: keep-alive
    Content-Length: 234
    Content-Type: application/json
    Date: Fri, 12 May 2017 04:49:59 GMT
    
    {
        "code": {
            "package": "true"
        },
        "created_at": "2017-05-12 04:49:59.659345",
        "description": null,
        "entry": "main.main",
        "id": "352e4c02-3c6b-4860-9b85-f72344b1f986",
        "name": "github_test",
        "runtime_id": "c1d78623-56bf-4487-9a72-1299b2c55e65"
    }

    Using CLI:

    $ openstack function create github_test \
      c1d78623-56bf-4487-9a72-1299b2c55e65 \
      '{"source": "package"}' \
      --package ~/qinling_test/qinling_test.zip
    +------------+--------------------------------------+
    | Field      | Value                                |
    +------------+--------------------------------------+
    | id         | 352e4c02-3c6b-4860-9b85-f72344b1f986 |
    | name       | github_test                          |
    | count      | 0                                    |
    | code       | {u'source': u'package'}              |
    | runtime_id | c1d78623-56bf-4487-9a72-1299b2c55e65 |
    | entry      | main.main                            |
    | created_at | 2017-05-12 04:49:59.659345           |
    | updated_at |                                      |
    +------------+--------------------------------------+
  5. Invoke the function by specifying function_id:

    $ http POST http://192.168.33.18:7070/v1/executions \
      function_id=352e4c02-3c6b-4860-9b85-f72344b1f986
    
    HTTP/1.1 201 Created
    Connection: keep-alive
    Content-Length: 255
    Content-Type: application/json
    Date: Thu, 11 May 2017 23:46:12 GMT
    
    {
        "created_at": "2017-05-12 04:51:10",
        "function_id": "352e4c02-3c6b-4860-9b85-f72344b1f986",
        "id": "80cd55be-d369-49b8-8bd5-e0bfc1d20d25",
        "input": null,
        "output": "{\"result\": 30}",
        "status": "success",
        "sync": true,
        "updated_at": "2017-05-12 04:51:23"
    }

    Using CLI:

    $ openstack function execution create 352e4c02-3c6b-4860-9b85-f72344b1f986
    +-------------+------------------------------------------------------------+
    | Field       | Value                                                      |
    +-------------+------------------------------------------------------------+
    | id          | 80cd55be-d369-49b8-8bd5-e0bfc1d20d25                       |
    | function_id | 352e4c02-3c6b-4860-9b85-f72344b1f986                       |
    | input       | {}                                                         |
    | output      | {"result": {"duration": 1.2511260509490967, "output": 30}} |
    | status      | success                                                    |
    | sync        | True                                                       |
    | created_at  | 2017-05-12 04:51:10                                        |
    | updated_at  | 2017-05-12 04:51:23                                        |
    +-------------+------------------------------------------------------------+

    If you invoke the same function again, you will find it is much faster thanks to Qinling cache mechanism.