Project LaOS aka Lambdas-on-OpenStack

Mission

Provide capabilities to run software in "serverless" way.

Serverless

Serverless is a new paradigm in computing that enables simplicity, efficiency and scalability for both developers and operators. It's important to distinguish the two, because the benefits differ:

Benefits for developers

The main benefits that most people refer to are on the developer side and they include:

• No servers to manage (serverless) -- you just upload your code and the platform deals with the infrastructure
• Super simple coding -- no more monoliths! Just simple little bits of code
• Pay by the milliseconds your code is executing -- unlike a typical application that runs 24/7, and you're paying 24/7, functions only run when needed

Benefits for operators

If you will be operating IronFunctions (the person who has to manage the servers behind the serverless), then the benefits are different, but related.

• Extremely efficient use of resources
  • Unlike an app/API/microservice that consumes resources 24/7 whether they are in use or not, functions are time sliced across your infrastructure and only consume resources while they are actually doing something
• Easy to manage and scale
  • Single system for code written in any language or any technology
  • Single system to monitor
  • Scaling is the same for all functions, you don't scale each app independently
  • Scaling is simply adding more IronFunctions nodes

System requirements

• Operating system: Linux/MacOS
• Python version: 3.5 or greater
• Database: MySQL 5.7 or greater

Quick-start guide

Install DevStack with IronFunctions enabled. Pull down Project LaOS sources.

Create Python3.5 virtualenv:

$ virtualenv -p python3.5 .venv
$ source .venv/bin/activate

Install dependencies:

$ pip install -r requirements.txt -r test-requirements.txt

Install LaOS itself:

$ pip install -e .

Install MySQL if you haven't already, and create a new database for functions.

$ mysql -uroot -p -e "CREATE DATABASE functions"

Migrations

Once all dependencies are installed it is necessary to run database migrations. Before that it is necessary to set env variable:

export LAOS_MIGRATIONS_DB=mysql+pymysql://root:root@localhost/functions

In this section please specify connection URI to your own MySQL database. Once the file is saved, just use alembic to apply the migrations:

$ alembic upgrade head

Starting a server

Once it is finished you will have a console script laos-api:

$ laos-api --help

Usage: laos-api [OPTIONS]

  Starts an Project Laos API service

Options:
  --host TEXT                    API service bind host.
  --port INTEGER                 API service bind port.
  --db-uri TEXT                  LaOS persistence storage URI.
  --keystone-endpoint TEXT       OpenStack Identity service endpoint.
  --functions-url TEXT           IronFunctions API URL
  --log-level TEXT               Logging file
  --log-file TEXT                Log file path
  --help                         Show this message and exit.

Minimum required options to start LaOS API service:

 --db-uri mysql://root:root@192.168.0.112/functions
 --keystone-endpoint http://192.168.0.112:5000/v3
 --functions-url http://192.168.0.112:8080/v1
 --log-level INFO

Creating and running LaOS inside Docker container

As part of regular Python distribution, LaOS also has its own Docker container to run. There are two options:

• run from sources
• run from Docker Hub

In order to build container from sources run following commands:

export DOCKER_HOST=tcp://<docker-host>:<docker-port>
docker build -t laos-api -f Dockerfile .

After that it is required to create correct version of Dockerfile.env. It container all required options to start LaOS API service properly. Once it is done run following commands:

docker run -d -p 10001:10001 --env-file Dockerfile.env laos-api

Navigate to your web browser to check if service is running:

<docker-host>:10001/api

or using CLI

curl -X GET http://<docker-host>:10001/api/swagger.json | python -mjson.tool

Examining API

In examples folder you can find a script that examines available API endpoints, but this script relays on:

• LAOS_API_URL - Project LaOS API endpoint
• OS_AUTH_URL - OpenStack Auth URL
• OS_PROJECT_ID - it can be found in OpenStack Dashboard or in CLI
• OS_USERNAME - OpenStack project-aligned username
• OS_PASSWORD - OpenStack project-aligned user password
• OS_DOMAIN - OpenStack project domain name
• OS_PROJECT_NAME - OpenStack project name

Then just run script:

OS_AUTH_URL=http://192.168.0.112:5000/v3 OS_PROJECT_ID=8fb76785313a4500ac5367eb44a31677 OS_USERNAME=admin OS_PASSWORD=root OS_DOMAIN=default OS_PROJECT_NAME=admin ./examples/hello-lambda.sh

Please note, that given values are project-specific, so they can't be reused.

API docs

As part of LaOS ReST API it is possible to discover API doc using Swagger Doc. Once server is launched you can navigate to:

http://<laos-host>:<laos-port>/api

to see recent API docs

Testing (general information)

In order to run tests you need to install Tox:

$ pip install tox

Also, you will need a running MySQL instance with the database migrations applied from the previous step. Tests are dependent on pre-created MySQL database for persistence. Please set env var

$ export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>

Testing: PEP8

In order to run PEP8 style checks run following command:

$ tox -e pep8

Testing: Functional

In order to run functional tests run following command:

$ tox -e py35-functional

Pros:

• lightweight (controllers and DB models testing)
• no OpenStack required
• no IronFunctions required

Cons:

• MySQL server required
• OpenStack authentication is not tested
• IronFunctions API stubbed with fake implementation

Testing: Integration

Integration tests are dependent on following env variables:

• TEST_DB_URI - similar to functional tests, database endpoint
• FUNCTIONS_API_URL - IronFunctions API URL (default value - http://localhost:8080/v1)
• OS_AUTH_URL - OpenStack Identity endpoint
• OS_PROJECT_NAME - OpenStack user-specific project name
• OS_USERNAME - OpenStack user name
• OS_PASSWORD - OpenStack user user password

To run tests use following command:

export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
export OS_PROJECT_NAME=<project-name>
export OS_USERNAME=<project-name>
export OS_PASSWORD=<project-name>
tox -epy35-integration

Testing: Coverage regression

In order to build quality software it is necessary to keep test coverage at its highest point. So, as part of Tox testing new check was added - functional test coverage regression. In order to run it use following command:

$ tox -e py35-functional-regression

3rd party bugs to resolve

IronFunctions:

• https://github.com/iron-io/functions/issues/298
• https://github.com/iron-io/functions/issues/296
• https://github.com/iron-io/functions/issues/275
• https://github.com/iron-io/functions/issues/274

TODOs

Swagger doc:

• Make swagger doc more explicit on HTTP POST/UPDATE body content
• HTTP headers requests

IronFunctions:

• Support app deletion in IronFunctions
• Support tasks listing/showing

Laos:

• Tests: integration, functional, units
• Better logging coverage

Python Functions client:

• Support logging instance passing in function-python
• python-laosclient (ReST API client and CLI tool)
• App writing examples

Contacts

Feel free to reach us out at:

• Slack channel
• Email