x/picasso
Code Issues Proposed changes

Initial commit

Browse Source 
Project LaOS aka Lambdas-on-OpenStack
This commit is contained in:
Denis Makogon 2016-11-14 17:53:16 +02:00
commit 4471a4081a
42 changed files with 1968 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
.coveragerc Normal file
View File

@ -0,0 +1,29 @@
# .coveragerc to control coverage.py
[run]
branch = True
source=laos
omit=laos/tests*,
[report]
# Regexes for lines to exclude from consideration
exclude_lines =
# Have to re-enable the standard pragma
pragma: no cover
# Don't complain about missing debug-only code:
def __repr__
if self\.debug
# Don't complain if tests don't hit defensive assertion code:
raise AssertionError
raise NotImplementedError
# Don't complain if non-runnable code isn't run:
if 0:
if __name__ == .__main__.:
ignore_errors = False
[html]
directory=cover

15
.gitignore vendored Normal file
View File

@ -0,0 +1,15 @@
*.pyc
.testrepository
.tox/*
dist/*
build/*
html/*
*.egg*
cover/*
.coverage
rdserver.txt
python-troveclient.iml
# Files created by releasenotes build
releasenotes/build
.coverage.*

6
.testr.conf Normal file
View File

@ -0,0 +1,6 @@
[DEFAULT]
test_command=PYTHONASYNCIODEBUG=${PYTHONASYNCIODEBUG:-1} \
${PYTHON:-python} -m subunit.run discover -t ./ ./ $LISTOPT $IDOPTION
test_id_option=--load-list $IDFILE
test_list_option=--list

176
LICENSE Normal file
View File

@ -0,0 +1,176 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

6
MANIFEST.in Normal file
View File

@ -0,0 +1,6 @@
include AUTHORS
include ChangeLog
exclude .gitignore
exclude .gitreview
global-exclude *.pyc

155
README.md Normal file
View File

@ -0,0 +1,155 @@
Project LaOS aka Lambdas-on-OpenStack
=====================================
Mission
-------
Provide capabilities to run software in "serverless" way.
Serverless
----------
Serverless is a new paradigm in computing that enables simplicity,
efficiency and scalability for both developers and operators.
It's important to distinguish the two, because the benefits differ:
Benefits for developers
-----------------------
The main benefits that most people refer to are on the developer side and they include:
* No servers to manage (serverless) -- you just upload your code and the platform deals with the infrastructure
* Super simple coding -- no more monoliths! Just simple little bits of code
* Pay by the milliseconds your code is executing -- unlike a typical application that runs 24/7, and you're paying
24/7, functions only run when needed
Benefits for operators
----------------------
If you will be operating IronFunctions (the person who has to manage the servers behind the serverless),
then the benefits are different, but related.
* Extremely efficient use of resources
* Unlike an app/API/microservice that consumes resources 24/7 whether they
are in use or not, functions are time sliced across your infrastructure and only consume resources while they are
actually doing something
* Easy to manage and scale
* Single system for code written in any language or any technology
* Single system to monitor
* Scaling is the same for all functions, you don't scale each app independently
* Scaling is simply adding more IronFunctions nodes
System requirements
-------------------
* Operating system: Linux/MacOS
* Python version: 3.5 or greater
* Database: MySQL 5.7 or greater
Quick-start guide
-----------------
Install DevStack with [IronFunctions enabled](https://github.com/iron-io/functions-devstack-plugin/blob/master/README.rst).
Pull down [Project LaOS sources](https://github.com/denismakogon/project-laos).
Create Python3.5 virtualenv:
$ virtualenv -p python3.5 .venv
$ source .venv/bin/activate
Install dependencies:
$ pip install -r requirements.txt -r test-requirements.txt
Install `functions_python` lib:
$ pip install -e git+ssh://git@github.com/iron-io/functions_python.git#egg=functions-python
Install LaOS itself:
$ pip install -e .
Migrations
----------
Once all dependencies are installed it is necessary to run database migrations.
Before that please adjust [alembic.ini](alembic.ini) row #32
sqlalchemy.url = mysql+pymysql://root:root@192.168.0.112/functions
In this section please specify connection URI to your own database.
Once it is done just hit alembic to apply migrations:
$ alembic upgrade head
Starting a server
-----------------
Once it is finished you will have a console script `laos-api`:
$ laos-api --help
Usage: laos-api [OPTIONS]
Starts an Project Laos API service
Options:
--host TEXT API service bind host.
--port INTEGER API service bind port.
--db-uri TEXT LaOS persistence storage URI.
--keystone-endpoint TEXT OpenStack Identity service endpoint.
--functions-host TEXT Functions API host
--functions-port INTEGER Functions API port
--functions-api-version TEXT Functions API version
--functions-api-protocol TEXT Functions API protocol
--log-level TEXT Logging file
--log-file TEXT Log file path
--help Show this message and exit.
Minimum required options to start LaOS API service:
--db-uri mysql://root:root@192.168.0.112/functions
--keystone-endpoint http://192.168.0.112:5000/v3
--functions-host 192.168.0.112
--functions-port 10501
--log-level INFO
Examining API
-------------
In [examples](examples/) folder you can find a script that examines available API endpoints, but this script relays on:
* `LAOS_API_URL` - Project LaOS API endpoint
* `OS_AUTH_URL` - OpenStack Auth URL
* `OS_PROJECT_ID` - it can be found in OpenStack Dashboard or in CLI
Along with that, you need to adjust [token_request.json](examples/token_request.json) in order to retrieve X-Auth-Token for further authentication against OpenStack and LaOS API service.
Then just run script:
OS_AUTH_URL=http://192.168.0.112:5000/v3 OS_PROJECT_ID=8fb76785313a4500ac5367eb44a31677 ./hello-lambda.sh
Please note, that given values are project-specific, so they can't be reused.
TODOs
-----
* Create aiohttp swagger API using [aiohttp-swagger](https://github.com/cr0hn/aiohttp-swagger)
* Support app deletion in IronFunctions
* Support tasks listing/showing
* Tests: integration, functional, units
* better logging coverage
* Support logging instance passing in [function-python](https://github.com/iron-io/functions_python)
* python-laosclient (ReST API client and CLI tool)
* App writing examples
Contacts
--------
Feel free to reach us out at:
* [Slack channel](https://open-iron.herokuapp.com/)
* [Email](https://github.com/denismakogon)

68
alembic.ini Normal file
View File

@ -0,0 +1,68 @@
# A generic, single database configuration.
[alembic]
# path to migration scripts
script_location = migrations
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s
# max length of characters to apply to the
# "slug" field
#truncate_slug_length = 40
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
# version location specification; this defaults
# to migrations/versions. When using multiple version
# directories, initial revisions must be specified with --version-path
# version_locations = %(here)s/bar %(here)s/bat migrations/versions
# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8
sqlalchemy.url = mysql+pymysql://root:root@192.168.0.112/functions
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

51
examples/hello-lambda.sh Executable file
View File

@ -0,0 +1,51 @@
#!/usr/bin/env bash
set +x
export LAOS_API_URL=${LAOS_API_URL:-http://localhost:10001}
export OS_AUTH_URL=${OS_AUTH_URL:-http://localhost:5000/v3}
export OS_PROJECT_ID=${OS_PROJECT_ID:-"dummy_project_id"}
export OS_TOKEN=`curl -si -d @token_request.json -H "Content-type: application/json" ${OS_AUTH_URL}/auth/tokens | awk '/X-Subject-Token/ {print $2}'`
echo -e "Listing apps\n"
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Creating app\n"
curl -X POST -d '{"app":{"name": "testapp"}}' ${LAOS_API_URL}/v1/${PROJECT_ID}/apps -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Listing apps\n"
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Showing app info\n"
export raw_app_info=`curl localhost:10001/v1/${OS_PROJECT_ID}/apps -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool | grep name | awk '{print $2}'`
export app_name=${raw_app_info:1:30}
unset $raw_app_info
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name} -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Listing app routes\n"
curl ${LAOS_API_URL}/v1/${PROJECT_ID}/apps/${APP_NAME}/routes -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Creating app sync route\n"
curl -X POST -d '{"route":{"type": "sync", "path": "/hello-sync", "image": "iron/hello"}}' ${LAOS_API_URL}/v1/${PROJECT_ID}/apps/${APP_NAME}/route -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json"
echo -e "Listing app routes\n"
curl ${LAOS_API_URL}/v1/${PROJECT_ID}/apps/${APP_NAME}/routes -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Show app route\n"
curl ${LAOS_API_URL}/v1/${PROJECT_ID}/apps/${APP_NAME}/routes/hello-sync -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Running app sync route\n"
curl -X POST -d '{"name": "Johnny"}' ${LAOS_API_URL}/r/${PROJECT_ID}/${APP_NAME}/hello-sync -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Deleting app route\n"
curl -X DELETE ${LAOS_API_URL}/v1/${PROJECT_ID}/apps/${APP_NAME}/routes/hello_sync -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Creating app async route\n"
curl -X POST -d '{"route":{"type": "async", "path": "/hello-async", "image": "iron/hello"}}' ${LAOS_API_URL}/v1/${PROJECT_ID}/apps/${APP_NAME}/route -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Running app async route\n"
curl -X POST -d '{"name": "Johnny"}' ${LAOS_API_URL}/r/${PROJECT_ID}/${APP_NAME}/hello-async -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Deleting app\n"
curl -X DELETE ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name} -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool

20
examples/token_request.json Normal file
View File

@ -0,0 +1,20 @@
{
"auth": {
"identity": {
"methods": ["password"],
"password": {
"user": {
"name": "admin",
"domain": { "id": "default" },
"password": "root"
}
}
},
"scope": {
"project": {
"name": "admin",
"domain": { "id": "default" }
}
}
}
}

0
laos/__init__.py Normal file
View File

0
laos/api/__init__.py Normal file
View File

0
laos/api/controllers/__init__.py Normal file
View File

146
laos/api/controllers/apps.py Normal file
View File

@ -0,0 +1,146 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from aiohttp import web
from laos.api.views import app as app_view
from laos.common.base import controllers
from laos.common import config
from laos.models import app as app_model
class AppV1Controller(controllers.ServiceControllerBase):
controller_name = "apps"
version = "v1"
@controllers.api_action(method='GET', route='{project_id}/apps')
async def list(self, request, **kwargs):
c = config.Config.config_instance()
log, fnclient = c.logger, c.functions_client
project_id = request.match_info.get('project_id')
log.info("Listing apps for project: {}".format(project_id))
stored_apps = await app_model.Apps.find_by(project_id=project_id)
final = []
for app in stored_apps:
fn_app = await fnclient.apps.show(app.name, loop=c.event_loop)
final.append(app_view.AppView(app, fn_app).view())
return web.json_response(
data={
self.controller_name: final,
'message': "Successfully listed applications"
},
status=200
)
@controllers.api_action(method='POST', route='{project_id}/apps')
async def create(self, request, **kwargs):
c = config.Config.config_instance()
log, fnclient = c.logger, c.functions_client
project_id = request.match_info.get('project_id')
data = await request.json()
log.info("Creating an app for project: {} with data {}"
.format(project_id, str(data)))
app_name = "{}-{}".format(
data["app"]["name"],
project_id)[:30]
if await app_model.Apps.exists(app_name, project_id):
return web.json_response(data={
"error": {
"message": "App {0} already exists".format(app_name)
}
}, status=409)
fn_app = await fnclient.apps.create(app_name, loop=c.event_loop)
stored_app = await app_model.Apps(
name=app_name, project_id=project_id,
description=data["app"].get(
"description",
"App for project {}".format(
project_id))).save()
return web.json_response(
data={
"app": app_view.AppView(stored_app, fn_app).view(),
"message": "App successfully created",
}, status=200
)
@controllers.api_action(method='GET', route='{project_id}/apps/{app}')
async def get(self, request, **kwargs):
c = config.Config.config_instance()
log, fnclient = c.logger, c.functions_client
project_id = request.match_info.get('project_id')
app = request.match_info.get('app')
log.info("Requesting an app for project: {} with name {}"
.format(project_id, app))
if not (await app_model.Apps.exists(app, project_id)):
return web.json_response(data={
"error": {
"message": "App {0} not found".format(app),
}
}, status=404)
stored_app = (await app_model.Apps.find_by(
project_id=project_id, name=app)).pop()
fn_app = await fnclient.apps.show(app, loop=c.event_loop)
return web.json_response(
data={
"app": app_view.AppView(stored_app, fn_app).view(),
"message": "Successfully loaded app",
},
status=200
)
# TODO(denismakogon): disabled until iron-io/functions/pull/259
#
# @controllers.api_action(method='PUT', route='{project_id}/apps/{app}')
# async def update(self, request, **kwargs):
# log = config.Config.config_instance().logger
# project_id = request.match_info.get('project_id')
# app = request.match_info.get('app')
# data = await request.json()
# log.info("Updating an app {} for project: {} with data {}"
# .format(app, project_id, str(data)))
# return web.json_response(
# data={
# "app": {}
# },
# status=200
# )
@controllers.api_action(method='DELETE', route='{project_id}/apps/{app}')
async def delete(self, request, **kwargs):
project_id = request.match_info.get('project_id')
app = request.match_info.get('app')
if not (await app_model.Apps.exists(app, project_id)):
return web.json_response(data={
"error": {
"message": "App {0} not found".format(app),
}
}, status=404)
await app_model.Apps.delete(
project_id=project_id, name=app)
# TODO(denismakogon): enable DELETE to IronFunctions when once
# https://github.com/iron-io/functions/issues/274 implemented
# fn_app = await fnclient.apps.delete(app, loop=c.event_loop)
return web.json_response(
data={
"message": "App successfully deleted",
}, status=200)

160
laos/api/controllers/routes.py Normal file
View File

@ -0,0 +1,160 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from aiohttp import web
from laos.api.views import app as app_view
from laos.common.base import controllers
from laos.common import config
from laos.models import app as app_model
class AppRouteV1Controller(controllers.ServiceControllerBase):
controller_name = "routes"
version = "v1"
@controllers.api_action(
method='GET', route='{project_id}/apps/{app}/routes')
async def list(self, request, **kwargs):
c = config.Config.config_instance()
log, fnclient = c.logger, c.functions_client
project_id = request.match_info.get('project_id')
app = request.match_info.get('app')
if not (await app_model.Apps.exists(app, project_id)):
return web.json_response(data={
"error": {
"message": "App {0} not found".format(app),
}
}, status=404)
fn_app_routes = (await (await fnclient.apps.show(
app, loop=c.event_loop)).routes.list(loop=c.event_loop))
log.info("Listing app {} routes for project: {}."
.format(app, project_id))
return web.json_response(data={
"routes": app_view.AppRouteView(fn_app_routes).view(),
"message": "Successfully loaded app routes",
}, status=200)
@controllers.api_action(
method='POST', route='{project_id}/apps/{app}/routes')
async def create(self, request, **kwargs):
c = config.Config.config_instance()
log, fnclient = c.logger, c.functions_client
project_id = request.match_info.get('project_id')
app = request.match_info.get('app')
if not (await app_model.Apps.exists(app, project_id)):
return web.json_response(data={
"error": {
"message": "App {0} not found".format(app),
}
}, status=404)
data = (await request.json())['route']
path = data['path']
try:
fn_app = await fnclient.apps.show(app, loop=c.event_loop)
except Exception as ex:
return web.json_response(data={
"error": {
"message": getattr(ex, "reason", str(ex)),
}
}, status=getattr(ex, "status", 500))
try:
await fn_app.routes.show(path, loop=c.event_loop)
return web.json_response(data={
"error": {
"message": (
"App {} route {} already exist"
.format(app, path)
)
}
}, status=409)
except Exception as ex:
if getattr(ex, "status", 500) != 404:
return web.json_response(data={
"error": {
"message": getattr(ex, "reason", str(ex)),
}
}, status=getattr(ex, "status", 500))
new_fn_route = (await (await fnclient.apps.show(
app, loop=c.event_loop)).routes.create(
**data, loop=c.event_loop))
log.info("Creating new route in app {} "
"for project: {} with data {}"
.format(app, project_id, str(data)))
return web.json_response(data={
"route": app_view.AppRouteView([new_fn_route]).view(),
"message": "App route successfully created"
}, status=200)
@controllers.api_action(
method='GET', route='{project_id}/apps/{app}/routes/{route}')
async def get(self, request, **kwargs):
c = config.Config.config_instance()
log, fnclient = c.logger, c.functions_client
project_id = request.match_info.get('project_id')
app = request.match_info.get('app')
path = request.match_info.get('route')
try:
fn_app = await fnclient.apps.show(app, loop=c.event_loop)
route = await fn_app.routes.show(
"/{}".format(path), loop=c.event_loop)
except Exception as ex:
return web.json_response(data={
"error": {
"message": getattr(ex, "reason", str(ex)),
}
}, status=getattr(ex, "status", 500))
log.info("Requesting route {} in app {} for project: {}"
.format(path, app, project_id))
return web.json_response(data={
"route": app_view.AppRouteView([route]).view().pop(),
"message": "App route successfully loaded"
}, status=200)
@controllers.api_action(
method='DELETE', route='{project_id}/apps/{app}/routes/{route}')
async def delete(self, request, **kwargs):
c = config.Config.config_instance()
log, fnclient = c.logger, c.functions_client
project_id = request.match_info.get('project_id')
app = request.match_info.get('app')
path = request.match_info.get('route')
log.info("Deleting route {} in app {} for project: {}"
.format(path, app, project_id))
try:
fn_app = await fnclient.apps.show(app, loop=c.event_loop)
await fn_app.routes.show("/{}".format(path), loop=c.event_loop)
await fn_app.routes.delete("/{}".format(path), loop=c.event_loop)
except Exception as ex:
return web.json_response(data={
"error": {
"message": getattr(ex, "reason", str(ex)),
}
}, status=getattr(ex, "status", 500))
return web.json_response(data={
"message": "Route successfully deleted",
}, status=200)

70
laos/api/controllers/runnable.py Normal file
View File

@ -0,0 +1,70 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from aiohttp import web
from laos.common.base import controllers
from laos.common import config
class RunnableV1Controller(controllers.ServiceControllerBase):
controller_name = "runnable"
# IronFunction uses `r` as runnable instead API version
version = "r"
@controllers.api_action(
method='POST', route='{project_id}/{app}/{route}')
async def run(self, request, **kwargs):
c = config.Config.config_instance()
fnclient = c.functions_client
app = request.match_info.get('app')
path = "/{}".format(request.match_info.get('route'))
try:
data = await request.json()
except Exception:
# in may appear that no data supplied with POST to function
data = {}
try:
fn_app = await fnclient.apps.show(
app, loop=c.event_loop)
route = await fn_app.routes.show(
path, loop=c.event_loop)
result = await fn_app.routes.execute(
path, loop=c.event_loop, **data)
except Exception as ex:
return web.json_response(data={
"error": {
"message": getattr(ex, "reason", str(ex)),
}
}, status=getattr(ex, "status", 500))
def process_result(res):
if route.type == "async":
_data = {
"task_id": res["call_id"],
"message": ("App {} async route {} "
"execution started".format(app, path))
}
else:
_data = {
"result": res,
"message": ("App {} sync route {} "
"execution finished".format(app, path))
}
return _data
return web.json_response(status=200, data=process_result(result))

62
laos/api/controllers/tasks.py Normal file
View File

@ -0,0 +1,62 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from aiohttp import web
# from laos.models import app as app_model
from laos.common.base import controllers
# from laos.common import config
# TODO(denismakogon): disabled until
# https://github.com/iron-io/functions/issues/275
class TasksV1Controller(controllers.ServiceControllerBase):
controller_name = "tasks"
version = "v1"
@controllers.api_action(
method='GET', route='{project_id}/tasks')
async def get(self, request, **kwargs):
# c = config.Config.config_instance()
# fnclient = c.functions_client
# project_id = request.match_info.get('project_id')
# stored_apps = await app_model.Apps.find_by(project_id=project_id)
# final = []
# for app in stored_apps:
# fn_app = await fnclient.apps.show(app.name, loop=c.event_loop)
return web.json_response(data={
"error": {
"message": "Not supported"
}
}, status=405)
@controllers.api_action(
method='GET', route='{project_id}/tasks/{task}')
async def show(self, request, **kwargs):
# c = config.Config.config_instance()
# fnclient = c.functions_client
# project_id = request.match_info.get('project_id')
# stored_apps = await app_model.Apps.find_by(project_id=project_id)
# final = []
# for app in stored_apps:
# fn_app = await fnclient.apps.show(app.name, loop=c.event_loop)
return web.json_response(data={
"error": {
"message": "Not supported"
}
}, status=405)

0
laos/api/middleware/__init__.py Normal file
View File

29
laos/api/middleware/content_type.py Normal file
View File

@ -0,0 +1,29 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from aiohttp import web
async def content_type_validator(app: web.Application, handler):
async def middleware_handler(request: web.Request):
headers = request.headers
content_type = headers.get("Content-Type")
if "application/json" not in content_type:
return web.json_response(
data={
"error": {
"message": "Invalid content type"
}
}, status=400)
return await handler(request)
return middleware_handler

46
laos/api/middleware/keystone.py Normal file
View File

@ -0,0 +1,46 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from keystoneauth1 import identity
from keystoneauth1 import session
from keystoneclient import client
from aiohttp import web
from laos.common import config
async def auth_through_token(app: web.Application, handler):
async def middleware_handler(request: web.Request):
headers = request.headers
x_auth_token = headers.get("X-Auth-Token")
project_id = request.match_info.get('project_id')
c = config.Config.config_instance()
try:
auth = identity.Token(c.auth_url,
token=x_auth_token,
project_id=project_id)
sess = session.Session(auth=auth)
ks = client.Client(session=sess,
project_id=project_id)
ks.authenticate(token=x_auth_token)
except Exception as ex:
return web.json_response(status=401, data={
"error": {
"message": ("Not authorized. Reason: {}"
.format(str(ex)))
}
})
return await handler(request)
return middleware_handler

0
laos/api/views/__init__.py Normal file
View File

48
laos/api/views/app.py Normal file
View File

@ -0,0 +1,48 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
class AppView(object):
def __init__(self, stored_app, fn_app):
self.app = stored_app
self.fn_app = fn_app
def view(self):
return {
"id": self.app.id,
"name": self.app.name,
"config": self.fn_app.config,
"description": self.app.description,
"created_at": self.app.created_at,
"updated_at": self.app.updated_at,
"project_id": self.app.project_id,
}
class AppRouteView(object):
def __init__(self, fn_app_routes):
self.routes = fn_app_routes
def view(self):
view = []
for route in self.routes:
view.append({
"path": route.path,
"type": route.type,
"memory": route.memory,
"image": route.image,
})
return view

0
laos/common/__init__.py Normal file
View File

0
laos/common/base/__init__.py Normal file
View File

78
laos/common/base/controllers.py Normal file
View File

@ -0,0 +1,78 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import functools
import inspect
from aiohttp import web
class ServiceControllerBase(object):
controller_name = 'abstract'
version = ""
def __get_handlers(self):
# when this method gets executed by child classes
# method list includes a method of parent class,
# so this code ignores it because it doesn't belong to controllers
methods = [getattr(self, _m)
for _m in dir(self) if inspect.ismethod(
getattr(self, _m)) and not _m.startswith("__")][1:]
return [[method,
method.arg_method,
method.arg_route] for method in methods]
def __init__(self, service: web.Application):
for fn, http_method, route in self.__get_handlers():
proxy_fn = '_'.join([fn.__name__, self.controller_name])
setattr(self, proxy_fn, fn)
service.router.add_route(http_method,
"/{}/{}".format(self.version, route),
getattr(self, proxy_fn),
name=proxy_fn)
def api_action(**outter_kwargs):
"""
Wrapps API controller action actions handler
:param outter_kwargs: API instance action key-value args
:return: _api_handler
Example:
class Controller(ControllerBase):
@api_action(method='GET')
async def index(self, request, **kwargs):
return web.Response(
text=str(request),
reason="dumb API",
status=201)
"""
def _api_handler(func):
@functools.wraps(func)
def wrapper(self, *args, **kwargs):
return func(self, *args, *kwargs)
for key, value in outter_kwargs.items():
setattr(wrapper, 'arg_{}'.format(key), value)
setattr(wrapper, 'is_module_function', True)
return wrapper
return _api_handler

73
laos/common/base/service.py Normal file
View File

@ -0,0 +1,73 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import asyncio
import os
import typing
from aiohttp import web
from laos.common.base import controllers as c
from laos.common import logger as log
class AbstractWebServer(object):
def __init__(self, host: str='127.0.0.1',
port: int= '10001',
controllers: typing.List[c.ServiceControllerBase]=None,
middlewares: list=None,
event_loop: asyncio.AbstractEventLoop=None,
logger=log.UnifiedLogger(
log_to_console=True,
level="INFO").setup_logger(__name__)):
"""
HTTP server abstraction class
:param host:
:param port:
:param controllers:
:param middlewares:
:param event_loop:
:param logger:
"""
self.host = host
self.port = port
self.controllers = controllers
self.event_loop = event_loop
self.service = web.Application(
loop=self.event_loop,
debug=os.environ.get('PYTHONASYNCIODEBUG', 0),
middlewares=middlewares if middlewares else [])
self.service_handler = None
self.server = None
self.logger = logger
def _apply_routers(self):
if self.controllers:
for controller in self.controllers:
controller(self.service)
def shutdown(self):
self.server.close()
self.event_loop.run_until_complete(self.server.wait_closed())
self.event_loop.run_until_complete(
self.service_handler.finish_connections(1.0))
self.event_loop.run_until_complete(self.service.cleanup())
def initialize(self):
self._apply_routers()
try:
web.run_app(self.service, host=self.host, port=self.port,
shutdown_timeout=10, access_log=self.logger)
except KeyboardInterrupt:
self.shutdown()

70
laos/common/config.py Normal file
View File

@ -0,0 +1,70 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import aiomysql
import asyncio
from laos.common import utils
from functionsclient.v1 import client
class Config(object, metaclass=utils.Singleton):
def __init__(self, **kwargs):
for k, v in kwargs.items():
setattr(self, k, v)
self.config = self
@classmethod
def config_instance(cls):
return cls._instance.config
class Connection(object, metaclass=utils.Singleton):
def __init__(self, db_uri, loop=None):
self.uri = db_uri
self.engine = loop.run_until_complete(self.get_engine(loop=loop))
self.loop = loop
def get_engine(self, loop=None):
username, password, host, port, db_name = utils.split_db_uri(self.uri)
return aiomysql.create_pool(host=host, port=port if port else 3306,
user=username, password=password,
db=db_name, loop=loop)
@classmethod
def from_class(cls):
return cls._instance.engine
class FunctionsClient(client.FunctionsAPIV1, metaclass=utils.Singleton):
def __init__(self, api_host: str,
api_port: int=8080,
api_protocol: str="http",
api_version: str="v1"):
# TODO(denismakogon): enable API version discovery
super(FunctionsClient, self).__init__(api_host, api_port, api_protocol)
self.client = self
async def ping(self,
loop: asyncio.AbstractEventLoop=asyncio.get_event_loop()):
await self.apps.list(loop=loop)
@classmethod
def from_class(cls):
return cls._instance.client

95
laos/common/logger.py Normal file
View File

@ -0,0 +1,95 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import datetime
import logging
import sys
from laos.common import utils
def common_logger_setup(
level=logging.DEBUG,
filename='/tmp/aiorchestra.log',
log_formatter='[%(asctime)s] - '
'%(name)s - '
'%(levelname)s - '
'%(module)s.py:%(lineno)d - '
'%(funcName)s - '
'%(message)s',
datetime_formatter='%Y-%m-%d %H:%M:%S',
log_to_console=False):
if log_to_console:
log_handler = logging.StreamHandler(sys.stdout)
else:
log_handler = logging.FileHandler(filename)
log_format = logging.Formatter(log_formatter, datetime_formatter)
log_handler.setFormatter(log_format)
return log_handler, level
def setup_logging(name,
filename='/tmp/laos-api-{}.log'.format(
datetime.datetime.now()),
level=logging.DEBUG,
log_to_console=False,
formatter=None):
log_file_handler, log_level = common_logger_setup(
filename=filename,
level=level,
log_to_console=log_to_console,
log_formatter=formatter)
logger = logging.getLogger(name)
logger.addHandler(log_file_handler)
logger.setLevel(log_level)
return logger
class Singleton(type):
_instance = None
def __call__(cls, *args, **kwargs):
if not cls._instance:
cls._instance = super(Singleton, cls).__call__(*args, **kwargs)
return cls._instance
class UnifiedLogger(object, metaclass=utils.Singleton):
def __init__(self,
filename='/tmp/laos-api-{}.log'.format(
datetime.datetime.now()),
level=logging.DEBUG, log_to_console=False):
self.filename = filename
self.level = level
if 'DEBUG' not in level:
self.log_formatter = (
'[%(asctime)s] - '
'%(message)s')
else:
self. log_formatter = (
'[%(asctime)s] - '
'%(name)s - '
'%(levelname)s - '
'%(module)s.py:%(lineno)d - '
'%(funcName)s - '
'%(message)s')
self.log_to_console = log_to_console
self.logger = self
def setup_logger(self, name):
return setup_logging(name, filename=self.filename,
level=self.level,
log_to_console=self.log_to_console,
formatter=self.log_formatter)

102
laos/common/persistence.py Normal file
View File

@ -0,0 +1,102 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import datetime
import uuid
from laos.common import config
class BaseDatabaseModel(object):
INSERT = "INSERT INTO {} VALUES {}"
SELECT = "SELECT * FROM {} {}"
WHERE = "WHERE {}"
AND = " AND "
DELETE = "DELETE FROM {} {}"
def __init__(self, **kwargs):
self.id = uuid.uuid4().hex
self.created_at = str(datetime.datetime.now())
self.updated_at = str(datetime.datetime.now())
for k, v in kwargs.items():
setattr(self, k, v)
async def save(self):
insert = self.INSERT.format(
self.table_name,
str(tuple([getattr(self, clmn) for clmn in self.columns]))
)
print(insert)
async with config.Connection.from_class().acquire() as conn:
async with conn.cursor() as cur:
await cur.execute(insert)
await conn.commit()
return self
@classmethod
async def delete(cls, **kwargs):
delete = cls.DELETE.format(
cls.table_name, cls.__define_where(**kwargs))
async with config.Connection.from_class().acquire() as conn:
async with conn.cursor() as cur:
await cur.execute(delete)
await conn.commit()
async def update(self, **kwargs):
async with config.Connection.from_class().acquire() as conn:
async with conn.cursor() as cur:
await cur.execute()
@classmethod
async def exists(cls, name, project_id):
return True if len(await cls.find_by(
name=name,
project_id=project_id)) else False
@classmethod
def __define_where(cls, **kwargs):
search_field_parts = []
final = []
for k, v in kwargs.items():
search_field_parts.append("{}='{}'".format(k, v))
for i in range(len(search_field_parts)):
final.extend([search_field_parts[i],
cls.AND if i != len(
search_field_parts) - 1 else ""])
return cls.WHERE.format("".join(final))
@classmethod
async def find_by(cls, **kwargs):
where = cls.__define_where(**kwargs)
async with config.Connection.from_class().acquire() as conn:
async with conn.cursor() as cur:
await cur.execute(cls.SELECT.format(
cls.table_name, where))
results = await cur.fetchall()
return [cls.from_tuple(instance)
for instance in results] if results else []
@classmethod
def from_tuple(cls, tpl):
items = []
for i in range(len(tpl)):
items.append((cls.columns[i], tpl[i]))
return cls(**dict(items))
def to_dict(self):
return self.__dict__

42
laos/common/utils.py Normal file
View File

@ -0,0 +1,42 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from urllib import parse
def split_db_uri(db_uri):
"""
Splits DB URI into consumable parts like:
- username
- password
- hostname
- port
- protocol schema
- path
:param db_uri:
:return:
"""
parts = parse.urlparse(db_uri)
return (parts.username, parts.password,
parts.hostname, parts.port, parts.path[1:])
class Singleton(type):
_instance = None
def __call__(cls, *args, **kwargs):
if not cls._instance:
cls._instance = super(
Singleton, cls).__call__(*args, **kwargs)
return cls._instance

0
laos/models/__init__.py Normal file
View File

28
laos/models/app.py Normal file
View File

@ -0,0 +1,28 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from laos.common import persistence
class Apps(persistence.BaseDatabaseModel):
table_name = "apps"
columns = (
"id",
"project_id",
"description",
"created_at",
"updated_at",
"name"
)

0
laos/service/__init__.py Normal file
View File

121
laos/service/laos_api.py Normal file
View File

@ -0,0 +1,121 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import asyncio
import click
import uvloop
from laos.api.controllers import apps
from laos.api.controllers import routes
from laos.api.controllers import runnable
from laos.api.controllers import tasks
from laos.api.middleware import content_type
from laos.api.middleware import keystone
from laos.common.base import service
from laos.common import config
from laos.common import logger as log
class API(service.AbstractWebServer):
def __init__(self, host: str='0.0.0.0',
port: int=10001,
loop: asyncio.AbstractEventLoop=asyncio.get_event_loop(),
logger=None):
super(API, self).__init__(
host=host,
port=port,
controllers=[
apps.AppV1Controller,
routes.AppRouteV1Controller,
runnable.RunnableV1Controller,
tasks.TasksV1Controller,
],
middlewares=[
keystone.auth_through_token,
content_type.content_type_validator,
],
event_loop=loop,
logger=logger,
)
@click.command(name='laos-api')
@click.option('--host', default='0.0.0.0', help='API service bind host.')
@click.option('--port', default=10001, help='API service bind port.')
@click.option('--db-uri', default='mysql://root:root@localhost/functions',
help='LaOS persistence storage URI.')
@click.option('--keystone-endpoint', default='http://localhost:5000/v3',
help='OpenStack Identity service endpoint.')
@click.option('--functions-host', default='localhost',
help='Functions API host')
@click.option('--functions-port', default=10501,
help='Functions API port')
@click.option('--functions-api-version', default='v1',
help='Functions API version')
@click.option('--functions-api-protocol', default='http',
help='Functions API protocol')
@click.option('--log-level', default='INFO',
help='Logging file')
@click.option('--log-file', default=None,
help='Log file path')
def server(host, port, db_uri,
keystone_endpoint,
functions_host,
functions_port,
functions_api_version,
functions_api_protocol,
log_level,
log_file,
):
"""
Starts an Project Laos API service
"""
logger = log.UnifiedLogger(
log_to_console=True if not log_file else False,
filename=None if not log_file else log_file,
level=log_level).setup_logger(__name__)
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
loop = asyncio.get_event_loop()
fnclient = config.FunctionsClient(
functions_host,
api_port=functions_port,
api_protocol=functions_api_protocol,
api_version=functions_api_version,
)
loop.run_until_complete(fnclient.ping(loop=loop))
conn = config.Connection(db_uri, loop=loop)
config.Config(
auth_url=keystone_endpoint,
functions_host=functions_host,
functions_port=functions_port,
functions_api_protocol=functions_api_protocol,
functions_api_version=functions_api_version,
functions_client=fnclient,
logger=logger,
connection=conn,
event_loop=loop,
)
API(host=host, port=port, loop=loop, logger=logger).initialize()
if __name__ == "__main__":
server()

1
migrations/README Normal file
View File

@ -0,0 +1 @@
Generic single-database configuration.

70
migrations/env.py Normal file
View File

@ -0,0 +1,70 @@
from __future__ import with_statement
from alembic import context
from sqlalchemy import engine_from_config, pool
from logging.config import fileConfig
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
# Interpret the config file for Python logging.
# This line sets up loggers basically.
fileConfig(config.config_file_name)
# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
# target_metadata = mymodel.Base.metadata
target_metadata = None
# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
def run_migrations_offline():
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url, target_metadata=target_metadata, literal_binds=True)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online():
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section),
prefix='sqlalchemy.',
poolclass=pool.NullPool)
with connectable.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata
)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

24
migrations/script.py.mako Normal file
View File

@ -0,0 +1,24 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
def upgrade():
${upgrades if upgrades else "pass"}
def downgrade():
${downgrades if downgrades else "pass"}

33
migrations/versions/7a2dcf8ac8bf_.py Normal file
View File

@ -0,0 +1,33 @@
"""empty message
Revision ID: 7a2dcf8ac8bf
Revises:
Create Date: 2016-11-9 16:24:27.886139
"""
# revision identifiers, used by Alembic.
revision = '7a2dcf8ac8bf'
down_revision = None
branch_labels = None
depends_on = None
import sqlalchemy as sa
from alembic import op
def upgrade():
op.create_table(
'apps',
sa.Column('id', sa.String(255), nullable=False),
sa.Column('project_id', sa.String(255), nullable=False),
sa.Column('description', sa.String(255), nullable=True),
sa.Column('created_at', sa.String(255)),
sa.Column('updated_at', sa.String(255)),
sa.Column('name', sa.String(255), nullable=False, primary_key=True),
)
def downgrade():
op.drop_table('apps')

12
requirements.txt Normal file
View File

@ -0,0 +1,12 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
uvloop
aiohttp # Apache-2.0
aiomysql
alembic>=0.8.4 # MIT
click
keystoneauth1>=2.14.0 # Apache-2.0
python-keystoneclient==3.6.0
SQLAlchemy<1.1.0,>=1.0.10 # MIT

2
setup.cfg Normal file
View File

@ -0,0 +1,2 @@
[bdist_wheel]
universal = 1

76
setup.py Normal file
View File

@ -0,0 +1,76 @@
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import os
import setuptools
def read(fname):
return open(os.path.join(os.path.dirname(__file__), fname)).read()
setuptools.setup(
name='laos',
version='0.0.1',
description='Project LaOS (Lambdas-on-OpenStack',
long_description=read('README.md'),
url='laos.readthedocs.org',
author='Denis Makogon',
author_email='denis@iron.io',
packages=setuptools.find_packages(),
install_requires=[
"uvloop",
"aiohttp",
"aiomysql",
"alembic>=0.8.4",
"click",
"keystoneauth1>=2.14.0",
"python-keystoneclient==3.6.0",
"SQLAlchemy<1.1.0,>=1.0.10",
],
license='License :: OSI Approved :: Apache Software License',
classifiers=[
'License :: OSI Approved :: Apache Software License',
'Intended Audience :: Information Technology',
'Intended Audience :: System Administrators',
'Intended Audience :: Developers',
'Environment :: No Input/Output (Daemon)',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Topic :: Software Development :: '
'Libraries :: Python Modules',
'Topic :: System :: Distributed Computing',
'Operating System :: Microsoft :: Windows',
'Operating System :: POSIX',
'Operating System :: Unix',
'Operating System :: MacOS',
],
keywords=['functions', 'lambdas', 'python API'],
platforms=['Linux', 'Mac OS-X', 'Unix'],
tests_require=[
'flake8==2.5.0',
'hacking<0.11,>=0.10.0',
'coverage>=4.0',
'sphinx!=1.3b1,<1.4,>=1.2.1',
'testrepository>=0.0.18',
'testtools>=1.4.0',
],
zip_safe=True,
entry_points={
'console_scripts': [
'laos-api = laos.service.laos_api:server',
]
},
)

10
test-requirements.txt Normal file
View File

@ -0,0 +1,10 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
flake8==2.5.0 # fix for https://gitlab.com/pycqa/flake8/issues/94
hacking<0.11,>=0.10.0
coverage>=4.0 # Apache-2.0
sphinx!=1.3b1,<1.4,>=1.2.1 # BSD
testrepository>=0.0.18 # Apache-2.0/BSD
testtools>=1.4.0 # MIT

44
tox.ini Normal file
View File

@ -0,0 +1,44 @@
# Function Python client
[tox]
envlist = py35,pep8
minversion = 1.6
skipsdist = True
[testenv]
passenv =
PYTHONASYNCIODEBUG
setenv = VIRTUAL_ENV={envdir}
usedevelop = True
install_command = pip install -U {opts} {packages}
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
commands = find . -type f -name "*.pyc" -delete
rm -f .testrepository/times.dbm
python setup.py testr --testr-args='{posargs}'
whitelist_externals = find
rm
[testenv:pep8]
commands = flake8
[testenv:venv]
commands = {posargs}
[testenv:cover]
commands =
coverage erase
python setup.py testr --coverage --testr-args='{posargs}'
coverage html
coverage report
[testenv:docs]
commands =
rm -rf doc/html doc/build
python setup.py build_sphinx
[flake8]
ignore = H202,H404,H405,H501
show-source = True
exclude=.venv,.git,.tox,dist,doc,*lib/python*,*egg,build,migrations