openstack/senlin
Code Issues Proposed changes

Refactored the doc subtree

Browse Source 
We need to start working on some introductory materials for the project
and service. We may also need to document our thoughts and design
decisions when moving forward. This patch tries to provide a basic
layout of the documents.

Change-Id: Iad4affead02d605c6fce2720adc85141fd1dc419
This commit is contained in:
tengqm
2015-06-05 05:10:04 -04:00
parent 12a4fa5cfa
commit 28547f011e
19 changed files with 316 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files

268
doc/source/developer/authorization.rst Normal file
View File

@@ -0,0 +1,268 @@
..
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
Senlin Authorization
====================
As a service to be consumed by end users and possibly other IT persons, Senlin
has some basic components and strategies to manage access control. The design
is meant to be as open as possible though the current focus as this document is
drafted is on enabling Keystone-based (aka. token-based) OpenStack
authorization.
This document presents an overview of the authentication and authorization
mechanisms provided by the Senlin API and its service engine. The top-most
design consideration of these mechanisms is to make it accommodating so that
the interactions with different authentication engines can be done using the
same framework. The reason behind this decision is to make Senlin cloud-backend
agnostic so it can be used to support clustering of resources in a multi-cloud,
or multi-region, or multi-availability-zone setups.
----------------
Major Components
----------------
In the context of an OpenStack cloud, the most important components invovled in
the authentication and the authorization process are:
- The Senlin client (i.e. the `python-senlinclient` package) which accepts
user credentials provided through environment variables and/or the command
line arguments and forwards them to the OpenStack SDK (i.e. the
`python-openstacksdk` package) when making service requests to Senlin API.
- The OpenStack SDK (`python-openstacksdk`) is used by Senlin engine to
interact with any other OpenStack services. The Senlin client also uses the
SDK to talk to the Senlin API. The SDK package translates the user-provided
credentials into a token by invoking the Keystone service.
- The Keystone middleware (i.e. `keystonemiddleware`) which backs the
`auth_token` WSGI middleware in the Senlin API pipeline provides a basic
validation filter. The filter is responsible to validate the token that
exists in the HTTP request header and then populates the HTTP request header
with detailed information for the downstream filters (including the API
itself) to use.
- The `context` WSGI middleware which is based on the `oslo.context` package
provides a constructor of the `RequestContext` data structure that
accompanies any requests down the WSGI application pipeline so that those
downstream components don't have to access the HTTP request header.
---------------
Usage Scenarios
---------------
There are several ways to raise a service request to the Senlin API, each of
which has its own characteristics that will affect the way authentication
and/or authorization is performed.
1) Users interact with Senlin service API using the Senlin client (i.e. the
`python-senlinclient` package) which includes a command line interface (CLI)
named `senlin`. The requests, after being preprocessed by the OpenStack SDK
will contain a valid Keystone token that can be validated by the
`auth_token` WSGI middleware.
2) Users interact with Senlin service API directly by making HTTP requests
where the requester's credentials have been validated by Keystone so the
requests will carry a valid Keystone token for verification by the
`auth_token` middleware as well.
3) Users interact with Senlin service API directly by making HTTP requests, but
the requests are "naked" ones which means that the requests do not contain
credentials as expected by Senlin API (or other OpenStack services). In
stead, the URI requested contains some special parameters for authentication
and/or authorization's purposes.
Scenario 1) and 2) are the most common ways for users to use Senlin API. They
share the same request format when the request arrives at the Senlin API
endpoint. Scenario 3) is a little bit different. What Senlin wants to achieve
is making no assumption where the service requests come from. That means it
cannot assume that the requester (could be any program) will fill in the
required headers in their service requests. One example of such use cases is
the Webhook API Senlin provides that enables a user to trigger an action on an
object managed by Senlin. Senlin provides a special support to these use cases.
-----------------------------
Operation Delegation (Trusts)
-----------------------------
Since Senlin models most operations as "Actions" that can be executed by worker
threads asynchronously, these operations have to be done on behalf of the
requester so that they can be properly traced, authenticated, audited or logged.
Credentials and Context
-----------------------
A generic solution to the delegation problem is to ask users to provide their
credentials to Senlin so Senlin can impersonate them when interacting with
other services. In fact, this may be the only solution that can be applied on
different cloud backends.
Senlin supports a `context` property for all "profile" types by default unless
overriden by a profile type implementation. This context can be treated as a
container for these credentials. Storing user credential in Senlin database
does imply a security risk. In future, we hope Senlin can make use of the
Barbican service for this purpose.
Senlin's implementation of context is based on the `oslo_context` package.
There are still room for improvement thanks to the new enhancements to that
package.
Trusts: Dealing with Token Expiration
-------------------------------------
In some cases, the solution above may be impractical because after the
client-side processing and/or the front-end middleware filtering, Senlin
cannot get the original user credentials (e.g. user name and password).
Senlin can only get a "token", which expires in an hour by default. This means
that after no more than one hour, Senlin won't be able to use this token for
authentication/authorization.
The OpenStack identity service (a.k.a Keystone) has considered this situation
and provided a solution. When a requester wants to delegate his/her roles in a
project to a 3rd party, he or she can create a "Trust" relationship between
him/her (the trustor) and that 3rd party (the trustee). The "Trust" has a
unique ID that can be used by the trustee when authenticating with Keystone.
Once trust ID is authenticated, the trustee can perform operations on behalf
of the trustor.
The trust extension in Keystone V3 can be used to solve the token expiration
problem. There are two ways to do this as shown below.
1) Requester Created Trusts: Before creating a profile, a requester can create
a trust with the trustee set to the `senlin` user. He or she can customize
the roles that can be assumed by `senlin`, which can be a subset of the
roles the requester currently has in that project. When the requester later
on creates a profile, he or she can provide the `trust_id` as a key of the
`context` property. Senlin can later on use this trust for authentication
and authorization's purpose.
2) Senlin Created Trusts: The solution above adds some burdens for an end user.
In order to make Senlin service easy of use, Senlin will do the trust
creation in the background. Whenever a new request comes in, Senlin will
check if there is an existing trust relationship between the requester and
the `senlin` user. Senlin will "hijack" the user's token and create a trust
with `senlin` as the trustee. This trust relationship is currently stored
in Senlin database, and the management of this sensitive information can be
delegated to Barbican as well in future.
Precedence Consideration
------------------------
Since there now exist more than one place for Senlin to get the credentials
for use, Senlin needs to impose a precedence among the credential sources.
When Senlin tries to contact a cloud service via a driver, the requests are
issued from a subclass of `Profile`. Senlin will check the `user` property of
the targeted cluster or node and retrieve the trust record from database using
the `user` as the key. By default, Senlin will try obtain a new token from
Keystone using the `senlin` user's credentials (configured in `senlin.conf`
file) and the `trust_id`. Before doing that, Senlin will check if the profile
used has a "customized" `context`. If there are credentials such as `password`
or `trust_id` in the context, Senlin deletes its current `trust_id` from the
context, and adds the credentials found in the profile into the context.
In this way, a user can specify the credentials Senlin should use when talking
to other cloud services by customizing the `context` property of a profile.
The specified credentials may and may not belong to the requester.
Trust Middleware
----------------
When a service request arrives at Senlin API, Senlin API checks if there is a
trust relationship built between the requester user and the `senlin` user. A
new trust is created if no such record is found.
Once a trust is found or created, the `trust_id` is saved into the current
`context` data structure. Down the invocation path, or during asynchronous
action executions, the `trust_id` will be used for token generation when
needed.
Senlin provides an internal database table to store the trust information. It
may be removed in future when there are better ways to handle this sensitive
information.
----------------------
Webhook Authentication
----------------------
Senlin provides a group of APIs for the creation and management "Webhooks".
A "Webhook" is a URI that can be accessed from any users or programs, provided
they possess some credentials that can be authenticated.
Webhook Creation
----------------
Webhooks are provided as a shortcut to trigger an action on an object on
behalf of a user. A webhook can be seen as a tuple (`object`, `action`,
`credentials`, `params`), where:
- `object`: an object which could be, e.g. a cluster, a node or a policy,
(In implementation, this field is represented by the `obj_type` and the
`obj_id` fields together);
- `action`: an action that will be performed;
- `credential`: on whom's behalf should the action be performed;
- `params`: optional parameters that will be used as inputs to the action.
When creating a webhook, a user has to specify an `object` and an `action`.
Optionally, the user can specify the `credential` to use when the action is
executed as a result of the webhook gets triggered. When omitted, Senlin will
use the `user` of the `object` and its associated trust saved in Senlin DB.
The only restriction is that the requester must be either the owner or the
project admin.
If the user does specify a `credential` for use, Senlin webhook middleware
will use the fields contained in `credential` to get a token from Keystone.
The newly created token will then be used for creating the webhook, as if the
request was from the user represented by the `credential` rather than the real
requester.
One special case is that the requester wants to use its own credential (which
is barely a token when arriving at Senlin API middleware) for the webhook. In
this case, the `credential` is specially formed by the client side.
In summary, the "requested user" can be one of 1) the requester; 2) the user
represented by the `credential`; 3) the owner of the object. In all cases,
the "requested user" must be either the owner of the object or the project
admin. In other words, the requests are still subject to policy checkings.
Encryption of Credentials
-------------------------
Since Senlin is creating a trust for each user. This trust information should
be treated as sensitive information. When storing them into database, it has
better be encrypted, and that is exactly how Senlin is doing now. In future,
this kind of data may be migrated to the Barbican service.
As of now, Senlin utilizes the `cryptography` package to do an encryption of
the data. The encrypted data can be decrypted only using the generated key.
Senlin will return the key to the requester, along with the UUID of the
generated webhook.
Triggering of Webhooks
----------------------
When a user wants to trigger a webhook, Senlin webhook middleware will first
verify if the referenced webook does exists. The middleware then use the key
provided by the user to decrypt the stored credential, i.e. the trust info.
By the way, the "key" data can appear as parameters on URI or data in the
request body. Senlin supports both ways. Senlin API will also extract any
parameters from the request for instantiating the action to execute.
When Senlin middleware has successfully decrypted the trust info, it will
try obtain a trust-scoped token using the `senlin` user's credential along
with the decrpted trust data. The returned trust-scoped token will then be
used to eventually trigger the creation on an action.

20
doc/source/developer/plugin_guide.rst Normal file
View File

@@ -0,0 +1,20 @@
..
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
Plugin Writer's Guide
=====================
Senlin provides an open design where developer can incorporate new profile
or policy implementations for different purposes.
[TBC]

35
doc/source/developer/profile_type.rst Normal file
View File

@@ -0,0 +1,35 @@
..
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
Development Guide for Profile Types
===================================
Implementation Hints
--------------------
Handling Context
^^^^^^^^^^^^^^^^
In the Profile class implementation, a profile can be stored into DB and then
loaded from DB given an ID. We don't record the context used by a profile. On
the contrary, the context is assigned to a profile when it is (re)intialized.
This enables a profile to be used by different context, which is usually the
context saved into an action. There won't be security problem if we have
recorded the correct context of an action.
Abstract Methods
^^^^^^^^^^^^^^^^
The Profile class provides abstract methods such as `do_create()`,
`do_delete()` and `do_update()` for sub-classes to override.

72
doc/source/developer/testing.rst Normal file
View File

@@ -0,0 +1,72 @@
Senlin testing
==============
All unit tests are to be placed in the senlin/tests directory, and tests can
be organized by the targeted subsystems/modules. Each subsystem directory
must contain a separate blank __init__.py for tests discovery to function.
An example directory structure::
senlin
`-- tests
|-- db
| |-- __init__.py
| |-- test_cluster_api.py
| `-- test_node_api.py
|-- engine
| |-- __init__.py
| |-- test_clusters.py
| `-- test_nodes.py
|-- __init__.py
`-- test_utils.py
Implementing a test
-------------------
Testrepository - http://pypi.python.org/pypi/testrepository is used to
find and run tests, parallelize their runs, and record timing/results.
If new dependencies are introduced upon the development of a test, the
test-requirements.txt file needs to be updated so that the virtual
environment will be able to successfully execute all tests.
The `test-requirements.txt` file needs to be synchronized with the
openstack/global-requirements project. Developers should try avoid
introducing additional package dependencies unless forced to.
Running Tests
-------------
Senlin uses `tox` for running unit tests, as practiced by many other OpenStack
projects::
$ tox
This by default will run unit tests suite with Python 2.7 and PEP8/HACKING
style checks. To run only one type of tests you can explicitly provide `tox`
with the test environment to use::
$ tox -epy27 # test suite on python 2.7
$ tox -epep8 # run full source code checker
To run only a subset of tests, you can provide `tox` with a regex argument::
$ tox -epy27 -- VolumeTests
To use debugger like `pdb` during test run, you have to run tests directly
with other, non-concurrent test runner instead of `testr`.
That also presumes that you have a virtual env with all senlin dependencies
installed and configured.
Below is an example bash script using `testtools` test runner that also allows
running single tests by providing a regex::
#! /usr/bin/env sh
testlist=$(mktemp)
testr list-tests "$1" > $testlist
python -m testtools.run --load-list $testlist
A more convenient way to run specific test is to name the unit test directly,
as shown below::
$ python -m testtools.run senlin.tests.db.test_cluster_api

99
doc/source/developer/webhook.rst Normal file
View File

@@ -0,0 +1,99 @@
..
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
Webhook
=======
A webhook is an URI that encodes a tuple (entity, action, params), where:
- the ``entity`` can be a cluster, a node, a policy object;
- ``action`` is the name of a built-in action supported by the object;
- ``params`` is a dictionary feeding values as arguments to the action.
Webhooks are used to trigger a specific action on a senlin entity, typically
scaling out/in action on a cluster. In the long term, senlin may support
user-provided actions where ``action`` will be interpreted as the UUID or name
of a user-provided action.
Design
------
Workflow
++++++++
1. User creates a webhook through webhook API. User needs to specify what action
and which senlin entity this webhook is bound to. Also for some specific
actions, user can define the parameters they want to use when invoking the
cluster action API, e.g. the adjustment of scaling operation; User also
`HAVE TO` specify the credential(usually a user_id and its password)
explicitly/implicitly. This credential will be used by Senlin service later
to execute the real action, e.g. cluster scaling, when webhook is triggered
later.
2. Senlin service receives the request and does three things:
- Creating a webhook object that contains all necessary information used
to trigger specific action of a senlin entity;
- Encrypting the credential information to ensure it won't be hacked and
then storing the encrypted password into DB;
- Generating a webhook url with the following format and return it to user:
http://{server_ip:port}/v1/{tenant_id}/webhooks/{webhook_id}/trigger?key=$KEY
NOTE: `$KEY` is the key to decrypted the password so user has to keep
it safely. Also the webhook_url can only be got at the first time the
webhook is created, so user also need to record it carefully.
3. User triggers the webhook by sending a post request to its url. No any extra
credential is needed here, e.g.
curl -i -X 'POST' $WEBHOOK_URL
Also user can choose to take some extra parameters for the action execution
when triggering the webhook, e.g.
curl -i -X 'POST' $WEBHOOK_URL -H 'Content-type: application/json' --data
'{"params": {"count": 2}}'
4. Webhook middleware of Senlin service handles this post request and decrypt
the credential. Then it tries to validate the credential by querying a
token based on it from keystone. If succeed, the token will be added to this
post request and then send to next middleware in pipeline, usually keystone
auth_token. If not, an exception will be raised and this webhook triggering
fail.
5. Senlin engine receives the webhook triggering request and generates action
based on the information stored in webhook object, e.g. obj_type, obj_id
and action name;
6. Action is dispatched and scheduled by Senlin scheduler to finish the expected
operation, e.g. cluster scaling in/out.
Implementation
--------------
DB model
++++++++
A webhook DB object includes the following properties:
- id: the uuid of webhook
- name: the name of webhook (optional)
- user: the id of user who created the webhook
- project: the project id of user who created the webhook.
- domain: the domain of the user who created the webhook
- created_time: time of webhook was created
- deleted_time: time of webhook was deleted
- obj_id: the id of senlin entity(e.g. cluster) that the webhook bound to
- obj_type: the type of senlin entity that the webhook bound to
- action: action(e.g. scalingin, scalingout) of the target the webhook
bound to
- credential: credential that will be used to invoke target action API, e.g.
clusters/$cluster_id/action when webhook is triggered
- params: parameters that will be included when invoke the target action API,
e.g. adjustment of scaling operation