openstack
/
watcher
Code Issues Proposed changes

initial version 

Change-Id: I699e0ab082657880998d8618fe29eb7f56c6c661
This commit is contained in:
David TARDIVEL 2015-06-04 15:26:55 +02:00
parent 073c6e49cb
commit d14e057da1
316 changed files with 27260 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
.coveragerc Normal file
View File

@ -0,0 +1,7 @@
[run]
branch = True
source = watcher
omit = watcher/tests/*,watcher/openstack/*
[report]
ignore-errors = True

62
.gitignore vendored Normal file
View File

@ -0,0 +1,62 @@
*.py[cod]
# C extensions
*.so
# Packages
*.egg
*.egg-info
dist
build
eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg
lib
lib64
# Installer logs
pip-log.txt
# Unit test / coverage reports
.coverage
.tox
nosetests.xml
.testrepository
.venv
# Translations
*.mo
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
# Complexity
output/*.html
output/*/index.html
# Sphinx
doc/build
# pbr generates these
AUTHORS
ChangeLog
# Editors
*~
.*.swp
.*sw?
sftp-config.json
/.idea/
/cover/
.settings/
.eclipse
.project
.pydevproject

3
.mailmap Normal file
View File

@ -0,0 +1,3 @@
# Format is:
# <preferred e-mail> <other e-mail 1>
# <preferred e-mail> <other e-mail 2>

7
.testr.conf Normal file
View File

@ -0,0 +1,7 @@
[DEFAULT]
test_command=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./watcher/tests} $LISTOPT $IDOPTION
test_id_option=--load-list $IDFILE
test_list_option=--list

16
CONTRIBUTING.rst Normal file
View File

@ -0,0 +1,16 @@
If you would like to contribute to the development of OpenStack,
you must follow the steps in this page:
http://docs.openstack.org/infra/manual/developers.html
Once those steps have been completed, changes to OpenStack
should be submitted for review via the Gerrit tool, following
the workflow documented at:
http://docs.openstack.org/infra/manual/developers.html#development-workflow
Pull requests submitted through GitHub will be ignored.
Bugs should be filed on Launchpad, not GitHub:
https://bugs.launchpad.net/watcher

4
HACKING.rst Normal file
View File

@ -0,0 +1,4 @@
watcher Style Commandments
===============================================
Read the OpenStack Style Commandments http://docs.openstack.org/developer/hacking/

176
LICENSE Normal file
View File

@ -0,0 +1,176 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

6
MANIFEST.in Normal file
View File

@ -0,0 +1,6 @@
include AUTHORS
include ChangeLog
exclude .gitignore
exclude .gitreview
global-exclude *.pyc

10
README.rst Normal file
View File

@ -0,0 +1,10 @@
===============================
watcher
===============================
Watcher takes advantage of CEP and ML algorithms/metaheuristics to improve physical resources usage through better VM placement. Watcher can improve your cloud optimization by reducing energy footprint and increasing profits.
* Free software: Apache license
* Documentation: http://docs.openstack.org/developer/watcher
* Source: http://git.openstack.org/cgit/openstack/watcher
* Bugs: http://bugs.launchpad.net/watcher

2
babel.cfg Normal file
View File

@ -0,0 +1,2 @@
[python: **.py]

213
doc/source/cmds/watcher-db-manage.rst Normal file
View File

@ -0,0 +1,213 @@
.. _watcher-db-manage:
=============
watcher-db-manage
=============
The :command:`watcher-db-manage` utility is used to create the database schema
tables that the watcher services will use for storage. It can also be used to
upgrade (or downgrade) existing database tables when migrating between
different versions of watcher.
The `Alembic library <http://alembic.readthedocs.org>`_ is used to perform
the database migrations.
Options
=======
This is a partial list of the most useful options. To see the full list,
run the following::
watcher-db-manage --help
.. program:: watcher-db-manage
.. option:: -h, --help
Show help message and exit.
.. option:: --config-dir <DIR>
Path to a config directory with configuration files.
.. option:: --config-file <PATH>
Path to a configuration file to use.
.. option:: -d, --debug
Print debugging output.
.. option:: -v, --verbose
Print more verbose output.
.. option:: --version
Show the program's version number and exit.
.. option:: upgrade, downgrade, stamp, revision, version, create_schema
The :ref:`command <db-manage_cmds>` to run.
Usage
=====
Options for the various :ref:`commands <db-manage_cmds>` for
:command:`watcher-db-manage` are listed when the :option:`-h` or :option:`--help`
option is used after the command.
For example::
watcher-db-manage create_schema --help
Information about the database is read from the watcher configuration file
used by the API server and conductor services. This file must be specified
with the :option:`--config-file` option::
watcher-db-manage --config-file /path/to/watcher.conf create_schema
The configuration file defines the database backend to use with the
*connection* database option::
[database]
connection=mysql://root@localhost/watcher
If no configuration file is specified with the :option:`--config-file` option,
:command:`watcher-db-manage` assumes an SQLite database.
.. _db-manage_cmds:
Command Options
===============
:command:`watcher-db-manage` is given a command that tells the utility what actions
to perform. These commands can take arguments. Several commands are available:
.. _create_schema:
create_schema
-------------
.. program:: create_schema
.. option:: -h, --help
Show help for create_schema and exit.
This command will create database tables based on the most current version.
It assumes that there are no existing tables.
An example of creating database tables with the most recent version::
watcher-db-manage --config-file=/etc/watcher/watcher.conf create_schema
downgrade
---------
.. program:: downgrade
.. option:: -h, --help
Show help for downgrade and exit.
.. option:: --revision <REVISION>
The revision number you want to downgrade to.
This command will revert existing database tables to a previous version.
The version can be specified with the :option:`--revision` option.
An example of downgrading to table versions at revision 2581ebaf0cb2::
watcher-db-manage --config-file=/etc/watcher/watcher.conf downgrade --revision 2581ebaf0cb2
revision
--------
.. program:: revision
.. option:: -h, --help
Show help for revision and exit.
.. option:: -m <MESSAGE>, --message <MESSAGE>
The message to use with the revision file.
.. option:: --autogenerate
Compares table metadata in the application with the status of the database
and generates migrations based on this comparison.
This command will create a new revision file. You can use the
:option:`--message` option to comment the revision.
This is really only useful for watcher developers making changes that require
database changes. This revision file is used during database migration and
will specify the changes that need to be made to the database tables. Further
discussion is beyond the scope of this document.
stamp
-----
.. program:: stamp
.. option:: -h, --help
Show help for stamp and exit.
.. option:: --revision <REVISION>
The revision number.
This command will 'stamp' the revision table with the version specified with
the :option:`--revision` option. It will not run any migrations.
upgrade
-------
.. program:: upgrade
.. option:: -h, --help
Show help for upgrade and exit.
.. option:: --revision <REVISION>
The revision number to upgrade to.
This command will upgrade existing database tables to the most recent version,
or to the version specified with the :option:`--revision` option.
If there are no existing tables, then new tables are created, beginning
with the oldest known version, and successively upgraded using all of the
database migration files, until they are at the specified version. Note
that this behavior is different from the :ref:`create_schema` command
that creates the tables based on the most recent version.
An example of upgrading to the most recent table versions::
watcher-db-manage --config-file=/etc/watcher/watcher.conf upgrade
.. note::
This command is the default if no command is given to
:command:`watcher-db-manage`.
.. warning::
The upgrade command is not compatible with SQLite databases since it uses
ALTER TABLE commands to upgrade the database tables. SQLite supports only
a limited subset of ALTER TABLE.
version
-------
.. program:: version
.. option:: -h, --help
Show help for version and exit.
This command will output the current database version.

98
doc/source/conf.py Executable file
View File

@ -0,0 +1,98 @@
# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
from watcher import version as watcher_version
sys.path.insert(0, os.path.abspath('../..'))
# -- General configuration ----------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
'sphinx.ext.autodoc',
# 'sphinx.ext.intersphinx',
'sphinx.ext.viewcode',
'sphinxcontrib.httpdomain',
'sphinxcontrib.pecanwsme.rest',
'wsmeext.sphinxext',
'oslosphinx'
]
wsme_protocols = ['restjson']
# autodoc generation is a bit aggressive and a nuisance when doing heavy
# text edit cycles.
# execute "export SPHINX_DEBUG=1" in your terminal to disable
# The suffix of source filenames.
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'watcher'
copyright = u'2015, OpenStack Foundation'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
# The full version, including alpha/beta/rc tags.
release = watcher_version.version_info.release_string()
# The short X.Y version.
version = watcher_version.version_info.version_string()
# A list of ignored prefixes for module index sorting.
modindex_common_prefix = ['watcher.']
# If true, '()' will be appended to :func: etc. cross-reference text.
add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
add_module_names = True
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# -- Options for HTML output --------------------------------------------------
# The theme to use for HTML and HTML Help pages. Major themes that come with
# Sphinx are currently 'default' and 'sphinxdoc'.
# html_theme_path = ["."]
# html_theme = '_theme'
# html_static_path = ['static']
html_theme_options = {'incubating': True}
# Output file base name for HTML help builder.
htmlhelp_basename = '%sdoc' % project
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass
# [howto/manual]).
latex_documents = [
('index',
'%s.tex' % project,
u'%s Documentation' % project,
u'OpenStack Foundation', 'manual'),
]
# Example configuration for intersphinx: refer to the Python standard library.
# intersphinx_mapping = {'http://docs.python.org/': None}

222
doc/source/deploy/installation.rst Normal file
View File

@ -0,0 +1,222 @@
.. _installation:
========================
Development Installation
========================
Watcher development uses virtualenv to track and manage Python dependencies while in development and testing. This allows you to install all of the Python package dependencies in a virtual environment or “virtualenv”, instead of installing the packages at the system level.
Linux Systems
-------------
Install the prerequisite packages.
On Ubuntu (tested on 12.04-64 and 14.04-64)::
sudo apt-get install python-dev libssl-dev python-pip git-core libmysqlclient-dev libffi-dev
On Fedora-based distributions e.g., Fedora/RHEL/CentOS/Scientific Linux (tested on CentOS 6.5)::
sudo yum install python-virtualenv openssl-devel python-pip git gcc libffi-devel mysql-devel postgresql-devel
On openSUSE-based distributions (SLES 12, openSUSE 13.1, Factory or Tumbleweed)::
sudo zypper install gcc git libmysqlclient-devel libopenssl-devel postgresql-devel python-devel python-pip
Manually installing and using the virtualenv
--------------------------------------------
If you have `virtualenvwrapper <https://virtualenvwrapper.readthedocs.org/en/latest/install.html>`_ installed::
$ mkvirtualenv watcher
$ git clone https://git.openstack.org/openstack/stackforge/watcher
$ cd watcher && python setup.py install
$ pip install -r ./requirements.txt
To run a specific test, use a positional argument for the unit tests::
# run a specific test for Python 2.7
tox -epy27 -- tests.api
You may pass options to the test programs using positional arguments::
# run all the Python 2.7 unit tests (in parallel!)
tox -epy27 -- --parallel
To run only the pep8/flake8 syntax and style checks::
tox -epep8
Configure Identity Service for Watcher
--------------------------------------
#. Create the Watcher service user (eg ``watcher``). The service uses this to
authenticate with the Identity Service. Use the ``service`` project and
give the user the ``admin`` role::
keystone user-create --name=watcher --pass=WATCHER_PASSWORD --email=watcher@example.com
keystone user-role-add --user=watcher --tenant=service --role=admin
or
openstack user create --password WATCHER_PASSWORD --enable --email watcher@example.com watcher
openstack role add --project services --user watcher admin
#. You must register the Watcher Service with the Identity Service so that
other OpenStack services can locate it. To register the service::
keystone service-create --name=watcher --type=infra-optim \
--description="Infrastructure Optimization service"
or
openstack service create --name watcher infra-optim
#. Create the endpoints by replacing YOUR_REGION and WATCHER_API_IP with your region and your Watcher Service's API node::
keystone endpoint-create \
--service-id=the_service_id_above \
--publicurl=http://WATCHER_API_IP:9322 \
--internalurl=http://WATCHER_API_IP:9322 \
--adminurl=http://WATCHER_API_IP:9322
or
openstack endpoint create --region YOUR_REGION watcher public http://WATCHER_API_IP:9322
openstack endpoint create --region YOUR_REGION watcher admin http://WATCHER_API_IP:9322
openstack endpoint create --region YOUR_REGION watcher internal http://WATCHER_API_IP:9322
Set up the Database for Watcher
-------------------------------
The Watcher Service stores information in a database. This guide uses the
MySQL database that is used by other OpenStack services.
#. In MySQL, create an ``watcher`` database that is accessible by the
``watcher`` user. Replace WATCHER_DBPASSWORD
with the actual password::
# mysql -u root -p
mysql> CREATE DATABASE watcher CHARACTER SET utf8;
mysql> GRANT ALL PRIVILEGES ON watcher.* TO 'watcher'@'localhost' \
IDENTIFIED BY 'WATCHER_DBPASSWORD';
mysql> GRANT ALL PRIVILEGES ON watcher.* TO 'watcher'@'%' \
IDENTIFIED BY 'WATCHER_DBPASSWORD';
Configure the Watcher Service
=============================
The Watcher Service is configured via its configuration file. This file
is typically located at ``/etc/watcher/watcher.conf``. You can copy the file ``etc/watcher/watcher.conf.sample`` from the GIT repo to your server and update it.
Although some configuration options are mentioned here, it is recommended that
you review all the available options so that the Watcher Service is
configured for your needs.
#. The Watcher Service stores information in a database. This guide uses the
MySQL database that is used by other OpenStack services.
Configure the location of the database via the ``connection`` option. In the
following, replace WATCHER_DBPASSWORD with the password of your ``watcher``
user, and replace DB_IP with the IP address where the DB server is located::
[database]
...
# The SQLAlchemy connection string used to connect to the
# database (string value)
#connection=<None>
connection = mysql://watcher:WATCHER_DBPASSWORD@DB_IP/watcher?charset=utf8
#. Configure the Watcher Service to use the RabbitMQ message broker by
setting one or more of these options. Replace RABBIT_HOST with the
address of the RabbitMQ server.::
[DEFAULT]
...
# The RabbitMQ broker address where a single node is used
# (string value)
rabbit_host=RABBIT_HOST
# The RabbitMQ userid (string value)
#rabbit_userid=guest
# The RabbitMQ password (string value)
#rabbit_password=guest
# The RabbitMQ virtual host (string value)
#rabbit_virtual_host=/
#. Configure the Watcher Service to use these credentials with the Identity
Service. Replace IDENTITY_IP with the IP of the Identity server, and
replace WATCHER_PASSWORD with the password you chose for the ``watcher``
user in the Identity Service::
[DEFAULT]
...
# Method to use for authentication: noauth or keystone.
# (string value)
auth_strategy=keystone
...
[keystone_authtoken]
# Complete public Identity API endpoint (string value)
#auth_uri=<None>
auth_uri=http://IDENTITY_IP:5000/v3
# Complete admin Identity API endpoint. This should specify the
# unversioned root endpoint e.g. https://localhost:35357/ (string
# value)
#identity_uri = <None>
identity_uri = http://IDENTITY_IP:5000
# Keystone account username (string value)
#admin_user=<None>
admin_user=watcher
# Keystone account password (string value)
#admin_password=<None>
admin_password=WATCHER_DBPASSWORD
# Keystone service account tenant name to validate user tokens
# (string value)
#admin_tenant_name=admin
admin_tenant_name=KEYSTONE_SERVICE_PROJECT_NAME
# Directory used to cache files related to PKI tokens (string
# value)
#signing_dir=<None>
#. Create the Watcher Service database tables::
watcher-db-manage --config-file /etc/watcher/watcher.conf create_schema
#. Start the Watcher Service::
watcher-api && watcher-decision-engine && watcher-applier
===============
Important notes
===============
#. Watcher must have admin role on supervized users' projects created on your IAAS, in order to be able to migrate project's instances if required by Watcher audits:
keystone user-role-add --user=watcher --tenant=<USER_PROJECT_NAME> --role=admin
or
openstack role add --project <USER_PROJECT_NAME> --user watcher admin
#. Please check also your hypervisor configuration to handle correctly instance migration:
`OpenStack - Configure Migrations <http://docs.openstack.org/admin-guide-cloud/content/section_configuring-compute-migrations.html>`_

75
doc/source/deploy/user-guide.rst Normal file
View File

@ -0,0 +1,75 @@
.. _user-guide:
=================================
Welcome to the Watcher User Guide
=================================
In the `architecture <https://wiki.openstack.org/wiki/WatcherArchitecture>`_ you got information about how it works.
In this guide we're going to take you through the fundamentals of using Watcher.
Getting started with Watcher
----------------------------
This guide assumes you have a working installation of Watcher. If you get "watcher: command not found" you may have to verify your installation.
Please refer to installation guide.
In order to use Watcher, you have to configure your credentials suitable for watcher command-line tools.
I you need help on a specific command, you can use "watcher help COMMAND"
Seeing what the Watcher CLI can do ?
------------------------------------
We can see all of the commands available with Watcher CLI by running the watcher binary without options.
``watcher``
How do I run an audit of my cluster ?
-------------------------------------
First, you need to create an audit template. An audit template defines an optimization goal to achieve.
This goal should be declared in the Watcher service configuration file.
``$ watcher audit-template-create my_first_audit SERVERS_CONSOLIDATION``
If you get "You must provide a username via either --os-username or via env[OS_USERNAME]" you may have to verify your credentials
Then, you can create an audit. An audit is a request for optimizing your cluster depending on the specified goal.
You can launch an audit on your cluster by referencing the audit template (i.e. the goal) that you want to use.
- Get the audit template UUID::
``$ watcher audit-template-list``
- Start an audit based on this audit template settings::
``$ watcher audit-create -a <your_audit_template_uuid>``
Watcher service will compute an Action Plan composed of a list of potential optimization actions according to the goal to achieve.
You can see all of the goals available in the Watcher service configuration file, section ``[watcher_strategies]``.
- Wait until the Watcher audit has produced a new action plan, and get it::
``$ watcher action-plan-list --audit <the_audit_uuid>``
- Have a look on the list of optimization of this new action plan::
``$ watcher action-list --action-plan <the_action_plan_uuid>``
Once you've learnt how to create an Action Plan it's time to go further by applying it to your cluster :
- Execute the action plan::
``$ watcher action-plan-start <the_action_plan_uuid>``
You can follow the states of the actions by calling periodically ``watcher action-list``
Frequently Asked Questions
--------------------------
Under specific circumstances, you may encounter the following errors :
- Why do I get a 'Unable to establish connection to ....' error message ?
You typically get this error when one of the watcher services is not running.
You can make sure every Watcher service is running by launching the following command :
``
initctl list | grep watcher
watcher-api start/running, process 33062
watcher-decision-engine start/running, process 35511
watcher-applier start/running, process 47359
``

9
doc/source/dev/architecture.rst Normal file
View File

@ -0,0 +1,9 @@
.. _architecture:
===================
System Architecture
===================
Please go to `Wiki Watcher Architecture <https://wiki.openstack.org/wiki/WatcherArchitecture>`_
.. _API service: ../webapi/v1.html

56
doc/source/dev/contributing.rst Normal file
View File

@ -0,0 +1,56 @@
.. _contributing:
======================
Contributing to Watcher
======================
If you're interested in contributing to the Watcher project,
the following will help get you started.
Contributor License Agreement
-----------------------------
.. index::
single: license; agreement
In order to contribute to the Watcher project, you need to have
signed OpenStack's contributor's agreement.
.. seealso::
* http://docs.openstack.org/infra/manual/developers.html
* http://wiki.openstack.org/CLA
LaunchPad Project
-----------------
Most of the tools used for OpenStack depend on a launchpad.net ID for
authentication. After signing up for a launchpad account, join the
"openstack" team to have access to the mailing list and receive
notifications of important events.
.. seealso::
* http://launchpad.net
* http://launchpad.net/watcher
* http://launchpad.net/~openstack
Project Hosting Details
-------------------------
Bug tracker
http://launchpad.net/watcher
Mailing list (prefix subjects with ``[watcher]`` for faster responses)
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Wiki
http://wiki.openstack.org/Watcher
Code Hosting
https://github.com/openstack/watcher
Code Review
https://review.openstack.org/#/q/status:open+project:stackforge/watcher,n,z

60
doc/source/index.rst Normal file
View File

@ -0,0 +1,60 @@
============================================
Welcome to Watcher's developer documentation
============================================
Introduction
============
Watcher is an OpenStack project ...
The developer documentation provided here is continually kept up-to-date based
on the latest code, and may not represent the state of the project at any
specific prior release.
Developer Guide
===============
Introduction
------------
.. toctree::
:maxdepth: 1
dev/architecture
dev/contributing
API References
--------------
.. toctree::
:maxdepth: 1
webapi/v1
Admin Guide
===========
Overview
--------
.. toctree::
:maxdepth: 1
deploy/user-guide
deploy/installation
Commands
--------
.. toctree::
:maxdepth: 1
cmds/watcher-db-manage
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

1
doc/source/readme.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../../README.rst

7
doc/source/usage.rst Normal file
View File

@ -0,0 +1,7 @@
========
Usage
========
To use watcher in a project::
import watcher

61
doc/source/webapi/v1.rst Normal file
View File

@ -0,0 +1,61 @@
=====================
RESTful Web API (v1)
=====================
Audit Templates
===============
.. rest-controller:: watcher.api.controllers.v1.audit_template:AuditTemplatesController
:webprefix: /v1/audit_template
.. autotype:: watcher.api.controllers.v1.audit_template.AuditTemplateCollection
:members:
.. autotype:: watcher.api.controllers.v1.audit_template.AuditTemplate
:members:
Audits
======
.. rest-controller:: watcher.api.controllers.v1.audit:AuditsController
:webprefix: /v1/audits
.. autotype:: watcher.api.controllers.v1.audit.AuditCollection
:members:
.. autotype:: watcher.api.controllers.v1.audit.Audit
:members:
Links
=====
.. autotype:: watcher.api.controllers.link.Link
:members:
ActionPlans
===========
.. rest-controller:: watcher.api.controllers.v1.action_plan:ActionPlansController
:webprefix: /v1/action_plans
.. autotype:: watcher.api.controllers.v1.action_plan.ActionPlan
:members:
.. autotype:: watcher.api.controllers.v1.action_plan.ActionPlanCollection
:members:
Actions
=======
.. rest-controller:: watcher.api.controllers.v1.action:ActionsController
:webprefix: /v1/actions
.. autotype:: watcher.api.controllers.v1.action.ActionCollection
:members:
.. autotype:: watcher.api.controllers.v1.action.Action
:members:

5
etc/watcher/policy.json Normal file
View File

@ -0,0 +1,5 @@
{
"admin_api": "role:admin or role:administrator",
"show_password": "!",
"default": "rule:admin_api"
}

473
etc/watcher/watcher.conf.sample Normal file
View File

@ -0,0 +1,473 @@
[DEFAULT]
#
# From watcher
#
# Log output to standard error. (boolean value)
#use_stderr = true
# Format string to use for log messages with context. (string value)
#logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s
# Format string to use for log messages without context. (string
# value)
#logging_default_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s
# Data to append to log format when level is DEBUG. (string value)
#logging_debug_format_suffix = %(funcName)s %(pathname)s:%(lineno)d
# Prefix each line of exception output with this format. (string
# value)
#logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d TRACE %(name)s %(instance)s
# List of logger=LEVEL pairs. (list value)
#default_log_levels = amqp=WARN,amqplib=WARN,boto=WARN,qpid=WARN,sqlalchemy=WARN,suds=INFO,oslo.messaging=INFO,iso8601=WARN,requests.packages.urllib3.connectionpool=WARN,urllib3.connectionpool=WARN,websocket=WARN,keystonemiddleware=WARN,routes.middleware=WARN,stevedore=WARN
# Enables or disables publication of error events. (boolean value)
#publish_errors = false
# Enables or disables fatal status of deprecations. (boolean value)
#fatal_deprecations = false
# The format for an instance that is passed with the log message.
# (string value)
#instance_format = "[instance: %(uuid)s] "
# The format for an instance UUID that is passed with the log message.
# (string value)
#instance_uuid_format = "[instance: %(uuid)s] "
# Print debugging output (set logging level to DEBUG instead of
# default WARNING level). (boolean value)
#debug = false
# Print more verbose output (set logging level to INFO instead of
# default WARNING level). (boolean value)
#verbose = false
# The name of a logging configuration file. This file is appended to
# any existing logging configuration files. For details about logging
# configuration files, see the Python logging module documentation.
# (string value)
# Deprecated group/name - [DEFAULT]/log_config
#log_config_append = <None>
# DEPRECATED. A logging.Formatter log message format string which may
# use any of the available logging.LogRecord attributes. This option
# is deprecated. Please use logging_context_format_string and
# logging_default_format_string instead. (string value)
#log_format = <None>
# Format string for %%(asctime)s in log records. Default: %(default)s
# . (string value)
#log_date_format = %Y-%m-%d %H:%M:%S
# (Optional) Name of log file to output to. If no default is set,
# logging will go to stdout. (string value)
# Deprecated group/name - [DEFAULT]/logfile
#log_file = <None>
# (Optional) The base directory used for relative --log-file paths.
# (string value)
# Deprecated group/name - [DEFAULT]/logdir
#log_dir = <None>
# Use syslog for logging. Existing syslog format is DEPRECATED during
# I, and will change in J to honor RFC5424. (boolean value)
#use_syslog = false
# (Optional) Enables or disables syslog rfc5424 format for logging. If
# enabled, prefixes the MSG part of the syslog message with APP-NAME
# (RFC5424). The format without the APP-NAME is deprecated in I, and
# will be removed in J. (boolean value)
#use_syslog_rfc_format = false
# Syslog facility to receive log lines. (string value)
#syslog_log_facility = LOG_USER
[api]
#
# From watcher
#
# The port for the watcher API server (integer value)
#port = 9322
# The listen IP for the watcher API server (string value)
#host = 0.0.0.0
# The maximum number of items returned in a single response from a
# collection resource. (integer value)
#max_limit = 1000
[database]
#
# From oslo.db
#
# The file name to use with SQLite. (string value)
# Deprecated group/name - [DEFAULT]/sqlite_db
#sqlite_db = oslo.sqlite
# If True, SQLite uses synchronous mode. (boolean value)
# Deprecated group/name - [DEFAULT]/sqlite_synchronous
#sqlite_synchronous = true
# The back end to use for the database. (string value)
# Deprecated group/name - [DEFAULT]/db_backend
#backend = sqlalchemy
# The SQLAlchemy connection string to use to connect to the database.
# (string value)
# Deprecated group/name - [DEFAULT]/sql_connection
# Deprecated group/name - [DATABASE]/sql_connection
# Deprecated group/name - [sql]/connection
#connection = <None>
# The SQLAlchemy connection string to use to connect to the slave
# database. (string value)
#slave_connection = <None>
# The SQL mode to be used for MySQL sessions. This option, including
# the default, overrides any server-set SQL mode. To use whatever SQL
# mode is set by the server configuration, set this to no value.
# Example: mysql_sql_mode= (string value)
#mysql_sql_mode = TRADITIONAL
# Timeout before idle SQL connections are reaped. (integer value)
# Deprecated group/name - [DEFAULT]/sql_idle_timeout
# Deprecated group/name - [DATABASE]/sql_idle_timeout
# Deprecated group/name - [sql]/idle_timeout
#idle_timeout = 3600
# Minimum number of SQL connections to keep open in a pool. (integer
# value)
# Deprecated group/name - [DEFAULT]/sql_min_pool_size
# Deprecated group/name - [DATABASE]/sql_min_pool_size
#min_pool_size = 1
# Maximum number of SQL connections to keep open in a pool. (integer
# value)
# Deprecated group/name - [DEFAULT]/sql_max_pool_size
# Deprecated group/name - [DATABASE]/sql_max_pool_size
#max_pool_size = <None>
# Maximum number of database connection retries during startup. Set to
# -1 to specify an infinite retry count. (integer value)
# Deprecated group/name - [DEFAULT]/sql_max_retries
# Deprecated group/name - [DATABASE]/sql_max_retries
#max_retries = 10
# Interval between retries of opening a SQL connection. (integer
# value)
# Deprecated group/name - [DEFAULT]/sql_retry_interval
# Deprecated group/name - [DATABASE]/reconnect_interval
#retry_interval = 10
# If set, use this value for max_overflow with SQLAlchemy. (integer
# value)
# Deprecated group/name - [DEFAULT]/sql_max_overflow
# Deprecated group/name - [DATABASE]/sqlalchemy_max_overflow
#max_overflow = <None>
# Verbosity of SQL debugging information: 0=None, 100=Everything.
# (integer value)
# Deprecated group/name - [DEFAULT]/sql_connection_debug
#connection_debug = 0
# Add Python stack traces to SQL as comment strings. (boolean value)
# Deprecated group/name - [DEFAULT]/sql_connection_trace
#connection_trace = false
# If set, use this value for pool_timeout with SQLAlchemy. (integer
# value)
# Deprecated group/name - [DATABASE]/sqlalchemy_pool_timeout
#pool_timeout = <None>
# Enable the experimental use of database reconnect on connection
# lost. (boolean value)
#use_db_reconnect = false
# Seconds between retries of a database transaction. (integer value)
#db_retry_interval = 1
# If True, increases the interval between retries of a database
# operation up to db_max_retry_interval. (boolean value)
#db_inc_retry_interval = true
# If db_inc_retry_interval is set, the maximum seconds between retries
# of a database operation. (integer value)
#db_max_retry_interval = 10
# Maximum retries in case of connection error or deadlock error before
# error is raised. Set to -1 to specify an infinite retry count.
# (integer value)
#db_max_retries = 20
[keystone_authtoken]
#
# From keystonemiddleware.auth_token
#
# Complete public Identity API endpoint. (string value)
#auth_uri = <None>
# API version of the admin Identity API endpoint. (string value)
#auth_version = <None>
# Do not handle authorization requests within the middleware, but
# delegate the authorization decision to downstream WSGI components.
# (boolean value)
#delay_auth_decision = false
# Request timeout value for communicating with Identity API server.
# (integer value)
#http_connect_timeout = <None>
# How many times are we trying to reconnect when communicating with
# Identity API Server. (integer value)
#http_request_max_retries = 3
# Env key for the swift cache. (string value)
#cache = <None>
# Required if identity server requires client certificate (string
# value)
#certfile = <None>
# Required if identity server requires client certificate (string
# value)
#keyfile = <None>
# A PEM encoded Certificate Authority to use when verifying HTTPs
# connections. Defaults to system CAs. (string value)
#cafile = <None>
# Verify HTTPS connections. (boolean value)
#insecure = false
# Directory used to cache files related to PKI tokens. (string value)
#signing_dir = <None>
# Optionally specify a list of memcached server(s) to use for caching.
# If left undefined, tokens will instead be cached in-process. (list
# value)
# Deprecated group/name - [DEFAULT]/memcache_servers
#memcached_servers = <None>
# In order to prevent excessive effort spent validating tokens, the
# middleware caches previously-seen tokens for a configurable duration
# (in seconds). Set to -1 to disable caching completely. (integer
# value)
#token_cache_time = 300
# Determines the frequency at which the list of revoked tokens is
# retrieved from the Identity service (in seconds). A high number of
# revocation events combined with a low cache duration may
# significantly reduce performance. (integer value)
#revocation_cache_time = 10
# (Optional) If defined, indicate whether token data should be
# authenticated or authenticated and encrypted. Acceptable values are
# MAC or ENCRYPT. If MAC, token data is authenticated (with HMAC) in
# the cache. If ENCRYPT, token data is encrypted and authenticated in
# the cache. If the value is not one of these options or empty,
# auth_token will raise an exception on initialization. (string value)
#memcache_security_strategy = <None>
# (Optional, mandatory if memcache_security_strategy is defined) This
# string is used for key derivation. (string value)
#memcache_secret_key = <None>
# (Optional) Number of seconds memcached server is considered dead
# before it is tried again. (integer value)
#memcache_pool_dead_retry = 300
# (Optional) Maximum total number of open connections to every
# memcached server. (integer value)
#memcache_pool_maxsize = 10
# (Optional) Socket timeout in seconds for communicating with a
# memcached server. (integer value)
#memcache_pool_socket_timeout = 3
# (Optional) Number of seconds a connection to memcached is held
# unused in the pool before it is closed. (integer value)
#memcache_pool_unused_timeout = 60
# (Optional) Number of seconds that an operation will wait to get a
# memcached client connection from the pool. (integer value)
#memcache_pool_conn_get_timeout = 10
# (Optional) Use the advanced (eventlet safe) memcached client pool.
# The advanced pool will only work under python 2.x. (boolean value)
#memcache_use_advanced_pool = false
# (Optional) Indicate whether to set the X-Service-Catalog header. If
# False, middleware will not ask for service catalog on token
# validation and will not set the X-Service-Catalog header. (boolean
# value)
#include_service_catalog = true
# Used to control the use and type of token binding. Can be set to:
# "disabled" to not check token binding. "permissive" (default) to
# validate binding information if the bind type is of a form known to
# the server and ignore it if not. "strict" like "permissive" but if
# the bind type is unknown the token will be rejected. "required" any
# form of token binding is needed to be allowed. Finally the name of a
# binding method that must be present in tokens. (string value)
#enforce_token_bind = permissive
# If true, the revocation list will be checked for cached tokens. This
# requires that PKI tokens are configured on the identity server.
# (boolean value)
#check_revocations_for_cached = false
# Hash algorithms to use for hashing PKI tokens. This may be a single
# algorithm or multiple. The algorithms are those supported by Python
# standard hashlib.new(). The hashes will be tried in the order given,
# so put the preferred one first for performance. The result of the
# first hash will be stored in the cache. This will typically be set
# to multiple values only while migrating from a less secure algorithm
# to a more secure one. Once all the old tokens are expired this
# option should be set to a single value for better performance. (list
# value)
#hash_algorithms = md5
# Prefix to prepend at the beginning of the path. Deprecated, use
# identity_uri. (string value)
#auth_admin_prefix =
# Host providing the admin Identity API endpoint. Deprecated, use
# identity_uri. (string value)
#auth_host = 127.0.0.1
# Port of the admin Identity API endpoint. Deprecated, use
# identity_uri. (integer value)
#auth_port = 35357
# Protocol of the admin Identity API endpoint (http or https).
# Deprecated, use identity_uri. (string value)
#auth_protocol = https
# Complete admin Identity API endpoint. This should specify the
# unversioned root endpoint e.g. https://localhost:35357/ (string
# value)
#identity_uri = <None>
# This option is deprecated and may be removed in a future release.
# Single shared secret with the Keystone configuration used for
# bootstrapping a Keystone installation, or otherwise bypassing the
# normal authentication process. This option should not be used, use
# `admin_user` and `admin_password` instead. (string value)
#admin_token = <None>
# Service username. (string value)
#admin_user = <None>
# Service user password. (string value)
#admin_password = <None>
# Service tenant name. (string value)
#admin_tenant_name = admin
[watcher_applier]
#
# From watcher
#
# The number of worker (integer value)
#applier_worker = 1
# The topic name used forcontrol events, this topic used for rpc call
# (string value)
#topic_control = watcher.applier.control
# The topic name used for status events, this topic is used so as to
# notifythe others components of the system (string value)
#topic_status = watcher.applier.status
# The identifier used by watcher module on the message broker (string
# value)
#publisher_id = watcher.applier.api
[watcher_decision_engine]
#
# From watcher
#
# The topic name used forcontrol events, this topic used for rpc call
# (string value)
#topic_control = watcher.decision.control
# The topic name used for status events, this topic is used so as to
# notifythe others components of the system (string value)
#topic_status = watcher.decision.status
# The identifier used by watcher module on the message broker (string
# value)
#publisher_id = watcher.decision.api
[watcher_goals]
#
# From watcher
#
# Goals used for the optimization (dict value)
#goals = BALANCE_LOAD:basic,MINIMIZE_ENERGY_CONSUMPTION:basic,MINIMIZE_LICENSING_COST:basic,PREPARE_PLANNED_OPERATION:basic,SERVERS_CONSOLIDATION:basic
[watcher_messaging]
#
# From watcher
#
# The name of the driver used by oslo messaging (string value)
#notifier_driver = messaging
# The name of a message executor, forexample: eventlet, blocking
# (string value)
#executor = eventlet
# The protocol used by the message broker, for example rabbit (string
# value)
#protocol = rabbit
# The username used by the message broker (string value)
#user = guest
# The password of user used by the message broker (string value)
#password = guest
# The host where the message brokeris installed (string value)
#host = localhost
# The port used bythe message broker (string value)
#port = 5672
# The virtual host used by the message broker (string value)
#virtual_host =
[watcher_strategies]
#
# From watcher
#
# Strategies used for the optimization (dict value)
#strategies = basic:watcher.decision_engine.strategies.basic_consolidation::BasicConsolidation

9
openstack-common.conf Normal file
View File

@ -0,0 +1,9 @@
[DEFAULT]
# The list of modules to copy from oslo-incubator.git
module=policy
module=versionutils
# The base module to hold the copy of openstack.common
base=watcher

29
requirements.txt Normal file
View File

@ -0,0 +1,29 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
pbr>=0.6,!=0.7,<1.0
oslo.config>=1.4.0
PasteDeploy==1.5.2
oslo.messaging
oslo.db
oslo.log
oslo.i18n
oslo.utils>=1.2.0 # Apache-2.0
pecan>=0.8
keystonemiddleware>=1.0.0
six>=1.7.0,<=1.9.0
sqlalchemy
stevedore>=1.1.0 # Apache-2.0
WSME>=0.6
jsonpatch>=1.1
enum34==1.0.4
# watcher Applier
python-novaclient
python-openstackclient
python-neutronclient
python-glanceclient
python-cinderclient
# Decision Engine
python-ceilometerclient

66
setup.cfg Normal file
View File

@ -0,0 +1,66 @@
[metadata]
name = watcher
summary = Watcher takes advantage of CEP and ML algorithms/metaheuristics to improve physical resources usage through better VM placement. Watcher can improve your cloud optimization by reducing energy footprint and increasing profits.
description-file =
README.rst
author = OpenStack
author-email = openstack-dev@lists.openstack.org
home-page = http://www.openstack.org/
classifier =
Environment :: OpenStack
Intended Audience :: Information Technology
Intended Audience :: System Administrators
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Programming Language :: Python :: 2.6
Programming Language :: Python :: 3
Programming Language :: Python :: 3.3
Programming Language :: Python :: 3.4
[files]
packages =
watcher
[global]
setup-hooks =
pbr.hooks.setup_hook
[entry_points]
oslo.config.opts =
watcher = watcher.opts:list_opts
console_scripts =
watcher-api = watcher.cmd.api:main
watcher-db-manage = watcher.cmd.dbmanage:main
watcher-decision-engine = watcher.cmd.decisionengine:main
watcher-applier = watcher.cmd.applier:main
watcher.database.migration_backend =
sqlalchemy = watcher.db.sqlalchemy.migration
[build_sphinx]
source-dir = doc/source
build-dir = doc/build
all_files = 1
[upload_sphinx]
upload-dir = doc/build/html
[compile_catalog]
directory = watcher/locale
domain = watcher
[update_catalog]
domain = watcher
output_dir = watcher/locale
input_file = watcher/locale/watcher.pot
[extract_messages]
keywords = _ gettext ngettext l_ lazy_gettext
mapping_file = babel.cfg
output_file = watcher/locale/watcher.pot

30
setup.py Executable file
View File

@ -0,0 +1,30 @@
#!/usr/bin/env python
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT
import setuptools
# In python < 2.7.4, a lazy loading of package `pbr` will break
# setuptools if some other modules registered functions in `atexit`.
# solution from: http://bugs.python.org/issue15881#msg170215
try:
import multiprocessing # noqa
except ImportError:
pass
setuptools.setup(
setup_requires=['pbr'],
pbr=True)

18
test-requirements.txt Normal file
View File

@ -0,0 +1,18 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
hacking<0.11,>=0.10.0
coverage>=3.6
discover
python-subunit>=0.0.18
oslotest>=1.2.0 # Apache-2.0
testrepository>=0.0.18
testscenarios>=0.4
testtools>=0.9.36,!=1.2.0
mock>=1.0
# Doc requirements
sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3
oslosphinx>=2.2.0 # Apache-2.0
sphinxcontrib-pecanwsme>=0.8

44
tox.ini Normal file
View File

@ -0,0 +1,44 @@
[tox]
minversion = 1.6
envlist = py33,py34,py26,py27,pypy,pep8
skipsdist = True
[testenv]
usedevelop = True
install_command = pip install -U {opts} {packages}
setenv =
VIRTUAL_ENV={envdir}
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
commands = python setup.py testr --slowest --testr-args='{posargs}'
[testenv:pep8]
commands = flake8
[testenv:venv]
commands = {posargs}
[testenv:cover]
commands = python setup.py testr --coverage --testr-args='{posargs}'
[testenv:docs]
commands = python setup.py build_sphinx
[testenv:debug]
commands = oslo_debug_helper {posargs}
[testenv:config]
sitepackages = False
commands =
oslo-config-generator --namespace watcher \
--namespace keystonemiddleware.auth_token \
--namespace oslo.db \
--output-file etc/watcher/watcher.conf
[flake8]
# E123, E125 skipped as they are invalid PEP-8.
show-source=True
ignore=E123,E125,H404,H405,H305
builtins= _
exclude=.venv,.git,.tox,dist,doc,*openstack/common*,*lib/python*,*egg,build,*sqlalchemy/alembic/versions/*

19
watcher/__init__.py Normal file
View File

@ -0,0 +1,19 @@
# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import pbr.version
__version__ = pbr.version.VersionInfo(
'watcher').version_string()

6
watcher/api/README.md Normal file
View File

@ -0,0 +1,6 @@
# Watcher API
This component implements the REST API provided by the Watcher system to the external world. It enables a cluster administrator to control and monitor the Watcher system via any interaction mechanism connected to this API :
* CLI
* Horizon plugin
* Python SDK

0
watcher/api/__init__.py Normal file
View File

49
watcher/api/acl.py Normal file
View File

@ -0,0 +1,49 @@
# -*- encoding: utf-8 -*-
#
# Copyright © 2012 New Dream Network, LLC (DreamHost)
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Access Control Lists (ACL's) control access the API server."""
from oslo_config import cfg
from watcher.api.middleware import auth_token
AUTH_OPTS = [
cfg.BoolOpt('enable_authentication',
default=True,
help='This option enables or disables user authentication '
'via keystone. Default value is True.'),
]
CONF = cfg.CONF
CONF.register_opts(AUTH_OPTS)
def install(app, conf, public_routes):
"""Install ACL check on application.
:param app: A WSGI applicatin.
:param conf: Settings. Dict'ified and passed to keystonemiddleware
:param public_routes: The list of the routes which will be allowed to
access without authentication.
:return: The same WSGI application with ACL installed.
"""
if not cfg.CONF.get('enable_authentication'):
return app
return auth_token.AuthTokenMiddleware(app,
conf=dict(conf),
public_api_routes=public_routes)

70
watcher/api/app.py Normal file
View File

@ -0,0 +1,70 @@
# -*- encoding: utf-8 -*-
# Copyright © 2012 New Dream Network, LLC (DreamHost)
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo_config import cfg
import pecan
from watcher.api import acl
from watcher.api import config as api_config
from watcher.api import middleware
from watcher.decision_engine.framework.strategy import strategy_selector
# Register options for the service
API_SERVICE_OPTS = [
cfg.IntOpt('port',
default=9322,
help='The port for the watcher API server'),
cfg.StrOpt('host',
default='0.0.0.0',
help='The listen IP for the watcher API server'),
cfg.IntOpt('max_limit',
default=1000,
help='The maximum number of items returned in a single '
'response from a collection resource.')
]
CONF = cfg.CONF
opt_group = cfg.OptGroup(name='api',
title='Options for the watcher-api service')
CONF.register_group(opt_group)
CONF.register_opts(API_SERVICE_OPTS, opt_group)
CONF.register_opts(strategy_selector.WATCHER_GOALS_OPTS)
def get_pecan_config():
# Set up the pecan configuration
filename = api_config.__file__.replace('.pyc', '.py')
return pecan.configuration.conf_from_file(filename)
def setup_app(config=None):
if not config:
config = get_pecan_config()
app_conf = dict(config.app)
app = pecan.make_app(
app_conf.pop('root'),
logging=getattr(config, 'logging', {}),
debug=CONF.debug,
wrap_app=middleware.ParsableErrorMiddleware,
**app_conf
)
return acl.install(app, CONF, config.app.acl_public_routes)

46
watcher/api/config.py Normal file
View File

@ -0,0 +1,46 @@
# -*- encoding: utf-8 -*-
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from oslo_config import cfg
from watcher.api import hooks
# Server Specific Configurations
# See https://pecan.readthedocs.org/en/latest/configuration.html#server-configuration # noqa
server = {
'port': '9322',
'host': '0.0.0.0'
}
# Pecan Application Configurations
# See https://pecan.readthedocs.org/en/latest/configuration.html#application-configuration # noqa
app = {
'root': 'watcher.api.controllers.root.RootController',
'modules': ['watcher.api'],
'hooks': [
hooks.ContextHook(),
hooks.NoExceptionTracebackHook(),
],
'static_root': '%(confdir)s/public',
'enable_acl': True,
'acl_public_routes': [
'/',
],
}
# WSME Configurations
# See https://wsme.readthedocs.org/en/latest/integrate.html#configuration
wsme = {
'debug': cfg.CONF.debug,
}

0
watcher/api/controllers/__init__.py Normal file
View File

51
watcher/api/controllers/base.py Normal file
View File

@ -0,0 +1,51 @@
# -*- encoding: utf-8 -*-
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import datetime
import wsme
from wsme import types as wtypes
class APIBase(wtypes.Base):
created_at = wsme.wsattr(datetime.datetime, readonly=True)
"""The time in UTC at which the object is created"""
updated_at = wsme.wsattr(datetime.datetime, readonly=True)
"""The time in UTC at which the object is updated"""
deleted_at = wsme.wsattr(datetime.datetime, readonly=True)
"""The time in UTC at which the object is deleted"""
def as_dict(self):
"""Render this object as a dict of its fields."""
return dict((k, getattr(self, k))
for k in self.fields
if hasattr(self, k) and
getattr(self, k) != wsme.Unset)
def unset_fields_except(self, except_list=None):
"""Unset fields so they don't appear in the message body.
:param except_list: A list of fields that won't be touched.
"""
if except_list is None:
except_list = []
for k in self.as_dict():
if k not in except_list:
setattr(self, k, wsme.Unset)

60
watcher/api/controllers/link.py Normal file
View File

@ -0,0 +1,60 @@
# -*- encoding: utf-8 -*-
# Copyright 2013 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import pecan
from wsme import types as wtypes
from watcher.api.controllers import base
def build_url(resource, resource_args, bookmark=False, base_url=None):
if base_url is None:
base_url = pecan.request.host_url
template = '%(url)s/%(res)s' if bookmark else '%(url)s/v1/%(res)s'
# FIXME(lucasagomes): I'm getting a 404 when doing a GET on
# a nested resource that the URL ends with a '/'.
# https://groups.google.com/forum/#!topic/pecan-dev/QfSeviLg5qs
template += '%(args)s' if resource_args.startswith('?') else '/%(args)s'
return template % {'url': base_url, 'res': resource, 'args': resource_args}
class Link(base.APIBase):
"""A link representation."""
href = wtypes.text
"""The url of a link."""
rel = wtypes.text
"""The name of a link."""
type = wtypes.text
"""Indicates the type of document/link."""
@staticmethod
def make_link(rel_name, url, resource, resource_args,
bookmark=False, type=wtypes.Unset):
href = build_url(resource, resource_args,
bookmark=bookmark, base_url=url)
return Link(href=href, rel=rel_name, type=type)
@classmethod
def sample(cls):
sample = cls(href="http://localhost:6385/chassis/"
"eaaca217-e7d8-47b4-bb41-3f99f20eed89",
rel="bookmark")
return sample

98
watcher/api/controllers/root.py Normal file
View File

@ -0,0 +1,98 @@
# -*- encoding: utf-8 -*-
#
# Copyright © 2012 New Dream Network, LLC (DreamHost)
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import pecan
from pecan import rest
from wsme import types as wtypes
import wsmeext.pecan as wsme_pecan
from watcher.api.controllers import base
from watcher.api.controllers import link
from watcher.api.controllers import v1
class Version(base.APIBase):
"""An API version representation."""
id = wtypes.text
"""The ID of the version, also acts as the release number"""
links = [link.Link]
"""A Link that point to a specific version of the API"""
@staticmethod
def convert(id):
version = Version()
version.id = id
version.links = [link.Link.make_link('self', pecan.request.host_url,
id, '', bookmark=True)]
return version
class Root(base.APIBase):
name = wtypes.text
"""The name of the API"""
description = wtypes.text
"""Some information about this API"""
versions = [Version]
"""Links to all the versions available in this API"""
default_version = Version
"""A link to the default version of the API"""
@staticmethod
def convert():
root = Root()
root.name = "OpenStack Watcher API"
root.description = ("Watcher is an OpenStack project which aims to "
"to improve physical resources usage through "
"better VM placement.")
root.versions = [Version.convert('v1')]
root.default_version = Version.convert('v1')
return root
class RootController(rest.RestController):
_versions = ['v1']
"""All supported API versions"""
_default_version = 'v1'
"""The default API version"""
v1 = v1.Controller()
@wsme_pecan.wsexpose(Root)
def get(self):
# NOTE: The reason why convert() it's being called for every
# request is because we need to get the host url from
# the request object to make the links.
return Root.convert()
@pecan.expose()
def _route(self, args):
"""Overrides the default routing behavior.
It redirects the request to the default version of the watcher API
if the version number is not specified in the url.
"""
if args[0] and args[0] not in self._versions:
args = [self._default_version] + args
return super(RootController, self)._route(args)

166
watcher/api/controllers/v1/__init__.py Normal file
View File

@ -0,0 +1,166 @@
# -*- encoding: utf-8 -*-
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
Version 1 of the Watcher API
NOTE: IN PROGRESS AND NOT FULLY IMPLEMENTED.
"""
import datetime
import pecan
from pecan import rest
import wsme
from wsme import types as wtypes
import wsmeext.pecan as wsme_pecan
from watcher.api.controllers import link
from watcher.api.controllers.v1 import action
from watcher.api.controllers.v1 import action_plan
from watcher.api.controllers.v1 import audit
from watcher.api.controllers.v1 import audit_template
class APIBase(wtypes.Base):
created_at = wsme.wsattr(datetime.datetime, readonly=True)
"""The time in UTC at which the object is created"""
updated_at = wsme.wsattr(datetime.datetime, readonly=True)
"""The time in UTC at which the object is updated"""
deleted_at = wsme.wsattr(datetime.datetime, readonly=True)
"""The time in UTC at which the object is deleted"""
def as_dict(self):
"""Render this object as a dict of its fields."""
return dict((k, getattr(self, k))
for k in self.fields
if hasattr(self, k) and
getattr(self, k) != wsme.Unset)
def unset_fields_except(self, except_list=None):
"""Unset fields so they don't appear in the message body.
:param except_list: A list of fields that won't be touched.
"""
if except_list is None:
except_list = []
for k in self.as_dict():
if k not in except_list:
setattr(self, k, wsme.Unset)
class MediaType(APIBase):
"""A media type representation."""
base = wtypes.text
type = wtypes.text
def __init__(self, base, type):
self.base = base
self.type = type
class V1(APIBase):
"""The representation of the version 1 of the API."""
id = wtypes.text
"""The ID of the version, also acts as the release number"""
media_types = [MediaType]
"""An array of supcontainersed media types for this version"""
audit_templates = [link.Link]
"""Links to the audit templates resource"""
audits = [link.Link]
"""Links to the audits resource"""
actions = [link.Link]
"""Links to the actions resource"""
action_plans = [link.Link]
"""Links to the action plans resource"""
links = [link.Link]
"""Links that point to a specific URL for this version and documentation"""
@staticmethod
def convert():
v1 = V1()
v1.id = "v1"
v1.links = [link.Link.make_link('self', pecan.request.host_url,
'v1', '', bookmark=True),
link.Link.make_link('describedby',
'http://docs.openstack.org',
'developer/watcher/dev',
'api-spec-v1.html',
bookmark=True, type='text/html')
]
v1.media_types = [MediaType('application/json',
'application/vnd.openstack.watcher.v1+json')]
v1.audit_templates = [link.Link.make_link('self',
pecan.request.host_url,
'audit_templates', ''),
link.Link.make_link('bookmark',
pecan.request.host_url,
'audit_templates', '',
bookmark=True)
]
v1.audits = [link.Link.make_link('self', pecan.request.host_url,
'audits', ''),
link.Link.make_link('bookmark',
pecan.request.host_url,
'audits', '',
bookmark=True)
]
v1.actions = [link.Link.make_link('self', pecan.request.host_url,
'actions', ''),
link.Link.make_link('bookmark',
pecan.request.host_url,
'actions', '',
bookmark=True)
]
v1.action_plans = [link.Link.make_link(
'self', pecan.request.host_url, 'action_plans', ''),
link.Link.make_link('bookmark',
pecan.request.host_url,
'action_plans', '',
bookmark=True)
]
return v1
class Controller(rest.RestController):
"""Version 1 API controller root."""
audits = audit.AuditsController()
audit_templates = audit_template.AuditTemplatesController()
actions = action.ActionsController()
action_plans = action_plan.ActionPlansController()
@wsme_pecan.wsexpose(V1)
def get(self):
# NOTE: The reason why convert() it's being called for every
# request is because we need to get the host url from
# the request object to make the links.
return V1.convert()
__all__ = (Controller)

397
watcher/api/controllers/v1/action.py Normal file
View File

@ -0,0 +1,397 @@
# -*- encoding: utf-8 -*-
# Copyright 2013 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import datetime
import pecan
from pecan import rest
import wsme
from wsme import types as wtypes
import wsmeext.pecan as wsme_pecan
from watcher.api.controllers import base
from watcher.api.controllers import link
from watcher.api.controllers.v1 import collection
from watcher.api.controllers.v1 import types
from watcher.api.controllers.v1 import utils as api_utils
from watcher.common import exception
from watcher import objects
class ActionPatchType(types.JsonPatchType):
@staticmethod
def mandatory_attrs():
return []
class Action(base.APIBase):
"""API representation of a action.
This class enforces type checking and value constraints, and converts
between the internal object model and the API representation of a action.
"""
_action_plan_uuid = None
_next_uuid = None
def _get_action_plan_uuid(self):
return self._action_plan_uuid
def _set_action_plan_uuid(self, value):
if value == wtypes.Unset:
self._action_plan_uuid = wtypes.Unset
elif value and self._action_plan_uuid != value:
try:
action_plan = objects.ActionPlan.get(
pecan.request.context, value)
self._action_plan_uuid = action_plan.uuid
self.action_plan_id = action_plan.id
except exception.ActionPlanNotFound:
self._action_plan_uuid = None
def _get_next_uuid(self):
return self._next_uuid
def _set_next_uuid(self, value):
if value == wtypes.Unset:
self._next_uuid = wtypes.Unset
elif value and self._next_uuid != value:
try:
action_next = objects.Action.get(
pecan.request.context, value)
self._next_uuid = action_next.uuid
self.next = action_next.id
except exception.ActionNotFound:
self.action_next_uuid = None
# raise e
uuid = types.uuid
"""Unique UUID for this action"""
action_plan_uuid = wsme.wsproperty(types.uuid, _get_action_plan_uuid,
_set_action_plan_uuid,
mandatory=True)
"""The action plan this action belongs to """
description = wtypes.text
"""Description of this action"""
state = wtypes.text
"""This audit state"""
alarm = types.uuid
"""An alarm UUID related to this action"""
applies_to = wtypes.text
"""Applies to"""
src = wtypes.text
"""Hypervisor source"""
dst = wtypes.text
"""Hypervisor source"""
action_type = wtypes.text
"""Action type"""
parameter = wtypes.text
"""Additionnal parameter"""
next_uuid = wsme.wsproperty(types.uuid, _get_next_uuid,
_set_next_uuid,
mandatory=True)
"""This next action UUID"""
links = wsme.wsattr([link.Link], readonly=True)
"""A list containing a self link and associated action links"""
def __init__(self, **kwargs):
super(Action, self).__init__()
self.fields = []
fields = list(objects.Action.fields)
# audit_template_uuid is not part of objects.Audit.fields
# because it's an API-only attribute.
fields.append('action_plan_uuid')
fields.append('next_uuid')
for field in fields:
# Skip fields we do not expose.
if not hasattr(self, field):
continue
self.fields.append(field)
setattr(self, field, kwargs.get(field, wtypes.Unset))
self.fields.append('action_plan_id')
setattr(self, 'action_plan_uuid', kwargs.get('action_plan_id',
wtypes.Unset))
setattr(self, 'next_uuid', kwargs.get('next',
wtypes.Unset))
@staticmethod
def _convert_with_links(action, url, expand=True):
if not expand:
action.unset_fields_except(['uuid', 'state', 'next', 'next_uuid',
'action_plan_uuid', 'action_plan_id',
'action_type'])
action.links = [link.Link.make_link('self', url,
'actions', action.uuid),
link.Link.make_link('bookmark', url,
'actions', action.uuid,
bookmark=True)
]
return action
@classmethod
def convert_with_links(cls, action, expand=True):
action = Action(**action.as_dict())
return cls._convert_with_links(action, pecan.request.host_url, expand)
@classmethod
def sample(cls, expand=True):
sample = cls(uuid='27e3153e-d5bf-4b7e-b517-fb518e17f34c',
description='action description',
state='PENDING',
alarm=None,
created_at=datetime.datetime.utcnow(),
deleted_at=None,
updated_at=datetime.datetime.utcnow())
sample._action_plan_uuid = '7ae81bb3-dec3-4289-8d6c-da80bd8001ae'
sample._next_uuid = '7ae81bb3-dec3-4289-8d6c-da80bd8001ae'
return cls._convert_with_links(sample, 'http://localhost:9322', expand)
class ActionCollection(collection.Collection):
"""API representation of a collection of actions."""
actions = [Action]
"""A list containing actions objects"""
def __init__(self, **kwargs):
self._type = 'actions'
@staticmethod
def convert_with_links(actions, limit, url=None, expand=False,
**kwargs):
collection = ActionCollection()
collection.actions = [Action.convert_with_links(p, expand)
for p in actions]
if 'sort_key' in kwargs:
reverse = False
if kwargs['sort_key'] == 'next_uuid':
if 'sort_dir' in kwargs:
reverse = True if kwargs['sort_dir'] == 'desc' else False
collection.actions = sorted(
collection.actions,
key=lambda action: action.next_uuid,
reverse=reverse)
collection.next = collection.get_next(limit, url=url, **kwargs)
return collection
@classmethod
def sample(cls):
sample = cls()
sample.actions = [Action.sample(expand=False)]
return sample
class ActionsController(rest.RestController):
"""REST controller for Actions."""
def __init__(self):
super(ActionsController, self).__init__()
from_actions = False
"""A flag to indicate if the requests to this controller are coming
from the top-level resource Actions."""
_custom_actions = {
'detail': ['GET'],
}
def _get_actions_collection(self, marker, limit,
sort_key, sort_dir, expand=False,
resource_url=None,
action_plan_uuid=None, audit_uuid=None):
limit = api_utils.validate_limit(limit)
sort_dir = api_utils.validate_sort_dir(sort_dir)
marker_obj = None
if marker:
marker_obj = objects.Action.get_by_uuid(pecan.request.context,
marker)
filters = {}
if action_plan_uuid:
filters['action_plan_uuid'] = action_plan_uuid
if audit_uuid:
filters['audit_uuid'] = audit_uuid
if sort_key == 'next_uuid':
sort_db_key = None
else:
sort_db_key = sort_key
actions = objects.Action.list(pecan.request.context,
limit,
marker_obj, sort_key=sort_db_key,
sort_dir=sort_dir,
filters=filters)
return ActionCollection.convert_with_links(actions, limit,
url=resource_url,
expand=expand,
sort_key=sort_key,
sort_dir=sort_dir)
@wsme_pecan.wsexpose(ActionCollection, types.uuid, types.uuid,
int, wtypes.text, wtypes.text, types.uuid,
types.uuid)
def get_all(self, action_uuid=None, marker=None, limit=None,
sort_key='id', sort_dir='asc', action_plan_uuid=None,
audit_uuid=None):
"""Retrieve a list of actions.
:param marker: pagination marker for large data sets.
:param limit: maximum number of resources to return in a single result.
:param sort_key: column to sort results by. Default: id.
:param sort_dir: direction to sort. "asc" or "desc". Default: asc.
:param action_plan_uuid: Optional UUID of an action plan,
to get only actions for that action plan.
:param audit_uuid: Optional UUID of an audit,
to get only actions for that audit.
"""
if action_plan_uuid and audit_uuid:
raise exception.ActionFilterCombinationProhibited
return self._get_actions_collection(
marker, limit, sort_key, sort_dir,
action_plan_uuid=action_plan_uuid, audit_uuid=audit_uuid)
@wsme_pecan.wsexpose(ActionCollection, types.uuid,
types.uuid, int, wtypes.text, wtypes.text,
types.uuid, types.uuid)
def detail(self, action_uuid=None, marker=None, limit=None,
sort_key='id', sort_dir='asc', action_plan_uuid=None,
audit_uuid=None):
"""Retrieve a list of actions with detail.
:param action_uuid: UUID of a action, to get only actions for that
action.
:param marker: pagination marker for large data sets.
:param limit: maximum number of resources to return in a single result.
:param sort_key: column to sort results by. Default: id.
:param sort_dir: direction to sort. "asc" or "desc". Default: asc.
:param action_plan_uuid: Optional UUID of an action plan,
to get only actions for that action plan.
:param audit_uuid: Optional UUID of an audit,
to get only actions for that audit.
"""
# NOTE(lucasagomes): /detail should only work agaist collections
parent = pecan.request.path.split('/')[:-1][-1]
if parent != "actions":
raise exception.HTTPNotFound
if action_plan_uuid and audit_uuid:
raise exception.ActionFilterCombinationProhibited
expand = True
resource_url = '/'.join(['actions', 'detail'])
return self._get_actions_collection(
marker, limit, sort_key, sort_dir, expand, resource_url,
action_plan_uuid=action_plan_uuid, audit_uuid=audit_uuid)
@wsme_pecan.wsexpose(Action, types.uuid)
def get_one(self, action_uuid):
"""Retrieve information about the given action.
:param action_uuid: UUID of a action.
"""
if self.from_actions:
raise exception.OperationNotPermitted
action = objects.Action.get_by_uuid(pecan.request.context,
action_uuid)
return Action.convert_with_links(action)
@wsme_pecan.wsexpose(Action, body=Action, status_code=201)
def post(self, action):
"""Create a new action.
:param action: a action within the request body.
"""
if self.from_actions:
raise exception.OperationNotPermitted
action_dict = action.as_dict()
context = pecan.request.context
new_action = objects.Action(context, **action_dict)
new_action.create(context)
# Set the HTTP Location Header
pecan.response.location = link.build_url('actions', new_action.uuid)
return Action.convert_with_links(new_action)
@wsme.validate(types.uuid, [ActionPatchType])
@wsme_pecan.wsexpose(Action, types.uuid, body=[ActionPatchType])
def patch(self, action_uuid, patch):
"""Update an existing action.
:param action_uuid: UUID of a action.
:param patch: a json PATCH document to apply to this action.
"""
if self.from_actions:
raise exception.OperationNotPermitted
action_to_update = objects.Action.get_by_uuid(pecan.request.context,
action_uuid)
try:
action_dict = action_to_update.as_dict()
action = Action(**api_utils.apply_jsonpatch(action_dict, patch))
except api_utils.JSONPATCH_EXCEPTIONS as e:
raise exception.PatchError(patch=patch, reason=e)
# Update only the fields that have changed
for field in objects.Action.fields:
try:
patch_val = getattr(action, field)
except AttributeError:
# Ignore fields that aren't exposed in the API
continue
if patch_val == wtypes.Unset:
patch_val = None
if action_to_update[field] != patch_val:
action_to_update[field] = patch_val
action_to_update.save()
return Action.convert_with_links(action_to_update)
@wsme_pecan.wsexpose(None, types.uuid, status_code=204)
def delete(self, action_uuid):
"""Delete a action.
:param action_uuid: UUID of a action.
"""
action_to_delete = objects.Action.get_by_uuid(
pecan.request.context,
action_uuid)
action_to_delete.soft_delete()

350
watcher/api/controllers/v1/action_plan.py Normal file
View File

@ -0,0 +1,350 @@
# -*- encoding: utf-8 -*-
# Copyright 2013 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import datetime
import pecan
from pecan import rest
import wsme
from wsme import types as wtypes
import wsmeext.pecan as wsme_pecan
from watcher.api.controllers import base
from watcher.api.controllers import link
from watcher.api.controllers.v1 import collection
from watcher.api.controllers.v1 import types
from watcher.api.controllers.v1 import utils as api_utils
from watcher.applier.framework.rpcapi import ApplierAPI
from watcher.common import exception
from watcher import objects
class ActionPlanPatchType(types.JsonPatchType):
@staticmethod
def mandatory_attrs():
return []
class ActionPlan(base.APIBase):
"""API representation of a action plan.
This class enforces type checking and value constraints, and converts
between the internal object model and the API representation of an
action plan.
"""
_audit_uuid = None
_first_action_uuid = None
def _get_audit_uuid(self):
return self._audit_uuid
def _set_audit_uuid(self, value):
if value == wtypes.Unset:
self._audit_uuid = wtypes.Unset
elif value and self._audit_uuid != value:
try:
audit = objects.Audit.get(pecan.request.context, value)
self._audit_uuid = audit.uuid
self.audit_id = audit.id
except exception.AuditNotFound:
self._audit_uuid = None
def _get_first_action_uuid(self):
return self._first_action_uuid
def _set_first_action_uuid(self, value):
if value == wtypes.Unset:
self._first_action_uuid = wtypes.Unset
elif value and self._first_action_uuid != value:
try:
first_action = objects.Action.get(pecan.request.context,
value)
self._first_action_uuid = first_action.uuid
self.first_action_id = first_action.id
except exception.ActionNotFound:
self._first_action_uuid = None
uuid = types.uuid
"""Unique UUID for this action plan"""
first_action_uuid = wsme.wsproperty(
types.uuid, _get_first_action_uuid, _set_first_action_uuid,
mandatory=True)
"""The UUID of the first action this action plans links to"""
audit_uuid = wsme.wsproperty(types.uuid, _get_audit_uuid, _set_audit_uuid,
mandatory=True)
"""The UUID of the audit this port belongs to"""
state = wtypes.text
"""This action plan state"""
links = wsme.wsattr([link.Link], readonly=True)
"""A list containing a self link and associated action links"""
def __init__(self, **kwargs):
super(ActionPlan, self).__init__()
self.fields = []
fields = list(objects.ActionPlan.fields)
fields.append('audit_uuid')
for field in fields:
# Skip fields we do not expose.
if not hasattr(self, field):
continue
self.fields.append(field)
setattr(self, field, kwargs.get(field, wtypes.Unset))
self.fields.append('audit_id')
setattr(self, 'audit_uuid', kwargs.get('audit_id', wtypes.Unset))
@staticmethod
def _convert_with_links(action_plan, url, expand=True):
if not expand:
action_plan.unset_fields_except(['uuid', 'state', 'updated_at',
'audit_uuid'])
action_plan.links = [link.Link.make_link(
'self', url,
'action_plans', action_plan.uuid),
link.Link.make_link(
'bookmark', url,
'action_plans', action_plan.uuid,
bookmark=True)]
return action_plan
@classmethod
def convert_with_links(cls, rpc_action_plan, expand=True):
action_plan = ActionPlan(**rpc_action_plan.as_dict())
return cls._convert_with_links(action_plan, pecan.request.host_url,
expand)
@classmethod
def sample(cls, expand=True):
sample = cls(uuid='9ef4d84c-41e8-4418-9220-ce55be0436af',
state='ONGOING',
created_at=datetime.datetime.utcnow(),
deleted_at=None,
updated_at=datetime.datetime.utcnow())
sample._first_action_uuid = '57eaf9ab-5aaa-4f7e-bdf7-9a140ac7a720'
sample._audit_uuid = 'abcee106-14d3-4515-b744-5a26885cf6f6'
return cls._convert_with_links(sample, 'http://localhost:9322', expand)
class ActionPlanCollection(collection.Collection):
"""API representation of a collection of action_plans."""
action_plans = [ActionPlan]
"""A list containing action_plans objects"""
def __init__(self, **kwargs):
self._type = 'action_plans'
@staticmethod
def convert_with_links(rpc_action_plans, limit, url=None, expand=False,
**kwargs):
collection = ActionPlanCollection()
collection.action_plans = [ActionPlan.convert_with_links(
p, expand) for p in rpc_action_plans]
if 'sort_key' in kwargs:
reverse = False
if kwargs['sort_key'] == 'audit_uuid':
if 'sort_dir' in kwargs:
reverse = True if kwargs['sort_dir'] == 'desc' else False
collection.action_plans = sorted(
collection.action_plans,
key=lambda action_plan: action_plan.audit_uuid,
reverse=reverse)
collection.next = collection.get_next(limit, url=url, **kwargs)
return collection
@classmethod
def sample(cls):
sample = cls()
sample.action_plans = [ActionPlan.sample(expand=False)]
return sample
class ActionPlansController(rest.RestController):
"""REST controller for Actions."""
def __init__(self):
super(ActionPlansController, self).__init__()
from_actionsPlans = False
"""A flag to indicate if the requests to this controller are coming
from the top-level resource ActionPlan."""
_custom_actions = {
'detail': ['GET'],
}
def _get_action_plans_collection(self, marker, limit,
sort_key, sort_dir, expand=False,
resource_url=None, audit_uuid=None):
limit = api_utils.validate_limit(limit)
sort_dir = api_utils.validate_sort_dir(sort_dir)
marker_obj = None
if marker:
marker_obj = objects.ActionPlan.get_by_uuid(
pecan.request.context, marker)
filters = {}
if audit_uuid:
filters['audit_uuid'] = audit_uuid
if sort_key == 'audit_uuid':
sort_db_key = None
else:
sort_db_key = sort_key
action_plans = objects.ActionPlan.list(
pecan.request.context,
limit,
marker_obj, sort_key=sort_db_key,
sort_dir=sort_dir, filters=filters)
return ActionPlanCollection.convert_with_links(
action_plans, limit,
url=resource_url,
expand=expand,
sort_key=sort_key,
sort_dir=sort_dir)
@wsme_pecan.wsexpose(ActionPlanCollection, types.uuid, types.uuid,
int, wtypes.text, wtypes.text, types.uuid)
def get_all(self, action_plan_uuid=None, marker=None, limit=None,
sort_key='id', sort_dir='asc', audit_uuid=None):
"""Retrieve a list of action plans.
:param marker: pagination marker for large data sets.
:param limit: maximum number of resources to return in a single result.
:param sort_key: column to sort results by. Default: id.
:param sort_dir: direction to sort. "asc" or "desc". Default: asc.
:param audit_uuid: Optional UUID of an audit, to get only actions
for that audit.
"""
return self._get_action_plans_collection(
marker, limit, sort_key, sort_dir, audit_uuid=audit_uuid)
@wsme_pecan.wsexpose(ActionPlanCollection, types.uuid, types.uuid,
int, wtypes.text, wtypes.text, types.uuid)
def detail(self, action_plan_uuid=None, marker=None, limit=None,
sort_key='id', sort_dir='asc', audit_uuid=None):
"""Retrieve a list of action_plans with detail.
:param action_plan_uuid: UUID of a action plan, to get only
:action_plans for that action.
:param marker: pagination marker for large data sets.
:param limit: maximum number of resources to return in a single result.
:param sort_key: column to sort results by. Default: id.
:param sort_dir: direction to sort. "asc" or "desc". Default: asc.
:param audit_uuid: Optional UUID of an audit, to get only actions
for that audit.
"""
# NOTE(lucasagomes): /detail should only work agaist collections
parent = pecan.request.path.split('/')[:-1][-1]
if parent != "action_plans":
raise exception.HTTPNotFound
expand = True
resource_url = '/'.join(['action_plans', 'detail'])
return self._get_action_plans_collection(
marker, limit,
sort_key, sort_dir, expand,
resource_url, audit_uuid=audit_uuid)
@wsme_pecan.wsexpose(ActionPlan, types.uuid)
def get_one(self, action_plan_uuid):
"""Retrieve information about the given action plan.
:param action_plan_uuid: UUID of a action plan.
"""
if self.from_actionsPlans:
raise exception.OperationNotPermitted
action_plan = objects.ActionPlan.get_by_uuid(
pecan.request.context, action_plan_uuid)
return ActionPlan.convert_with_links(action_plan)
@wsme_pecan.wsexpose(None, types.uuid, status_code=204)
def delete(self, action_plan_uuid):
"""Delete an action plan.
:param action_plan_uuid: UUID of a action.
"""
action_plan_to_delete = objects.ActionPlan.get_by_uuid(
pecan.request.context,
action_plan_uuid)
action_plan_to_delete.soft_delete()
@wsme.validate(types.uuid, [ActionPlanPatchType])
@wsme_pecan.wsexpose(ActionPlan, types.uuid,
body=[ActionPlanPatchType])
def patch(self, action_plan_uuid, patch):
"""Update an existing audit template.
:param audit template_uuid: UUID of a audit template.
:param patch: a json PATCH document to apply to this audit template.
"""
launch_action_plan = True
if self.from_actionsPlans:
raise exception.OperationNotPermitted
action_plan_to_update = objects.ActionPlan.get_by_uuid(
pecan.request.context,
action_plan_uuid)
try:
action_plan_dict = action_plan_to_update.as_dict()
action_plan = ActionPlan(**api_utils.apply_jsonpatch(
action_plan_dict, patch))
except api_utils.JSONPATCH_EXCEPTIONS as e:
raise exception.PatchError(patch=patch, reason=e)
launch_action_plan = False
# Update only the fields that have changed
for field in objects.ActionPlan.fields:
try:
patch_val = getattr(action_plan, field)
except AttributeError:
# Ignore fields that aren't exposed in the API
continue
if patch_val == wtypes.Unset:
patch_val = None
if action_plan_to_update[field] != patch_val:
action_plan_to_update[field] = patch_val
if field == 'state' and patch_val == 'STARTING':
launch_action_plan = True
action_plan_to_update.save()
if launch_action_plan:
applier_client = ApplierAPI()
applier_client.launch_action_plan(pecan.request.context,
action_plan.uuid)
action_plan_to_update = objects.ActionPlan.get_by_uuid(
pecan.request.context,
action_plan_uuid)
return ActionPlan.convert_with_links(action_plan_to_update)

351
watcher/api/controllers/v1/audit.py Normal file
View File

@ -0,0 +1,351 @@
# -*- encoding: utf-8 -*-
# Copyright 2013 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import datetime
import pecan
from pecan import rest
import wsme
from wsme import types as wtypes
import wsmeext.pecan as wsme_pecan
from watcher.api.controllers import base
from watcher.api.controllers import link
from watcher.api.controllers.v1 import collection
from watcher.api.controllers.v1 import types
from watcher.api.controllers.v1 import utils as api_utils
from watcher.common import exception
from watcher.common import utils
from watcher.decision_engine.framework.rpcapi import DecisionEngineAPI
from watcher import objects
class AuditPatchType(types.JsonPatchType):
@staticmethod
def mandatory_attrs():
return ['/audit_template_uuid']
class Audit(base.APIBase):
"""API representation of a audit.
This class enforces type checking and value constraints, and converts
between the internal object model and the API representation of a audit.
"""
_audit_template_uuid = None
def _get_audit_template_uuid(self):
return self._audit_template_uuid
def _set_audit_template_uuid(self, value):
if value == wtypes.Unset:
self._audit_template_uuid = wtypes.Unset
elif value and self._audit_template_uuid != value:
try:
if utils.is_uuid_like(value) or utils.is_int_like(value):
audit_template = objects.AuditTemplate.get(
pecan.request.context, value)
else:
audit_template = objects.AuditTemplate.get_by_name(
pecan.request.context, value)
self._audit_template_uuid = audit_template.uuid
self.audit_template_id = audit_template.id
except exception.AuditTemplateNotFound:
self._audit_template_uuid = None
uuid = types.uuid
"""Unique UUID for this audit"""
type = wtypes.text
"""Type of this audit"""
deadline = datetime.datetime
"""deadline of the audit"""
state = wtypes.text
"""This audit state"""
audit_template_uuid = wsme.wsproperty(wtypes.text,
_get_audit_template_uuid,
_set_audit_template_uuid,
mandatory=True)
"""The UUID of the node this port belongs to"""
links = wsme.wsattr([link.Link], readonly=True)
"""A list containing a self link and associated audit links"""
def __init__(self, **kwargs):
self.fields = []
fields = list(objects.Audit.fields)
# audit_template_uuid is not part of objects.Audit.fields
# because it's an API-only attribute.
fields.append('audit_template_uuid')
for k in fields:
# Skip fields we do not expose.
if not hasattr(self, k):
continue
self.fields.append(k)
setattr(self, k, kwargs.get(k, wtypes.Unset))
self.fields.append('audit_template_id')
setattr(self, 'audit_template_uuid', kwargs.get('audit_template_id',
wtypes.Unset))
@staticmethod
def _convert_with_links(audit, url, expand=True):
if not expand:
audit.unset_fields_except(['uuid', 'type', 'deadline',
'state', 'audit_template_uuid'])
# The numeric ID should not be exposed to
# the user, it's internal only.
audit.audit_template_id = wtypes.Unset
audit.links = [link.Link.make_link('self', url,
'audits', audit.uuid),
link.Link.make_link('bookmark', url,
'audits', audit.uuid,
bookmark=True)
]
return audit
@classmethod
def convert_with_links(cls, rpc_audit, expand=True):
audit = Audit(**rpc_audit.as_dict())
return cls._convert_with_links(audit, pecan.request.host_url, expand)
@classmethod
def sample(cls, expand=True):
sample = cls(uuid='27e3153e-d5bf-4b7e-b517-fb518e17f34c',
type='ONESHOT',
state='PENDING',
deadline=None,
created_at=datetime.datetime.utcnow(),
deleted_at=None,
updated_at=datetime.datetime.utcnow())
sample._audit_template_uuid = '7ae81bb3-dec3-4289-8d6c-da80bd8001ae'
return cls._convert_with_links(sample, 'http://localhost:9322', expand)
class AuditCollection(collection.Collection):
"""API representation of a collection of audits."""
audits = [Audit]
"""A list containing audits objects"""
def __init__(self, **kwargs):
self._type = 'audits'
@staticmethod
def convert_with_links(rpc_audits, limit, url=None, expand=False,
**kwargs):
collection = AuditCollection()
collection.audits = [Audit.convert_with_links(p, expand)
for p in rpc_audits]
if 'sort_key' in kwargs:
reverse = False
if kwargs['sort_key'] == 'audit_template_uuid':
if 'sort_dir' in kwargs:
reverse = True if kwargs['sort_dir'] == 'desc' else False
collection.audits = sorted(
collection.audits,
key=lambda audit: audit.audit_template_uuid,
reverse=reverse)
collection.next = collection.get_next(limit, url=url, **kwargs)
return collection
@classmethod
def sample(cls):
sample = cls()
sample.audits = [Audit.sample(expand=False)]
return sample
class AuditsController(rest.RestController):
"""REST controller for Audits."""
def __init__(self):
super(AuditsController, self).__init__()
from_audits = False
"""A flag to indicate if the requests to this controller are coming
from the top-level resource Audits."""
_custom_actions = {
'detail': ['GET'],
}
def _get_audits_collection(self, marker, limit,
sort_key, sort_dir, expand=False,
resource_url=None, audit_template=None):
limit = api_utils.validate_limit(limit)
sort_dir = api_utils.validate_sort_dir(sort_dir)
marker_obj = None
if marker:
marker_obj = objects.Audit.get_by_uuid(pecan.request.context,
marker)
filters = {}
if audit_template:
if utils.is_uuid_like(audit_template):
filters['audit_template_uuid'] = audit_template
else:
filters['audit_template_name'] = audit_template
if sort_key == 'audit_template_uuid':
sort_db_key = None
else:
sort_db_key = sort_key
audits = objects.Audit.list(pecan.request.context,
limit,
marker_obj, sort_key=sort_db_key,
sort_dir=sort_dir, filters=filters)
return AuditCollection.convert_with_links(audits, limit,
url=resource_url,
expand=expand,
sort_key=sort_key,
sort_dir=sort_dir)
@wsme_pecan.wsexpose(AuditCollection, types.uuid,
types.uuid, int, wtypes.text,
wtypes.text, wtypes.text)
def get_all(self, audit_uuid=None, marker=None, limit=None,
sort_key='id', sort_dir='asc', audit_template=None):
"""Retrieve a list of audits.
:param marker: pagination marker for large data sets.
:param limit: maximum number of resources to return in a single result.
:param sort_key: column to sort results by. Default: id.
:param sort_dir: direction to sort. "asc" or "desc". Default: asc.
:param audit_template: Optional UUID or description of an audit
template, to get only audits for that audit template.
"""
return self._get_audits_collection(marker, limit, sort_key,
sort_dir,
audit_template=audit_template)
@wsme_pecan.wsexpose(AuditCollection, types.uuid,
types.uuid, int, wtypes.text, wtypes.text)
def detail(self, audit_uuid=None, marker=None, limit=None,
sort_key='id', sort_dir='asc'):
"""Retrieve a list of audits with detail.
:param audit_uuid: UUID of a audit, to get only audits for that audit.
:param marker: pagination marker for large data sets.
:param limit: maximum number of resources to return in a single result.
:param sort_key: column to sort results by. Default: id.
:param sort_dir: direction to sort. "asc" or "desc". Default: asc.
"""
# NOTE(lucasagomes): /detail should only work agaist collections
parent = pecan.request.path.split('/')[:-1][-1]
if parent != "audits":
raise exception.HTTPNotFound
expand = True
resource_url = '/'.join(['audits', 'detail'])
return self._get_audits_collection(marker, limit,
sort_key, sort_dir, expand,
resource_url)
@wsme_pecan.wsexpose(Audit, types.uuid)
def get_one(self, audit_uuid):
"""Retrieve information about the given audit.
:param audit_uuid: UUID of a audit.
"""
if self.from_audits:
raise exception.OperationNotPermitted
rpc_audit = objects.Audit.get_by_uuid(pecan.request.context,
audit_uuid)
return Audit.convert_with_links(rpc_audit)
@wsme_pecan.wsexpose(Audit, body=Audit, status_code=201)
def post(self, audit):
"""Create a new audit.
:param audit: a audit within the request body.
"""
if self.from_audits:
raise exception.OperationNotPermitted
audit_dict = audit.as_dict()
context = pecan.request.context
new_audit = objects.Audit(context, **audit_dict)
new_audit.create(context)
# Set the HTTP Location Header
pecan.response.location = link.build_url('audits', new_audit.uuid)
# trigger decision-engine to run the audit
dc_client = DecisionEngineAPI()
dc_client.trigger_audit(context, new_audit.uuid)
return Audit.convert_with_links(new_audit)
@wsme.validate(types.uuid, [AuditPatchType])
@wsme_pecan.wsexpose(Audit, types.uuid, body=[AuditPatchType])
def patch(self, audit_uuid, patch):
"""Update an existing audit.
:param audit_uuid: UUID of a audit.
:param patch: a json PATCH document to apply to this audit.
"""
if self.from_audits:
raise exception.OperationNotPermitted
audit_to_update = objects.Audit.get_by_uuid(pecan.request.context,
audit_uuid)
try:
audit_dict = audit_to_update.as_dict()
audit = Audit(**api_utils.apply_jsonpatch(audit_dict, patch))
except api_utils.JSONPATCH_EXCEPTIONS as e:
raise exception.PatchError(patch=patch, reason=e)
# Update only the fields that have changed
for field in objects.Audit.fields:
try:
patch_val = getattr(audit, field)
except AttributeError:
# Ignore fields that aren't exposed in the API
continue
if patch_val == wtypes.Unset:
patch_val = None
if audit_to_update[field] != patch_val:
audit_to_update[field] = patch_val
audit_to_update.save()
return Audit.convert_with_links(audit_to_update)
@wsme_pecan.wsexpose(None, types.uuid, status_code=204)
def delete(self, audit_uuid):
"""Delete a audit.
:param audit_uuid: UUID of a audit.
"""
audit_to_delete = objects.Audit.get_by_uuid(
pecan.request.context,
audit_uuid)
audit_to_delete.soft_delete()

327
watcher/api/controllers/v1/audit_template.py Normal file
View File

@ -0,0 +1,327 @@
# -*- encoding: utf-8 -*-
# Copyright 2013 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import datetime
import pecan
from pecan import rest
import wsme
from wsme import types as wtypes
import wsmeext.pecan as wsme_pecan
from watcher.api.controllers import base
from watcher.api.controllers import link
from watcher.api.controllers.v1 import collection
from watcher.api.controllers.v1 import types
from watcher.api.controllers.v1 import utils as api_utils
from watcher.common import exception
from watcher.common import utils as common_utils
from watcher import objects
class AuditTemplatePatchType(types.JsonPatchType):
@staticmethod
def mandatory_attrs():
return []
class AuditTemplate(base.APIBase):
"""API representation of a audit template.
This class enforces type checking and value constraints, and converts
between the internal object model and the API representation of an
audit template.
"""
uuid = types.uuid
"""Unique UUID for this audit template"""
name = wtypes.text
"""Name of this audit template"""
description = wtypes.text
"""Short description of this audit template"""
deadline = datetime.datetime
"""deadline of the audit template"""
host_aggregate = wtypes.IntegerType(minimum=1)
"""ID of the Nova host aggregate targeted by the audit template"""
extra = {wtypes.text: types.jsontype}
"""The metadata of the audit template"""
goal = wtypes.text
"""Goal type of the audit template"""
version = wtypes.text
"""Internal version of the audit template"""
audits = wsme.wsattr([link.Link], readonly=True)
"""Links to the collection of audits contained in this audit template"""
links = wsme.wsattr([link.Link], readonly=True)
"""A list containing a self link and associated audit template links"""
def __init__(self, **kwargs):
super(AuditTemplate, self).__init__()
self.fields = []
for field in objects.AuditTemplate.fields:
# Skip fields we do not expose.
if not hasattr(self, field):
continue
self.fields.append(field)
setattr(self, field, kwargs.get(field, wtypes.Unset))
@staticmethod
def _convert_with_links(audit_template, url, expand=True):
if not expand:
audit_template.unset_fields_except(['uuid', 'name',
'host_aggregate', 'goal'])
audit_template.links = [link.Link.make_link('self', url,
'audit_templates',
audit_template.uuid),
link.Link.make_link('bookmark', url,
'audit_templates',
audit_template.uuid,
bookmark=True)
]
return audit_template
@classmethod
def convert_with_links(cls, rpc_audit_template, expand=True):
audit_template = AuditTemplate(**rpc_audit_template.as_dict())
return cls._convert_with_links(audit_template, pecan.request.host_url,
expand)
@classmethod
def sample(cls, expand=True):
sample = cls(uuid='27e3153e-d5bf-4b7e-b517-fb518e17f34c',
name='My Audit Template',
description='Description of my audit template',
host_aggregate=5,
goal='SERVERS_CONSOLIDATION',
extra={'automatic': True},
created_at=datetime.datetime.utcnow(),
deleted_at=None,
updated_at=datetime.datetime.utcnow())
return cls._convert_with_links(sample, 'http://localhost:9322', expand)
class AuditTemplateCollection(collection.Collection):
"""API representation of a collection of audit templates."""
audit_templates = [AuditTemplate]
"""A list containing audit templates objects"""
def __init__(self, **kwargs):
self._type = 'audit_templates'
@staticmethod
def convert_with_links(rpc_audit_templates, limit, url=None, expand=False,
**kwargs):
collection = AuditTemplateCollection()
collection.audit_templates = \
[AuditTemplate.convert_with_links(p, expand)
for p in rpc_audit_templates]
collection.next = collection.get_next(limit, url=url, **kwargs)
return collection
@classmethod
def sample(cls):
sample = cls()
sample.audit_templates = [AuditTemplate.sample(expand=False)]
return sample
class AuditTemplatesController(rest.RestController):
"""REST controller for AuditTemplates."""
def __init__(self):
super(AuditTemplatesController, self).__init__()
from_audit_templates = False
"""A flag to indicate if the requests to this controller are coming
from the top-level resource AuditTemplates."""
_custom_actions = {
'detail': ['GET'],
}
def _get_audit_templates_collection(self, marker, limit,
sort_key, sort_dir, expand=False,
resource_url=None):
limit = api_utils.validate_limit(limit)
sort_dir = api_utils.validate_sort_dir(sort_dir)
marker_obj = None
if marker:
marker_obj = objects.AuditTemplate.get_by_uuid(
pecan.request.context,
marker)
audit_templates = objects.AuditTemplate.list(
pecan.request.context,
limit,
marker_obj, sort_key=sort_key,
sort_dir=sort_dir)
return AuditTemplateCollection.convert_with_links(audit_templates,
limit,
url=resource_url,
expand=expand,
sort_key=sort_key,
sort_dir=sort_dir)
@wsme_pecan.wsexpose(AuditTemplateCollection, types.uuid, int,
wtypes.text, wtypes.text)
def get_all(self, marker=None, limit=None,
sort_key='id', sort_dir='asc'):
"""Retrieve a list of audit templates.
:param marker: pagination marker for large data sets.
:param limit: maximum number of resources to return in a single result.
:param sort_key: column to sort results by. Default: id.
:param sort_dir: direction to sort. "asc" or "desc". Default: asc.
"""
return self._get_audit_templates_collection(marker, limit, sort_key,
sort_dir)
@wsme_pecan.wsexpose(AuditTemplateCollection, types.uuid, int,
wtypes.text, wtypes.text)
def detail(self, marker=None, limit=None,
sort_key='id', sort_dir='asc'):
"""Retrieve a list of audit templates with detail.
:param marker: pagination marker for large data sets.
:param limit: maximum number of resources to return in a single result.
:param sort_key: column to sort results by. Default: id.
:param sort_dir: direction to sort. "asc" or "desc". Default: asc.
"""
# NOTE(lucasagomes): /detail should only work agaist collections
parent = pecan.request.path.split('/')[:-1][-1]
if parent != "audit_templates":
raise exception.HTTPNotFound
expand = True
resource_url = '/'.join(['audit_templates', 'detail'])
return self._get_audit_templates_collection(marker, limit,
sort_key, sort_dir, expand,
resource_url)
@wsme_pecan.wsexpose(AuditTemplate, wtypes.text)
def get_one(self, audit_template):
"""Retrieve information about the given audit template.
:param audit template_uuid: UUID or name of an audit template.
"""
if self.from_audit_templates:
raise exception.OperationNotPermitted
if common_utils.is_uuid_like(audit_template):
rpc_audit_template = objects.AuditTemplate.get_by_uuid(
pecan.request.context,
audit_template)
else:
rpc_audit_template = objects.AuditTemplate.get_by_name(
pecan.request.context,
audit_template)
return AuditTemplate.convert_with_links(rpc_audit_template)
@wsme_pecan.wsexpose(AuditTemplate, body=AuditTemplate, status_code=201)
def post(self, audit_template):
"""Create a new audit template.
:param audit template: a audit template within the request body.
"""
if self.from_audit_templates:
raise exception.OperationNotPermitted
audit_template_dict = audit_template.as_dict()
context = pecan.request.context
new_audit_template = objects.AuditTemplate(context,
**audit_template_dict)
new_audit_template.create(context)
# Set the HTTP Location Header
pecan.response.location = link.build_url('audit_templates',
new_audit_template.uuid)
return AuditTemplate.convert_with_links(new_audit_template)
@wsme.validate(types.uuid, [AuditTemplatePatchType])
@wsme_pecan.wsexpose(AuditTemplate, wtypes.text,
body=[AuditTemplatePatchType])
def patch(self, audit_template, patch):
"""Update an existing audit template.
:param audit template_uuid: UUID of a audit template.
:param patch: a json PATCH document to apply to this audit template.
"""
if self.from_audit_templates:
raise exception.OperationNotPermitted
if common_utils.is_uuid_like(audit_template):
audit_template_to_update = objects.AuditTemplate.get_by_uuid(
pecan.request.context,
audit_template)
else:
audit_template_to_update = objects.AuditTemplate.get_by_name(
pecan.request.context,
audit_template)
try:
audit_template_dict = audit_template_to_update.as_dict()
audit_template = AuditTemplate(**api_utils.apply_jsonpatch(
audit_template_dict, patch))
except api_utils.JSONPATCH_EXCEPTIONS as e:
raise exception.PatchError(patch=patch, reason=e)
# Update only the fields that have changed
for field in objects.AuditTemplate.fields:
try:
patch_val = getattr(audit_template, field)
except AttributeError:
# Ignore fields that aren't exposed in the API
continue
if patch_val == wtypes.Unset:
patch_val = None
if audit_template_to_update[field] != patch_val:
audit_template_to_update[field] = patch_val
audit_template_to_update.save()
return AuditTemplate.convert_with_links(audit_template_to_update)
@wsme_pecan.wsexpose(None, wtypes.text, status_code=204)
def delete(self, audit_template):
"""Delete a audit template.
:param audit template_uuid: UUID or name of an audit template.
"""
if common_utils.is_uuid_like(audit_template):
audit_template_to_delete = objects.AuditTemplate.get_by_uuid(
pecan.request.context,
audit_template)
else:
audit_template_to_delete = objects.AuditTemplate.get_by_name(
pecan.request.context,
audit_template)
audit_template_to_delete.soft_delete()

50
watcher/api/controllers/v1/collection.py Normal file
View File

@ -0,0 +1,50 @@
# -*- encoding: utf-8 -*-
# Copyright 2013 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import pecan
from wsme import types as wtypes
from watcher.api.controllers import base
from watcher.api.controllers import link
class Collection(base.APIBase):
next = wtypes.text
"""A link to retrieve the next subset of the collection"""
@property
def collection(self):
return getattr(self, self._type)
def has_next(self, limit):
"""Return whether collection has more items."""
return len(self.collection) and len(self.collection) == limit
def get_next(self, limit, url=None, **kwargs):
"""Return a link to the next subset of the collection."""
if not self.has_next(limit):
return wtypes.Unset
resource_url = url or self._type
q_args = ''.join(['%s=%s&' % (key, kwargs[key]) for key in kwargs])
next_args = '?%(args)slimit=%(limit)d&marker=%(marker)s' % {
'args': q_args, 'limit': limit,
'marker': self.collection[-1].uuid}
return link.Link.make_link('next', pecan.request.host_url,
resource_url, next_args).href

237
watcher/api/controllers/v1/types.py Normal file
View File

@ -0,0 +1,237 @@
# coding: utf-8
#
# Copyright 2013 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import json
from oslo_utils import strutils
import six
import wsme
from wsme import types as wtypes
from watcher.common import exception
from watcher.common.i18n import _
from watcher.common import utils
class UuidOrNameType(wtypes.UserType):
"""A simple UUID or logical name type."""
basetype = wtypes.text
name = 'uuid_or_name'
# FIXME(lucasagomes): When used with wsexpose decorator WSME will try
# to get the name of the type by accessing it's __name__ attribute.
# Remove this __name__ attribute once it's fixed in WSME.
# https://bugs.launchpad.net/wsme/+bug/1265590
__name__ = name
@staticmethod
def validate(value):
if not (utils.is_uuid_like(value) or utils.is_hostname_safe(value)):
raise exception.InvalidUuidOrName(name=value)
return value
@staticmethod
def frombasetype(value):
if value is None:
return None
return UuidOrNameType.validate(value)
class NameType(wtypes.UserType):
"""A simple logical name type."""
basetype = wtypes.text
name = 'name'
# FIXME(lucasagomes): When used with wsexpose decorator WSME will try
# to get the name of the type by accessing it's __name__ attribute.
# Remove this __name__ attribute once it's fixed in WSME.
# https://bugs.launchpad.net/wsme/+bug/1265590
__name__ = name
@staticmethod
def validate(value):
if not utils.is_hostname_safe(value):
raise exception.InvalidName(name=value)
return value
@staticmethod
def frombasetype(value):
if value is None:
return None
return NameType.validate(value)
class UuidType(wtypes.UserType):
"""A simple UUID type."""
basetype = wtypes.text
name = 'uuid'
# FIXME(lucasagomes): When used with wsexpose decorator WSME will try
# to get the name of the type by accessing it's __name__ attribute.
# Remove this __name__ attribute once it's fixed in WSME.
# https://bugs.launchpad.net/wsme/+bug/1265590
__name__ = name
@staticmethod
def validate(value):
if not utils.is_uuid_like(value):
raise exception.InvalidUUID(uuid=value)
return value
@staticmethod
def frombasetype(value):
if value is None:
return None
return UuidType.validate(value)
class BooleanType(wtypes.UserType):
"""A simple boolean type."""
basetype = wtypes.text
name = 'boolean'
# FIXME(lucasagomes): When used with wsexpose decorator WSME will try
# to get the name of the type by accessing it's __name__ attribute.
# Remove this __name__ attribute once it's fixed in WSME.
# https://bugs.launchpad.net/wsme/+bug/1265590
__name__ = name
@staticmethod
def validate(value):
try:
return strutils.bool_from_string(value, strict=True)
except ValueError as e:
# raise Invalid to return 400 (BadRequest) in the API
raise exception.Invalid(e)
@staticmethod
def frombasetype(value):
if value is None:
return None
return BooleanType.validate(value)
class JsonType(wtypes.UserType):
"""A simple JSON type."""
basetype = wtypes.text
name = 'json'
# FIXME(lucasagomes): When used with wsexpose decorator WSME will try
# to get the name of the type by accessing it's __name__ attribute.
# Remove this __name__ attribute once it's fixed in WSME.
# https://bugs.launchpad.net/wsme/+bug/1265590
__name__ = name
def __str__(self):
# These are the json serializable native types
return ' | '.join(map(str, (wtypes.text, six.integer_types, float,
BooleanType, list, dict, None)))
@staticmethod
def validate(value):
try:
json.dumps(value)
except TypeError:
raise exception.Invalid(_('%s is not JSON serializable') % value)
else:
return value
@staticmethod
def frombasetype(value):
return JsonType.validate(value)
uuid = UuidType()
boolean = BooleanType()
jsontype = JsonType()
class MultiType(wtypes.UserType):
"""A complex type that represents one or more types.
Used for validating that a value is an instance of one of the types.
:param types: Variable-length list of types.
"""
def __init__(self, *types):
self.types = types
def __str__(self):
return ' | '.join(map(str, self.types))
def validate(self, value):
for t in self.types:
if t is wsme.types.text and isinstance(value, wsme.types.bytes):
value = value.decode()
if isinstance(value, t):
return value
else:
raise ValueError(
_("Wrong type. Expected '%(type)s', got '%(value)s'")
% {'type': self.types, 'value': type(value)})
class JsonPatchType(wtypes.Base):
"""A complex type that represents a single json-patch operation."""
path = wtypes.wsattr(wtypes.StringType(pattern='^(/[\w-]+)+$'),
mandatory=True)
op = wtypes.wsattr(wtypes.Enum(str, 'add', 'replace', 'remove'),
mandatory=True)
value = wsme.wsattr(jsontype, default=wtypes.Unset)
@staticmethod
def internal_attrs():
"""Returns a list of internal attributes.
Internal attributes can't be added, replaced or removed. This
method may be overwritten by derived class.
"""
return ['/created_at', '/id', '/links', '/updated_at',
'/deleted_at', '/uuid']
@staticmethod
def mandatory_attrs():
"""Retruns a list of mandatory attributes.
Mandatory attributes can't be removed from the document. This
method should be overwritten by derived class.
"""
return []
@staticmethod
def validate(patch):
_path = '/' + patch.path.split('/')[1]
if _path in patch.internal_attrs():
msg = _("'%s' is an internal attribute and can not be updated")
raise wsme.exc.ClientSideError(msg % patch.path)
if patch.path in patch.mandatory_attrs() and patch.op == 'remove':
msg = _("'%s' is a mandatory attribute and can not be removed")
raise wsme.exc.ClientSideError(msg % patch.path)
if patch.op != 'remove':
if patch.value is wsme.Unset:
msg = _("'add' and 'replace' operations needs value")
raise wsme.exc.ClientSideError(msg)
ret = {'path': patch.path, 'op': patch.op}
if patch.value is not wsme.Unset:
ret['value'] = patch.value
return ret

52
watcher/api/controllers/v1/utils.py Normal file
View File

@ -0,0 +1,52 @@
# Copyright 2013 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import jsonpatch
from oslo_config import cfg
import wsme
from watcher.common.i18n import _
CONF = cfg.CONF
JSONPATCH_EXCEPTIONS = (jsonpatch.JsonPatchException,
jsonpatch.JsonPointerException,
KeyError)
def validate_limit(limit):
if limit is not None and limit <= 0:
raise wsme.exc.ClientSideError(_("Limit must be positive"))
return min(CONF.api.max_limit, limit) or CONF.api.max_limit
def validate_sort_dir(sort_dir):
if sort_dir not in ['asc', 'desc']:
raise wsme.exc.ClientSideError(_("Invalid sort direction: %s. "
"Acceptable values are "
"'asc' or 'desc'") % sort_dir)
return sort_dir
def apply_jsonpatch(doc, patch):
for p in patch:
if p['op'] == 'add' and p['path'].count('/') == 1:
if p['path'].lstrip('/') not in doc:
msg = _('Adding a new attribute (%s) to the root of '
' the resource is not allowed')
raise wsme.exc.ClientSideError(msg % p['path'])
return jsonpatch.apply_patch(doc, jsonpatch.JsonPatch(patch))

113
watcher/api/hooks.py Normal file
View File

@ -0,0 +1,113 @@
# -*- encoding: utf-8 -*-
#
# Copyright © 2012 New Dream Network, LLC (DreamHost)
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo_config import cfg
from oslo_utils import importutils
from pecan import hooks
from watcher.common import context
class ContextHook(hooks.PecanHook):
"""Configures a request context and attaches it to the request.
The following HTTP request headers are used:
X-User:
Used for context.user.
X-User-Id:
Used for context.user_id.
X-Project-Name:
Used for context.project.
X-Project-Id:
Used for context.project_id.
X-Auth-Token:
Used for context.auth_token.
"""
def before(self, state):
headers = state.request.headers
user = headers.get('X-User')
user_id = headers.get('X-User-Id')
project = headers.get('X-Project-Name')
project_id = headers.get('X-Project-Id')
domain_id = headers.get('X-User-Domain-Id')
domain_name = headers.get('X-User-Domain-Name')
auth_token = headers.get('X-Storage-Token')
auth_token = headers.get('X-Auth-Token', auth_token)
show_deleted = headers.get('X-Show-Deleted')
auth_token_info = state.request.environ.get('keystone.token_info')
auth_url = headers.get('X-Auth-Url')
if auth_url is None:
importutils.import_module('keystonemiddleware.auth_token')
auth_url = cfg.CONF.keystone_authtoken.auth_uri
state.request.context = context.make_context(
auth_token=auth_token,
auth_url=auth_url,
auth_token_info=auth_token_info,
user=user,
user_id=user_id,
project=project,
project_id=project_id,
domain_id=domain_id,
domain_name=domain_name,
show_deleted=show_deleted)
class NoExceptionTracebackHook(hooks.PecanHook):
"""Workaround rpc.common: deserialize_remote_exception.
deserialize_remote_exception builds rpc exception traceback into error
message which is then sent to the client. Such behavior is a security
concern so this hook is aimed to cut-off traceback from the error message.
"""
# NOTE(max_lobur): 'after' hook used instead of 'on_error' because
# 'on_error' never fired for wsme+pecan pair. wsme @wsexpose decorator
# catches and handles all the errors, so 'on_error' dedicated for unhandled
# exceptions never fired.
def after(self, state):
# Omit empty body. Some errors may not have body at this level yet.
if not state.response.body:
return
# Do nothing if there is no error.
if 200 <= state.response.status_int < 400:
return
json_body = state.response.json
# Do not remove traceback when server in debug mode (except 'Server'
# errors when 'debuginfo' will be used for traces).
if cfg.CONF.debug and json_body.get('faultcode') != 'Server':
return
faultstring = json_body.get('faultstring')
traceback_marker = 'Traceback (most recent call last):'
if faultstring and (traceback_marker in faultstring):
# Cut-off traceback.
faultstring = faultstring.split(traceback_marker, 1)[0]
# Remove trailing newlines and spaces if any.
json_body['faultstring'] = faultstring.rstrip()
# Replace the whole json. Cannot change original one beacause it's
# generated on the fly.
state.response.json = json_body

25
watcher/api/middleware/__init__.py Normal file
View File

@ -0,0 +1,25 @@
# -*- encoding: utf-8 -*-
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from watcher.api.middleware import auth_token
from watcher.api.middleware import parsable_error
ParsableErrorMiddleware = parsable_error.ParsableErrorMiddleware
AuthTokenMiddleware = auth_token.AuthTokenMiddleware
__all__ = (ParsableErrorMiddleware,
AuthTokenMiddleware)

61
watcher/api/middleware/auth_token.py Normal file
View File

@ -0,0 +1,61 @@
# -*- encoding: utf-8 -*-
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import re
from keystonemiddleware import auth_token
from watcher.common import exception
from watcher.common.i18n import _
from watcher.common import utils
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
class AuthTokenMiddleware(auth_token.AuthProtocol):
"""A wrapper on Keystone auth_token middleware.
Does not perform verification of authentication tokens
for public routes in the API.
"""
def __init__(self, app, conf, public_api_routes=[]):
route_pattern_tpl = '%s(\.json|\.xml)?$'
try:
self.public_api_routes = [re.compile(route_pattern_tpl % route_tpl)
for route_tpl in public_api_routes]
except re.error as e:
msg = _('Cannot compile public API routes: %s') % e
LOG.error(msg)
raise exception.ConfigInvalid(error_msg=msg)
super(AuthTokenMiddleware, self).__init__(app, conf)
def __call__(self, env, start_response):
path = utils.safe_rstrip(env.get('PATH_INFO'), '/')
# The information whether the API call is being performed against the
# public API is required for some other components. Saving it to the
# WSGI environment is reasonable thereby.
env['is_public_api'] = any(map(lambda pattern: re.match(pattern, path),
self.public_api_routes))
if env['is_public_api']:
return self._app(env, start_response)
return super(AuthTokenMiddleware, self).__call__(env, start_response)

90
watcher/api/middleware/parsable_error.py Normal file
View File

@ -0,0 +1,90 @@
# -*- encoding: utf-8 -*-
#
# Copyright © 2012 New Dream Network, LLC (DreamHost)
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Middleware to replace the plain text message body of an error
response with one formatted so the client can parse it.
Based on pecan.middleware.errordocument
"""
import json
from xml import etree as et
import webob
from watcher.common.i18n import _
from watcher.common.i18n import _LE
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
class ParsableErrorMiddleware(object):
"""Replace error body with something the client can parse."""
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
# Request for this state, modified by replace_start_response()
# and used when an error is being reported.
state = {}
def replacement_start_response(status, headers, exc_info=None):
"""Overrides the default response to make errors parsable."""
try:
status_code = int(status.split(' ')[0])
state['status_code'] = status_code
except (ValueError, TypeError): # pragma: nocover
raise Exception(_(
'ErrorDocumentMiddleware received an invalid '
'status %s') % status)
else:
if (state['status_code'] // 100) not in (2, 3):
# Remove some headers so we can replace them later
# when we have the full error message and can
# compute the length.
headers = [(h, v)
for (h, v) in headers
if h not in ('Content-Length', 'Content-Type')
]
# Save the headers in case we need to modify them.
state['headers'] = headers
return start_response(status, headers, exc_info)
app_iter = self.app(environ, replacement_start_response)
if (state['status_code'] // 100) not in (2, 3):
req = webob.Request(environ)
if (req.accept.best_match(['application/json', 'application/xml'])
== 'application/xml'):
try:
# simple check xml is valid
body = [et.ElementTree.tostring(
et.ElementTree.fromstring('<error_message>'
+ '\n'.join(app_iter)
+ '</error_message>'))]
except et.ElementTree.ParseError as err:
LOG.error(_LE('Error parsing HTTP response: %s'), err)
body = ['<error_message>%s' % state['status_code']
+ '</error_message>']
state['headers'].append(('Content-Type', 'application/xml'))
else:
body = [json.dumps({'error_message': '\n'.join(app_iter)})]
state['headers'].append(('Content-Type', 'application/json'))
state['headers'].append(('Content-Length', len(body[0])))
else:
body = app_iter
return body

11
watcher/applier/README.md Normal file
View File

@ -0,0 +1,11 @@
# Watcher Actions Applier
This component is in charge of executing the plan of actions built by the Watcher Actions Planner.
For each action of the workflow, this component may call directly the component responsible for this kind of action (Example : Nova API for an instance migration) or via some publish/subscribe pattern on the message bus.
It notifies continuously of the current progress of the Action Plan (and atomic Actions), sending status messages on the bus. Those events may be used by the CEP to trigger new actions.
This component is also connected to the Watcher MySQL database in order to:
* get the description of the action plan to execute
* persist its current state so that if it is restarted, it can restore each Action plan context and restart from the last known safe point of each ongoing workflow.

0
watcher/applier/__init__.py Normal file
View File

0
watcher/applier/api/__init__.py Normal file
View File

21
watcher/applier/api/applier.py Normal file
View File

@ -0,0 +1,21 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
class Applier(object):
def execute(self, action_plan_uuid):
raise NotImplementedError("Should have implemented this")

21
watcher/applier/api/command_mapper.py Normal file
View File

@ -0,0 +1,21 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
class CommandMapper(object):
def build_primitive_command(self, action):
raise NotImplementedError("Should have implemented this")

0
watcher/applier/api/messaging/__init__.py Normal file
View File

21
watcher/applier/api/messaging/applier_command.py Normal file
View File

@ -0,0 +1,21 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
class ApplierCommand(object):
def execute(self):
raise NotImplementedError("Should have implemented this")

26
watcher/applier/api/primitive_command.py Normal file
View File

@ -0,0 +1,26 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from watcher.applier.api.promise import Promise
class PrimitiveCommand(object):
@Promise
def execute(self):
raise NotImplementedError("Should have implemented this")
@Promise
def undo(self):
raise NotImplementedError("Should have implemented this")

48
watcher/applier/api/promise.py Normal file
View File

@ -0,0 +1,48 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from concurrent.futures import Future
from concurrent.futures import ThreadPoolExecutor
class Promise(object):
executor = ThreadPoolExecutor(
max_workers=10)
def __init__(self, func):
self.func = func
def resolve(self, *args, **kwargs):
resolved_args = []
resolved_kwargs = {}
for i, arg in enumerate(args):
if isinstance(arg, Future):
resolved_args.append(arg.result())
else:
resolved_args.append(arg)
for kw, arg in kwargs.items():
if isinstance(arg, Future):
resolved_kwargs[kw] = arg.result()
else:
resolved_kwargs[kw] = arg
return self.func(*resolved_args, **resolved_kwargs)
def __call__(self, *args, **kwargs):
return self.executor.submit(self.resolve, *args, **kwargs)

0
watcher/applier/framework/__init__.py Normal file
View File

0
watcher/applier/framework/command/__init__.py Normal file
View File

76
watcher/applier/framework/command/hypervisor_state_command.py Normal file
View File

@ -0,0 +1,76 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from keystoneclient.auth.identity import v3
from keystoneclient import session
from oslo_config import cfg
from watcher.applier.api.primitive_command import PrimitiveCommand
from watcher.applier.api.promise import Promise
from watcher.applier.framework.command.wrapper.nova_wrapper import NovaWrapper
from watcher.decision_engine.framework.model.hypervisor_state import \
HypervisorState
CONF = cfg.CONF
class HypervisorStateCommand(PrimitiveCommand):
def __init__(self, host, status):
self.host = host
self.status = status
def nova_manage_service(self, status):
creds = \
{'auth_url': CONF.keystone_authtoken.auth_uri,
'username': CONF.keystone_authtoken.admin_user,
'password': CONF.keystone_authtoken.admin_password,
'project_name': CONF.keystone_authtoken.admin_tenant_name,
'user_domain_name': "default",
'project_domain_name': "default"}
auth = v3.Password(auth_url=creds['auth_url'],
username=creds['username'],
password=creds['password'],
project_name=creds['project_name'],
user_domain_name=creds[
'user_domain_name'],
project_domain_name=creds[
'project_domain_name'])
sess = session.Session(auth=auth)
# todo(jed) refactoring
wrapper = NovaWrapper(creds, session=sess)
if status is True:
return wrapper.enable_service_nova_compute(self.host)
else:
return wrapper.disable_service_nova_compute(self.host)
@Promise
def execute(self):
if self.status == HypervisorState.OFFLINE.value:
state = False
elif self.status == HypervisorState.ONLINE.value:
state = True
return self.nova_manage_service(state)
@Promise
def undo(self):
if self.status == HypervisorState.OFFLINE.value:
state = True
elif self.status == HypervisorState.ONLINE.value:
state = False
return self.nova_manage_service(state)

100
watcher/applier/framework/command/migrate_command.py Normal file
View File

@ -0,0 +1,100 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from keystoneclient.auth.identity import v3
from keystoneclient import session
from oslo_config import cfg
from watcher.applier.api.primitive_command import PrimitiveCommand
from watcher.applier.api.promise import Promise
from watcher.applier.framework.command.wrapper.nova_wrapper import NovaWrapper
from watcher.decision_engine.framework.default_planner import Primitives
CONF = cfg.CONF
class MigrateCommand(PrimitiveCommand):
def __init__(self, vm_uuid=None,
migration_type=None,
source_hypervisor=None,
destination_hypervisor=None):
self.instance_uuid = vm_uuid
self.migration_type = migration_type
self.source_hypervisor = source_hypervisor
self.destination_hypervisor = destination_hypervisor
def migrate(self, destination):
creds = \
{'auth_url': CONF.keystone_authtoken.auth_uri,
'username': CONF.keystone_authtoken.admin_user,
'password': CONF.keystone_authtoken.admin_password,
'project_name': CONF.keystone_authtoken.admin_tenant_name,
'user_domain_name': "default",
'project_domain_name': "default"}
auth = v3.Password(auth_url=creds['auth_url'],
username=creds['username'],
password=creds['password'],
project_name=creds['project_name'],
user_domain_name=creds[
'user_domain_name'],
project_domain_name=creds[
'project_domain_name'])
sess = session.Session(auth=auth)
# todo(jed) add class
wrapper = NovaWrapper(creds, session=sess)
instance = wrapper.find_instance(self.instance_uuid)
if instance:
project_id = getattr(instance, "tenant_id")
creds2 = \
{'auth_url': CONF.keystone_authtoken.auth_uri,
'username': CONF.keystone_authtoken.admin_user,
'password': CONF.keystone_authtoken.admin_password,
'project_id': project_id,
'user_domain_name': "default",
'project_domain_name': "default"}
auth2 = v3.Password(auth_url=creds2['auth_url'],
username=creds2['username'],
password=creds2['password'],
project_id=creds2['project_id'],
user_domain_name=creds2[
'user_domain_name'],
project_domain_name=creds2[
'project_domain_name'])
sess2 = session.Session(auth=auth2)
wrapper2 = NovaWrapper(creds2, session=sess2)
# todo(jed) remove Primitves
if self.migration_type is Primitives.COLD_MIGRATE:
return wrapper2.live_migrate_instance(
instance_id=self.instance_uuid,
dest_hostname=destination,
block_migration=True)
elif self.migration_type is Primitives.LIVE_MIGRATE:
return wrapper2.live_migrate_instance(
instance_id=self.instance_uuid,
dest_hostname=destination,
block_migration=False)
@Promise
def execute(self):
return self.migrate(self.destination_hypervisor)
@Promise
def undo(self):
return self.migrate(self.source_hypervisor)

31
watcher/applier/framework/command/nop_command.py Normal file
View File

@ -0,0 +1,31 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from watcher.applier.api.primitive_command import PrimitiveCommand
from watcher.applier.api.promise import Promise
class NopCommand(PrimitiveCommand):
def __init__(self):
pass
@Promise
def execute(self):
return True
@Promise
def undo(self):
return True

33
watcher/applier/framework/command/power_state_command.py Normal file
View File

@ -0,0 +1,33 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from watcher.applier.api.primitive_command import PrimitiveCommand
from watcher.applier.api.promise import Promise
class PowerStateCommand(PrimitiveCommand):
def __init__(self):
pass
@Promise
def execute(self):
pass
@Promise
def undo(self):
# TODO(jde): migrate VM from target_hypervisor
# to current_hypervisor in model
return True

0
watcher/applier/framework/command/wrapper/__init__.py Normal file
View File

694
watcher/applier/framework/command/wrapper/nova_wrapper.py Normal file
View File

@ -0,0 +1,694 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import random
import time
import cinderclient.exceptions as ciexceptions
import cinderclient.v2.client as ciclient
import glanceclient.v2.client as glclient
import keystoneclient.v3.client as ksclient
import neutronclient.neutron.client as netclient
import novaclient.exceptions as nvexceptions
import novaclient.v2.client as nvclient
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
class NovaWrapper(object):
def __init__(self, creds, session):
self.user = creds['username']
self.session = session
self.neutron = None
self.cinder = None
self.nova = nvclient.Client("3", session=session)
self.keystone = ksclient.Client(**creds)
self.glance = None
def get_hypervisors_list(self):
return self.nova.hypervisors.list()
def find_instance(self, instance_id):
search_opts = {'all_tenants': True}
instances = self.nova.servers.list(detailed=True,
search_opts=search_opts)
instance = None
for _instance in instances:
if _instance.id == instance_id:
instance = _instance
break
return instance
def watcher_non_live_migrate_instance(self, instance_id, hypervisor_id,
keep_original_image_name=True):
"""This method migrates a given instance
using an image of this instance and creating a new instance
from this image. It saves some configuration information
about the original instance : security group, list of networks
,list of attached volumes, floating IP, ...
in order to apply the same settings to the new instance.
At the end of the process the original instance is deleted.
It returns True if the migration was successful,
False otherwise.
:param instance_id: the unique id of the instance to migrate.
:param keep_original_image_name: flag indicating whether the
image name from which the original instance was built must be
used as the name of the intermediate image used for migration.
If this flag is False, a temporary image name is built
"""
new_image_name = ""
LOG.debug(
"Trying a non-live migrate of instance '%s' "
"using a temporary image ..." % instance_id)
# Looking for the instance to migrate
instance = self.find_instance(instance_id)
if not instance:
LOG.debug("Instance %s not found !" % instance_id)
return False
else:
host_name = getattr(instance, "OS-EXT-SRV-ATTR:host")
# https://bugs.launchpad.net/nova/+bug/1182965
LOG.debug(
"Instance %s found on host '%s'." % (instance_id, host_name))
if not keep_original_image_name:
# randrange gives you an integral value
irand = random.randint(0, 1000)
# Building the temporary image name
# which will be used for the migration
new_image_name = "tmp-migrate-%s-%s" % (instance_id, irand)
else:
# Get the image name of the current instance.
# We'll use the same name for the new instance.
imagedict = getattr(instance, "image")
image_id = imagedict["id"]
image = self.nova.images.get(image_id)
new_image_name = getattr(image, "name")
instance_name = getattr(instance, "name")
flavordict = getattr(instance, "flavor")
# a_dict = dict([flavorstr.strip('{}').split(":"),])
flavor_id = flavordict["id"]
flavor = self.nova.flavors.get(flavor_id)
flavor_name = getattr(flavor, "name")
keypair_name = getattr(instance, "key_name")
addresses = getattr(instance, "addresses")
floating_ip = ""
network_names_list = []
for network_name, network_conf_obj in addresses.items():
LOG.debug(
"Extracting network configuration for network '%s'" %
network_name)
network_names_list.append(network_name)
for net_conf_item in network_conf_obj:
if net_conf_item['OS-EXT-IPS:type'] == "floating":
floating_ip = net_conf_item['addr']
break
sec_groups_list = getattr(instance, "security_groups")
sec_groups = []
for sec_group_dict in sec_groups_list:
sec_groups.append(sec_group_dict['name'])
# Stopping the old instance properly so
# that no new data is sent to it and to its attached volumes
stopped_ok = self.stop_instance(instance_id)
if not stopped_ok:
LOG.debug("Could not stop instance: %s" % instance_id)
return False
# Building the temporary image which will be used
# to re-build the same instance on another target host
image_uuid = self.create_image_from_instance(instance_id,
new_image_name)
if not image_uuid:
LOG.debug(
"Could not build temporary image of instance: %s" %
instance_id)
return False
#
# We need to get the list of attached volumes and detach
# them from the instance in order to attache them later
# to the new instance
#
blocks = []
# Looks like this :
# os-extended-volumes:volumes_attached |
# [{u'id': u'c5c3245f-dd59-4d4f-8d3a-89d80135859a'}]
attached_volumes = getattr(instance,
"os-extended-volumes:volumes_attached")
for attached_volume in attached_volumes:
volume_id = attached_volume['id']
try:
if self.cinder is None:
self.cinder = ciclient.Client('2',
session=self.session)
volume = self.cinder.volumes.get(volume_id)
attachments_list = getattr(volume, "attachments")
device_name = attachments_list[0]['device']
# When a volume is attached to an instance
# it contains the following property :
# attachments = [{u'device': u'/dev/vdb',
# u'server_id': u'742cc508-a2f2-4769-a794-bcdad777e814',
# u'id': u'f6d62785-04b8-400d-9626-88640610f65e',
# u'host_name': None, u'volume_id':
# u'f6d62785-04b8-400d-9626-88640610f65e'}]
# boot_index indicates a number
# designating the boot order of the device.
# Use -1 for the boot volume,
# choose 0 for an attached volume.
block_device_mapping_v2_item = {"device_name": device_name,
"source_type": "volume",
"destination_type":
"volume",
"uuid": volume_id,
"boot_index": "0"}
blocks.append(
block_device_mapping_v2_item)
LOG.debug("Detaching volume %s from instance: %s" % (
volume_id, instance_id))
# volume.detach()
self.nova.volumes.delete_server_volume(instance_id,
volume_id)
if not self.wait_for_volume_status(volume, "available", 5,
10):
LOG.debug(
"Could not detach volume %s from instance: %s" % (
volume_id, instance_id))
return False
except ciexceptions.NotFound:
LOG.debug("Volume '%s' not found " % image_id)
return False
# We create the new instance from
# the intermediate image of the original instance
new_instance = self. \
create_instance(hypervisor_id,
instance_name,
image_uuid,
flavor_name,
sec_groups,
network_names_list=network_names_list,
keypair_name=keypair_name,
create_new_floating_ip=False,
block_device_mapping_v2=blocks)
if not new_instance:
LOG.debug(
"Could not create new instance "
"for non-live migration of instance %s" % instance_id)
return False
try:
LOG.debug("Detaching floating ip '%s' from instance %s" % (
floating_ip, instance_id))
# We detach the floating ip from the current instance
instance.remove_floating_ip(floating_ip)
LOG.debug(
"Attaching floating ip '%s' to the new instance %s" % (
floating_ip, new_instance.id))
# We attach the same floating ip to the new instance
new_instance.add_floating_ip(floating_ip)
except Exception as e:
LOG.debug(e)
new_host_name = getattr(new_instance, "OS-EXT-SRV-ATTR:host")
# Deleting the old instance (because no more useful)
delete_ok = self.delete_instance(instance_id)
if not delete_ok:
LOG.debug("Could not delete instance: %s" % instance_id)
return False
LOG.debug(
"Instance %s has been successfully migrated "
"to new host '%s' and its new id is %s." % (
instance_id, new_host_name, new_instance.id))
return True
def built_in_non_live_migrate_instance(self, instance_id, hypervisor_id):
"""This method uses the Nova built-in non-live migrate()
action to migrate a given instance.
It returns True if the migration was successful, False otherwise.
:param instance_id: the unique id of the instance to migrate.
"""
LOG.debug(
"Trying a Nova built-in non-live "
"migrate of instance %s ..." % instance_id)
# Looking for the instance to migrate
instance = self.find_instance(instance_id)
if not instance:
LOG.debug("Instance not found: %s" % instance_id)
return False
else:
host_name = getattr(instance, 'OS-EXT-SRV-ATTR:host')
LOG.debug(
"Instance %s found on host '%s'." % (instance_id, host_name))
instance.migrate()
# Poll at 5 second intervals, until the status is as expected
if self.wait_for_instance_status(instance,
('VERIFY_RESIZE', 'ERROR'),
5, 10):
instance = self.nova.servers.get(instance.id)
if instance.status == 'VERIFY_RESIZE':
host_name = getattr(instance, 'OS-EXT-SRV-ATTR:host')
LOG.debug(
"Instance %s has been successfully "
"migrated to host '%s'." % (
instance_id, host_name))
# We need to confirm that the resize() operation
# has succeeded in order to
# get back instance state to 'ACTIVE'
instance.confirm_resize()
return True
elif instance.status == 'ERROR':
LOG.debug("Instance %s migration failed" % instance_id)
return False
def live_migrate_instance(self, instance_id, dest_hostname,
block_migration=True, retry=120):
"""This method uses the Nova built-in live_migrate()
action to do a live migration of a given instance.
It returns True if the migration was successful,
False otherwise.
:param instance_id: the unique id of the instance to migrate.
:param dest_hostname: the name of the destination compute node.
:param block_migration: No shared storage is required.
"""
LOG.debug("Trying a live migrate of instance %s to host '%s'" % (
instance_id, dest_hostname))
# Looking for the instance to migrate
instance = self.find_instance(instance_id)
if not instance:
LOG.debug("Instance not found: %s" % instance_id)
return False
else:
host_name = getattr(instance, 'OS-EXT-SRV-ATTR:host')
LOG.debug(
"Instance %s found on host '%s'." % (instance_id, host_name))
instance.live_migrate(host=dest_hostname,
block_migration=block_migration,
disk_over_commit=True)
while getattr(instance,
'OS-EXT-SRV-ATTR:host') != dest_hostname \
and retry:
instance = self.nova.servers.get(instance.id)
LOG.debug(
"Waiting the migration of " + str(
instance.human_id) + " to " +
getattr(instance,
'OS-EXT-SRV-ATTR:host'))
time.sleep(1)
retry -= 1
host_name = getattr(instance, 'OS-EXT-SRV-ATTR:host')
if host_name != dest_hostname:
return False
LOG.debug(
"Live migration succeeded : "
"instance %s is now on host '%s'." % (
instance_id, host_name))
return True
return False
def enable_service_nova_compute(self, hostname):
if self.nova.services.enable(host=hostname,
binary='nova-compute'). \
status == 'enabled':
return True
else:
return False
def disable_service_nova_compute(self, hostname):
if self.nova.services.disable(host=hostname,
binary='nova-compute'). \
status == 'disabled':
return True
else:
return False
def set_host_offline(self, hostname):
# See API on http://developer.openstack.org/api-ref-compute-v2.1.html
# especially the PUT request
# regarding this resource : /v2.1/os-hosts/​{host_name}​
#
# The following body should be sent :
# {
# "host": {
# "host": "65c5d5b7e3bd44308e67fc50f362aee6",
# "maintenance_mode": "off_maintenance",
# "status": "enabled"
# }
# }
# Voir ici
# https://github.com/openstack/nova/
# blob/master/nova/virt/xenapi/host.py
# set_host_enabled(self, enabled):
# Sets the compute host's ability to accept new instances.
# host_maintenance_mode(self, host, mode):
# Start/Stop host maintenance window.
# On start, it triggers guest VMs evacuation.
host = self.nova.hosts.get(hostname)
if not host:
LOG.debug("host not found: %s" % hostname)
return False
else:
host[0].update(
{"maintenance_mode": "disable", "status": "disable"})
return True
def create_image_from_instance(self, instance_id, image_name,
metadata={"reason": "instance_migrate"}):
"""This method creates a new image from a given instance.
It waits for this image to be in 'active' state before returning.
It returns the unique UUID of the created image if successful,
None otherwise
:param instance_id: the uniqueid of
the instance to backup as an image.
:param image_name: the name of the image to create.
:param metadata: a dictionary containing the list of
key-value pairs to associate to the image as metadata.
"""
if self.glance is None:
glance_endpoint = self.keystone. \
service_catalog.url_for(service_type='image',
endpoint_type='publicURL')
self.glance = glclient.Client(glance_endpoint,
token=self.keystone.auth_token)
LOG.debug(
"Trying to create an image from instance %s ..." % instance_id)
# Looking for the instance
instance = self.find_instance(instance_id)
if not instance:
LOG.debug("Instance not found: %s" % instance_id)
return None
else:
host_name = getattr(instance, 'OS-EXT-SRV-ATTR:host')
LOG.debug(
"Instance %s found on host '%s'." % (instance_id, host_name))
# We need to wait for an appropriate status
# of the instance before we can build an image from it
if self.wait_for_instance_status(instance, ('ACTIVE', 'SHUTOFF'),
5,
10):
image_uuid = self.nova.servers.create_image(instance_id,
image_name,
metadata)
image = self.glance.images.get(image_uuid)
# Waiting for the new image to be officially in ACTIVE state
# in order to make sure it can be used
status = image.status
retry = 10
while status != 'active' and status != 'error' and retry:
time.sleep(5)
retry -= 1
# Retrieve the instance again so the status field updates
image = self.glance.images.get(image_uuid)
status = image.status
LOG.debug("Current image status: %s" % status)
if not image:
LOG.debug("Image not found: %s" % image_uuid)
else:
LOG.debug(
"Image %s successfully created for instance %s" % (
image_uuid, instance_id))
return image_uuid
return None
def delete_instance(self, instance_id):
"""This method deletes a given instance.
:param instance_id: the unique id of the instance to delete.
"""
LOG.debug("Trying to remove instance %s ..." % instance_id)
instance = self.find_instance(instance_id)
if not instance:
LOG.debug("Instance not found: %s" % instance_id)
return False
else:
self.nova.servers.delete(instance_id)
LOG.debug("Instance %s removed." % instance_id)
return True
def stop_instance(self, instance_id):
"""This method stops a given instance.
:param instance_id: the unique id of the instance to stop.
"""
LOG.debug("Trying to stop instance %s ..." % instance_id)
instance = self.find_instance(instance_id)
if not instance:
LOG.debug("Instance not found: %s" % instance_id)
return False
else:
self.nova.servers.stop(instance_id)
if self.wait_for_vm_state(instance, "stopped", 8, 10):
LOG.debug("Instance %s stopped." % instance_id)
return True
else:
return False
def wait_for_vm_state(self, server, vm_state, retry, sleep):
"""Waits for server to be in vm_state which can be one of the following :
active, stopped
:param server: server object.
:param vm_state: for which state we are waiting for
:param retry: how many times to retry
:param sleep: seconds to sleep between the retries
"""
if not server:
return False
while getattr(server, 'OS-EXT-STS:vm_state') != vm_state and retry:
time.sleep(sleep)
server = self.nova.servers.get(server)
retry -= 1
return getattr(server, 'OS-EXT-STS:vm_state') == vm_state
def wait_for_instance_status(self, instance, status_list, retry, sleep):
"""Waits for instance to be in status which can be one of the following
: BUILD, ACTIVE, ERROR, VERIFY_RESIZE, SHUTOFF
:param instance: instance object.
:param status_list: tuple containing the list of
status we are waiting for
:param retry: how many times to retry
:param sleep: seconds to sleep between the retries
"""
if not instance:
return False
while instance.status not in status_list and retry:
LOG.debug("Current instance status: %s" % instance.status)
time.sleep(sleep)
instance = self.nova.servers.get(instance.id)
retry -= 1
LOG.debug("Current instance status: %s" % instance.status)
return instance.status in status_list
def create_instance(self, hypervisor_id, inst_name="test", image_id=None,
flavor_name="m1.tiny",
sec_group_list=["default"],
network_names_list=["private"], keypair_name="mykeys",
create_new_floating_ip=True,
block_device_mapping_v2=None):
"""This method creates a new instance.
It also creates, if requested, a new floating IP and associates
it with the new instance
It returns the unique id of the created instance.
"""
LOG.debug(
"Trying to create new instance '%s' "
"from image '%s' with flavor '%s' ..." % (
inst_name, image_id, flavor_name))
# TODO(jed) wait feature
# Allow admin users to view any keypair
# https://bugs.launchpad.net/nova/+bug/1182965
if not self.nova.keypairs.findall(name=keypair_name):
LOG.debug("Key pair '%s' not found with user '%s'" % (
keypair_name, self.user))
return
else:
LOG.debug("Key pair '%s' found with user '%s'" % (
keypair_name, self.user))
try:
image = self.nova.images.get(image_id)
except nvexceptions.NotFound:
LOG.debug("Image '%s' not found " % image_id)
return
try:
flavor = self.nova.flavors.find(name=flavor_name)
except nvexceptions.NotFound:
LOG.debug("Flavor '%s' not found " % flavor_name)
return
# Make sure all security groups exist
for sec_group_name in sec_group_list:
try:
self.nova.security_groups.find(name=sec_group_name)
except nvexceptions.NotFound:
LOG.debug("Security group '%s' not found " % sec_group_name)
return
net_list = list()
for network_name in network_names_list:
nic_id = self.get_network_id_from_name(network_name)
if not nic_id:
LOG.debug("Network '%s' not found " % network_name)
return
net_obj = {"net-id": nic_id}
net_list.append(net_obj)
s = self.nova.servers
instance = s.create(inst_name,
image, flavor=flavor,
key_name=keypair_name,
security_groups=sec_group_list,
nics=net_list,
block_device_mapping_v2=block_device_mapping_v2,
availability_zone="nova:" +
hypervisor_id)
# Poll at 5 second intervals, until the status is no longer 'BUILD'
if instance:
if self.wait_for_instance_status(instance,
('ACTIVE', 'ERROR'), 5, 10):
instance = self.nova.servers.get(instance.id)
if create_new_floating_ip and instance.status == 'ACTIVE':
LOG.debug(
"Creating a new floating IP"
" for instance '%s'" % instance.id)
# Creating floating IP for the new instance
floating_ip = self.nova.floating_ips.create()
instance.add_floating_ip(floating_ip)
LOG.debug("Instance %s associated to Floating IP '%s'" % (
instance.id, floating_ip.ip))
return instance
def get_network_id_from_name(self, net_name="private"):
"""This method returns the unique id of the provided network name"""
if self.neutron is None:
self.neutron = netclient.Client('2.0', session=self.session)
self.neutron.format = 'json'
networks = self.neutron.list_networks(name=net_name)
# LOG.debug(networks)
network_id = networks['networks'][0]['id']
return network_id
def get_vms_by_hypervisor(self, host):
return [vm for vm in
self.nova.servers.list(search_opts={"all_tenants": True})
if self.get_hostname(vm) == host]
def get_hostname(self, vm):
return str(getattr(vm, 'OS-EXT-SRV-ATTR:host'))
def get_flavor_instance(self, instance, cache):
fid = instance.flavor['id']
if fid in cache:
flavor = cache.get(fid)
else:
try:
flavor = self.nova.flavors.get(fid)
except ciexceptions.NotFound:
flavor = None
cache[fid] = flavor
attr_defaults = [('name', 'unknown-id-%s' % fid),
('vcpus', 0), ('ram', 0), ('disk', 0),
('ephemeral', 0)]
for attr, default in attr_defaults:
if not flavor:
instance.flavor[attr] = default
continue
instance.flavor[attr] = getattr(flavor, attr, default)

73
watcher/applier/framework/command_executor.py Normal file
View File

@ -0,0 +1,73 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from watcher.applier.framework.default_command_mapper import \
DefaultCommandMapper
from watcher.applier.framework.deploy_phase import DeployPhase
from watcher.applier.framework.messaging.events import Events
from watcher.common.messaging.events.event import Event
from watcher.objects import Action
from watcher.objects.action_plan import Status
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
class CommandExecutor(object):
def __init__(self, manager_applier, context):
self.manager_applier = manager_applier
self.context = context
self.deploy = DeployPhase(self)
self.mapper = DefaultCommandMapper()
def get_primitive(self, action):
return self.mapper.build_primitive_command(action)
def notify(self, action, state):
db_action = Action.get_by_uuid(self.context, action.uuid)
db_action.state = state
db_action.save()
event = Event()
event.set_type(Events.LAUNCH_ACTION)
event.set_data({})
payload = {'action_uuid': action.uuid,
'action_status': state}
self.manager_applier.topic_status.publish_event(event.get_type().name,
payload)
def execute(self, actions):
for action in actions:
try:
self.notify(action, Status.ONGOING)
primitive = self.get_primitive(action)
result = self.deploy.execute_primitive(primitive)
if result is False:
self.notify(action, Status.FAILED)
self.deploy.rollback()
return False
else:
self.deploy.populate(primitive)
self.notify(action, Status.SUCCESS)
except Exception as e:
LOG.error(
"The applier module failed to execute the action" + str(
action) + " with the exception : " + unicode(e))
LOG.error("Trigger a rollback")
self.notify(action, Status.FAILED)
self.deploy.rollback()
return False
return True

35
watcher/applier/framework/default_applier.py Normal file
View File

@ -0,0 +1,35 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from watcher.applier.api.applier import Applier
from watcher.applier.framework.command_executor import CommandExecutor
from watcher.objects import Action
from watcher.objects import ActionPlan
class DefaultApplier(Applier):
def __init__(self, manager_applier, context):
self.manager_applier = manager_applier
self.context = context
self.executor = CommandExecutor(manager_applier, context)
def execute(self, action_plan_uuid):
action_plan = ActionPlan.get_by_uuid(self.context, action_plan_uuid)
# todo(jed) remove direct access to dbapi need filter in object
actions = Action.dbapi.get_action_list(self.context,
filters={
'action_plan_id':
action_plan.id})
return self.executor.execute(actions)

46
watcher/applier/framework/default_command_mapper.py Normal file
View File

@ -0,0 +1,46 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from watcher.applier.api.command_mapper import CommandMapper
from watcher.applier.framework.command.hypervisor_state_command import \
HypervisorStateCommand
from watcher.applier.framework.command.migrate_command import MigrateCommand
from watcher.applier.framework.command.nop_command import NopCommand
from watcher.applier.framework.command.power_state_command import \
PowerStateCommand
from watcher.common.exception import ActionNotFound
from watcher.decision_engine.framework.default_planner import Primitives
class DefaultCommandMapper(CommandMapper):
def build_primitive_command(self, action):
if action.action_type == Primitives.COLD_MIGRATE.value:
return MigrateCommand(action.applies_to, Primitives.COLD_MIGRATE,
action.src,
action.dst)
elif action.action_type == Primitives.LIVE_MIGRATE.value:
return MigrateCommand(action.applies_to, Primitives.COLD_MIGRATE,
action.src,
action.dst)
elif action.action_type == Primitives.HYPERVISOR_STATE.value:
return HypervisorStateCommand(action.applies_to, action.parameter)
elif action.action_type == Primitives.POWER_STATE.value:
return PowerStateCommand()
elif action.action_type == Primitives.NOP.value:
return NopCommand()
else:
raise ActionNotFound()

46
watcher/applier/framework/deploy_phase.py Normal file
View File

@ -0,0 +1,46 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
class DeployPhase(object):
def __init__(self, executor):
# todo(jed) oslo_conf 10 secondes
self.maxTimeout = 100000
self.commands = []
self.executor = executor
def set_max_time(self, mt):
self.maxTimeout = mt
def get_max_time(self):
return self.maxTimeout
def populate(self, action):
self.commands.append(action)
def execute_primitive(self, primitive):
futur = primitive.execute(primitive)
return futur.result(self.get_max_time())
def rollback(self):
reverted = sorted(self.commands, reverse=True)
for primitive in reverted:
try:
self.execute_primitive(primitive)
except Exception as e:
LOG.error(e)

98
watcher/applier/framework/manager_applier.py Normal file
View File

@ -0,0 +1,98 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from concurrent.futures import ThreadPoolExecutor
from oslo_config import cfg
from watcher.applier.framework.messaging.trigger_action_plan import \
TriggerActionPlan
from watcher.common.messaging.messaging_core import MessagingCore
from watcher.common.messaging.notification_handler import NotificationHandler
from watcher.decision_engine.framework.messaging.events import Events
from watcher.openstack.common import log
CONF = cfg.CONF
LOG = log.getLogger(__name__)
# Register options
APPLIER_MANAGER_OPTS = [
cfg.IntOpt('applier_worker', default='1', help='The number of worker'),
cfg.StrOpt('topic_control',
default='watcher.applier.control',
help='The topic name used for'
'control events, this topic '
'used for rpc call '),
cfg.StrOpt('topic_status',
default='watcher.applier.status',
help='The topic name used for '
'status events, this topic '
'is used so as to notify'
'the others components '
'of the system'),
cfg.StrOpt('publisher_id',
default='watcher.applier.api',
help='The identifier used by watcher '
'module on the message broker')
]
CONF = cfg.CONF
opt_group = cfg.OptGroup(name='watcher_applier',
title='Options for the Applier messaging'
'core')
CONF.register_group(opt_group)
CONF.register_opts(APPLIER_MANAGER_OPTS, opt_group)
CONF.import_opt('admin_user', 'keystonemiddleware.auth_token',
group='keystone_authtoken')
CONF.import_opt('admin_tenant_name', 'keystonemiddleware.auth_token',
group='keystone_authtoken')
CONF.import_opt('admin_password', 'keystonemiddleware.auth_token',
group='keystone_authtoken')
CONF.import_opt('auth_uri', 'keystonemiddleware.auth_token',
group='keystone_authtoken')
class ApplierManager(MessagingCore):
API_VERSION = '1.0'
# todo(jed) need workflow
def __init__(self):
MessagingCore.__init__(self, CONF.watcher_applier.publisher_id,
CONF.watcher_applier.topic_control,
CONF.watcher_applier.topic_status)
# shared executor of the workflow
self.executor = ThreadPoolExecutor(max_workers=1)
self.handler = NotificationHandler(self.publisher_id)
self.handler.register_observer(self)
self.add_event_listener(Events.ALL, self.event_receive)
# trigger action_plan
self.topic_control.add_endpoint(TriggerActionPlan(self))
def join(self):
self.topic_control.join()
self.topic_status.join()
def event_receive(self, event):
try:
request_id = event.get_request_id()
event_type = event.get_type()
data = event.get_data()
LOG.debug("request id => %s" % request_id)
LOG.debug("type_event => %s" % str(event_type))
LOG.debug("data => %s" % str(data))
except Exception as e:
LOG.error("evt %s" % e.message)
raise e

0
watcher/applier/framework/messaging/__init__.py Normal file
View File

22
watcher/applier/framework/messaging/events.py Normal file
View File

@ -0,0 +1,22 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from enum import Enum
class Events(Enum):
LAUNCH_ACTION_PLAN = "launch_action_plan"
LAUNCH_ACTION = "launch_action"

65
watcher/applier/framework/messaging/launch_action_plan.py Normal file
View File

@ -0,0 +1,65 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from watcher.applier.api.messaging.applier_command import ApplierCommand
from watcher.applier.framework.default_applier import DefaultApplier
from watcher.applier.framework.messaging.events import Events
from watcher.common.messaging.events.event import Event
from watcher.objects.action_plan import ActionPlan
from watcher.objects.action_plan import Status
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
class LaunchActionPlanCommand(ApplierCommand):
def __init__(self, context, manager_applier, action_plan_uuid):
self.ctx = context
self.action_plan_uuid = action_plan_uuid
self.manager_applier = manager_applier
def notify(self, uuid, event_type, status):
action_plan = ActionPlan.get_by_uuid(self.ctx, uuid)
action_plan.state = status
action_plan.save()
event = Event()
event.set_type(event_type)
event.set_data({})
payload = {'action_plan__uuid': uuid,
'action_plan_status': status}
self.manager_applier.topic_status.publish_event(event.get_type().name,
payload)
def execute(self):
try:
# update state
self.notify(self.action_plan_uuid,
Events.LAUNCH_ACTION_PLAN,
Status.ONGOING)
applier = DefaultApplier(self.manager_applier, self.ctx)
result = applier.execute(self.action_plan_uuid)
except Exception as e:
result = False
LOG.error("Launch Action Plan " + unicode(e))
finally:
if result is True:
status = Status.SUCCESS
else:
status = Status.FAILED
# update state
self.notify(self.action_plan_uuid, Events.LAUNCH_ACTION_PLAN,
status)

44
watcher/applier/framework/messaging/trigger_action_plan.py Normal file
View File

@ -0,0 +1,44 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from watcher.applier.framework.messaging.launch_action_plan import \
LaunchActionPlanCommand
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
class TriggerActionPlan(object):
def __init__(self, manager_applier):
self.manager_applier = manager_applier
def do_launch_action_plan(self, context, action_plan_uuid):
try:
cmd = LaunchActionPlanCommand(context,
self.manager_applier,
action_plan_uuid)
cmd.execute()
except Exception as e:
LOG.error("do_launch_action_plan " + unicode(e))
def launch_action_plan(self, context, action_plan_uuid):
LOG.debug("Trigger ActionPlan %s" % action_plan_uuid)
# submit
self.manager_applier.executor.submit(self.do_launch_action_plan,
context,
action_plan_uuid)
return action_plan_uuid

70
watcher/applier/framework/rpcapi.py Normal file
View File

@ -0,0 +1,70 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from oslo_config import cfg
import oslo_messaging as om
from watcher.applier.framework.manager_applier import APPLIER_MANAGER_OPTS
from watcher.applier.framework.manager_applier import opt_group
from watcher.common import exception
from watcher.common import utils
from watcher.common.messaging.messaging_core import MessagingCore
from watcher.common.messaging.notification_handler import NotificationHandler
from watcher.common.messaging.utils.transport_url_builder import \
TransportUrlBuilder
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
CONF = cfg.CONF
CONF.register_group(opt_group)
CONF.register_opts(APPLIER_MANAGER_OPTS, opt_group)
class ApplierAPI(MessagingCore):
MessagingCore.API_VERSION = '1.0'
def __init__(self):
MessagingCore.__init__(self, CONF.watcher_applier.publisher_id,
CONF.watcher_applier.topic_control,
CONF.watcher_applier.topic_status)
self.handler = NotificationHandler(self.publisher_id)
self.handler.register_observer(self)
self.topic_status.add_endpoint(self.handler)
transport = om.get_transport(CONF, TransportUrlBuilder().url)
target = om.Target(
topic=CONF.watcher_applier.topic_control,
version=MessagingCore.API_VERSION)
self.client = om.RPCClient(transport, target,
serializer=self.serializer)
def launch_action_plan(self, context, action_plan_uuid=None):
if not utils.is_uuid_like(action_plan_uuid):
raise exception.InvalidUuidOrName(name=action_plan_uuid)
return self.client.call(
context.to_dict(), 'launch_action_plan',
action_plan_uuid=action_plan_uuid)
def event_receive(self, event):
try:
pass
except Exception as e:
LOG.error("evt %s" % e.message)
raise e

0
watcher/cmd/__init__.py Normal file
View File

57
watcher/cmd/api.py Normal file
View File

@ -0,0 +1,57 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2013 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Starter script for the Watcher API service."""
import logging as std_logging
import os
from wsgiref import simple_server
from oslo_config import cfg
from watcher.api import app as api_app
from watcher.common.i18n import _
from watcher.openstack.common import log as logging
from watcher import service
LOG = logging.getLogger(__name__)
def main():
service.prepare_service()
app = api_app.setup_app()
# Create the WSGI server and start it
host, port = cfg.CONF.api.host, cfg.CONF.api.port
srv = simple_server.make_server(host, port, app)
logging.setup('watcher')
LOG.info(_('Starting server in PID %s') % os.getpid())
LOG.debug("Watcher configuration:")
cfg.CONF.log_opt_values(LOG, std_logging.DEBUG)
if host == '0.0.0.0':
LOG.info(_('serving on 0.0.0.0:%(port)s, '
'view at http://127.0.0.1:%(port)s') %
dict(port=port))
else:
LOG.info(_('serving on http://%(host)s:%(port)s') %
dict(host=host, port=port))
srv.serve_forever()

44
watcher/cmd/applier.py Normal file
View File

@ -0,0 +1,44 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2013 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Starter script for the Applier service."""
import logging as std_logging
import os
import sys
from oslo_config import cfg
from watcher.applier.framework.manager_applier import ApplierManager
from watcher.openstack.common._i18n import _LI
from watcher.openstack.common import log as logging
LOG = logging.getLogger(__name__)
def main():
cfg.CONF(sys.argv[1:], project='watcher')
logging.setup('watcher')
LOG.info(_LI('Starting server in PID %s') % os.getpid())
LOG.debug("Configuration:")
cfg.CONF.log_opt_values(LOG, std_logging.DEBUG)
server = ApplierManager()
server.connect()
server.join()

115
watcher/cmd/dbmanage.py Normal file
View File

@ -0,0 +1,115 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2013 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Run storage database migration.
"""
import sys
from oslo_config import cfg
from watcher.common import service
from watcher.db import migration
CONF = cfg.CONF
class DBCommand(object):
def upgrade(self):
migration.upgrade(CONF.command.revision)
def downgrade(self):
migration.downgrade(CONF.command.revision)
def revision(self):
migration.revision(CONF.command.message, CONF.command.autogenerate)
def stamp(self):
migration.stamp(CONF.command.revision)
def version(self):
print(migration.version())
def create_schema(self):
migration.create_schema()
def add_command_parsers(subparsers):
command_object = DBCommand()
parser = subparsers.add_parser(
'upgrade',
help="Upgrade the database schema to the latest version. "
"Optionally, use --revision to specify an alembic revision "
"string to upgrade to.")
parser.set_defaults(func=command_object.upgrade)
parser.add_argument('--revision', nargs='?')
parser = subparsers.add_parser(
'downgrade',
help="Downgrade the database schema to the oldest revision. "
"While optional, one should generally use --revision to "
"specify the alembic revision string to downgrade to.")
parser.set_defaults(func=command_object.downgrade)
parser.add_argument('--revision', nargs='?')
parser = subparsers.add_parser('stamp')
parser.add_argument('--revision', nargs='?')
parser.set_defaults(func=command_object.stamp)
parser = subparsers.add_parser(
'revision',
help="Create a new alembic revision. "
"Use --message to set the message string.")
parser.add_argument('-m', '--message')
parser.add_argument('--autogenerate', action='store_true')
parser.set_defaults(func=command_object.revision)
parser = subparsers.add_parser(
'version',
help="Print the current version information and exit.")
parser.set_defaults(func=command_object.version)
parser = subparsers.add_parser(
'create_schema',
help="Create the database schema.")
parser.set_defaults(func=command_object.create_schema)
command_opt = cfg.SubCommandOpt('command',
title='Command',
help='Available commands',
handler=add_command_parsers)
CONF.register_cli_opt(command_opt)
def main():
# this is hack to work with previous usage of watcher-dbsync
# pls change it to watcher-dbsync upgrade
valid_commands = set([
'upgrade', 'downgrade', 'revision',
'version', 'stamp', 'create_schema',
])
if not set(sys.argv) & valid_commands:
sys.argv.append('upgrade')
service.prepare_service(sys.argv)
CONF.command.func()

45
watcher/cmd/decisionengine.py Normal file
View File

@ -0,0 +1,45 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2013 Hewlett-Packard Development Company, L.P.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Starter script for the Decision Engine manager service."""
import logging as std_logging
import os
import sys
from oslo_config import cfg
from watcher.decision_engine.framework.manager_decision_engine import \
DecisionEngineManager
from watcher.openstack.common._i18n import _LI
from watcher.openstack.common import log as logging
LOG = logging.getLogger(__name__)
def main():
cfg.CONF(sys.argv[1:], project='watcher')
logging.setup('watcher')
LOG.info(_LI('Starting server in PID %s') % os.getpid())
LOG.debug("Configuration:")
cfg.CONF.log_opt_values(LOG, std_logging.DEBUG)
server = DecisionEngineManager()
server.connect()
server.join()

0
watcher/common/__init__.py Normal file
View File

30
watcher/common/config.py Normal file
View File

@ -0,0 +1,30 @@
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
# Copyright 2012 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo_config import cfg
from watcher.common import rpc
from watcher import version
def parse_args(argv, default_config_files=None):
rpc.set_defaults(control_exchange='watcher')
cfg.CONF(argv[1:],
project='watcher',
version=version.version_info.release_string(),
default_config_files=default_config_files)
rpc.init(cfg.CONF)

72
watcher/common/context.py Normal file
View File

@ -0,0 +1,72 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo_context import context
class RequestContext(context.RequestContext):
"""Extends security contexts from the OpenStack common library."""
def __init__(self, auth_token=None, auth_url=None, domain_id=None,
domain_name=None, user=None, user_id=None, project=None,
project_id=None, is_admin=False, is_public_api=False,
read_only=False, show_deleted=False, request_id=None,
trust_id=None, auth_token_info=None):
"""Stores several additional request parameters:
:param domain_id: The ID of the domain.
:param domain_name: The name of the domain.
:param is_public_api: Specifies whether the request should be processed
without authentication.
"""
self.is_public_api = is_public_api
self.user_id = user_id
self.project = project
self.project_id = project_id
self.domain_id = domain_id
self.domain_name = domain_name
self.auth_url = auth_url
self.auth_token_info = auth_token_info
self.trust_id = trust_id
super(RequestContext, self).__init__(auth_token=auth_token,
user=user, tenant=project,
is_admin=is_admin,
read_only=read_only,
show_deleted=show_deleted,
request_id=request_id)
def to_dict(self):
return {'auth_token': self.auth_token,
'auth_url': self.auth_url,
'domain_id': self.domain_id,
'domain_name': self.domain_name,
'user': self.user,
'user_id': self.user_id,
'project': self.project,
'project_id': self.project_id,
'is_admin': self.is_admin,
'is_public_api': self.is_public_api,
'read_only': self.read_only,
'show_deleted': self.show_deleted,
'request_id': self.request_id,
'trust_id': self.trust_id,
'auth_token_info': self.auth_token_info}
@classmethod
def from_dict(cls, values):
return cls(**values)
def make_context(*args, **kwargs):
return RequestContext(*args, **kwargs)

253
watcher/common/exception.py Normal file
View File

@ -0,0 +1,253 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Watcher base exception handling.
Includes decorator for re-raising Watcher-type exceptions.
SHOULD include dedicated exception logging.
"""
from oslo_config import cfg
import six
from watcher.common.i18n import _
from watcher.common.i18n import _LE
from watcher.openstack.common import log as logging
LOG = logging.getLogger(__name__)
exc_log_opts = [
cfg.BoolOpt('fatal_exception_format_errors',
default=False,
help='Make exception message format errors fatal.'),
]
CONF = cfg.CONF
CONF.register_opts(exc_log_opts)
def _cleanse_dict(original):
"""Strip all admin_password, new_pass, rescue_pass keys from a dict."""
return dict((k, v) for k, v in original.iteritems() if "_pass" not in k)
class WatcherException(Exception):
"""Base Watcher Exception
To correctly use this class, inherit from it and define
a 'message' property. That message will get printf'd
with the keyword arguments provided to the constructor.
"""
message = _("An unknown exception occurred.")
code = 500
headers = {}
safe = False
def __init__(self, message=None, **kwargs):
self.kwargs = kwargs
if 'code' not in self.kwargs:
try:
self.kwargs['code'] = self.code
except AttributeError:
pass
if not message:
try:
message = self.message % kwargs
except Exception as e:
# kwargs doesn't match a variable in the message
# log the issue and the kwargs
LOG.exception(_LE('Exception in string format operation'))
for name, value in kwargs.iteritems():
LOG.error("%s: %s" % (name, value))
if CONF.fatal_exception_format_errors:
raise e
else:
# at least get the core message out if something happened
message = self.message
super(WatcherException, self).__init__(message)
def __str__(self):
"""Encode to utf-8 then wsme api can consume it as well."""
if not six.PY3:
return unicode(self.args[0]).encode('utf-8')
else:
return self.args[0]
def __unicode__(self):
return self.message
def format_message(self):
if self.__class__.__name__.endswith('_Remote'):
return self.args[0]
else:
return six.text_type(self)
class NotAuthorized(WatcherException):
message = _("Not authorized.")
code = 403
class OperationNotPermitted(NotAuthorized):
message = _("Operation not permitted.")
class Invalid(WatcherException):
message = _("Unacceptable parameters.")
code = 400
class ObjectNotFound(WatcherException):
message = _("The %(name)s %(id)s could not be found.")
class Conflict(WatcherException):
message = _('Conflict.')
code = 409
class ResourceNotFound(ObjectNotFound):
message = _("The %(name)s resource %(id)s could not be found.")
code = 404
class InvalidIdentity(Invalid):
message = _("Expected an uuid or int but received %(identity)s.")
class InvalidGoal(Invalid):
message = _("Goal %(goal)s is not defined in Watcher configuration file.")
# Cannot be templated as the error syntax varies.
# msg needs to be constructed when raised.
class InvalidParameterValue(Invalid):
message = _("%(err)s")
class InvalidUUID(Invalid):
message = _("Expected a uuid but received %(uuid)s.")
class InvalidName(Invalid):
message = _("Expected a logical name but received %(name)s.")
class InvalidUuidOrName(Invalid):
message = _("Expected a logical name or uuid but received %(name)s.")
class AuditTemplateNotFound(ResourceNotFound):
message = _("AuditTemplate %(audit_template)s could not be found.")
class AuditTemplateAlreadyExists(Conflict):
message = _("An audit_template with UUID %(uuid)s or name %(name)s "
"already exists.")
class AuditTemplateReferenced(Invalid):
message = _("AuditTemplate %(audit_template)s is referenced by one or "
"multiple audit.")
class AuditNotFound(ResourceNotFound):
message = _("Audit %(audit)s could not be found.")
class AuditAlreadyExists(Conflict):
message = _("An audit with UUID %(uuid)s already exists.")
class AuditReferenced(Invalid):
message = _("Audit %(audit)s is referenced by one or multiple action "
"plans.")
class ActionPlanNotFound(ResourceNotFound):
message = _("ActionPlan %(action plan)s could not be found.")
class ActionPlanAlreadyExists(Conflict):
message = _("An action plan with UUID %(uuid)s already exists.")
class ActionPlanReferenced(Invalid):
message = _("Action Plan %(action_plan)s is referenced by one or "
"multiple actions.")
class ActionNotFound(ResourceNotFound):
message = _("Action %(action)s could not be found.")
class ActionAlreadyExists(Conflict):
message = _("An action with UUID %(uuid)s already exists.")
class ActionReferenced(Invalid):
message = _("Action plan %(action_plan)s is referenced by one or "
"multiple goals.")
class ActionFilterCombinationProhibited(Invalid):
message = _("Filtering actions on both audit and action-plan is "
"prohibited.")
class HTTPNotFound(ResourceNotFound):
pass
class PatchError(Invalid):
message = _("Couldn't apply patch '%(patch)s'. Reason: %(reason)s")
# decision engine
class ClusterEmpty(WatcherException):
message = _("The list of hypervisor(s) in the cluster is empty.'")
class MetricCollectorNotDefined(WatcherException):
message = _("The metrics resource collector is not defined.'")
class ClusteStateNotDefined(WatcherException):
message = _("the cluster state is not defined")
# Model
class VMNotFound(WatcherException):
message = _("The VM could not be found.")
class HypervisorNotFound(WatcherException):
message = _("The hypervisor could not be found.")
class MetaActionNotFound(WatcherException):
message = _("The Meta-Action could not be found.")

26
watcher/common/i18n.py Normal file
View File

@ -0,0 +1,26 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import oslo_i18n
_translators = oslo_i18n.TranslatorFactory(domain='watcher')
oslo_i18n.enable_lazy()
_ = _translators.primary
_LI = _translators.log_info
_LW = _translators.log_warning
_LE = _translators.log_error
_LC = _translators.log_critical

0
watcher/common/messaging/__init__.py Normal file
View File

0
watcher/common/messaging/events/__init__.py Normal file
View File

48
watcher/common/messaging/events/event.py Normal file
View File

@ -0,0 +1,48 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
class Event(object):
"""Generic event to use with EventDispatcher"""
def __init__(self, event_type=None, data=None, request_id=None):
"""Default constructor
:param event_type: the type of the event
:param data: a dictionary which contains data
:param request_id: a string which represent the uuid of the request
"""
self._type = event_type
self._data = data
self._request_id = request_id
def get_type(self):
return self._type
def set_type(self, type):
self._type = type
def get_data(self):
return self._data
def set_data(self, data):
self._data = data
def set_request_id(self, id):
self._request_id = id
def get_request_id(self):
return self._request_id

78
watcher/common/messaging/events/event_dispatcher.py Normal file
View File

@ -0,0 +1,78 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from watcher.decision_engine.framework.messaging.events import Events
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
class EventDispatcher(object):
"""Generic event dispatcher which listen and dispatch events"""
def __init__(self):
self._events = dict()
def __del__(self):
self._events = None
def has_listener(self, event_type, listener):
"""Return true if listener is register to event_type """
# Check for event type and for the listener
if event_type in self._events.keys():
return listener in self._events[event_type]
else:
return False
def dispatch_event(self, event):
LOG.debug("dispatch evt : %s" % str(event.get_type()))
"""
Dispatch an instance of Event class
"""
if Events.ALL in self._events.keys():
listeners = self._events[Events.ALL]
for listener in listeners:
listener(event)
# Dispatch the event to all the associated listeners
if event.get_type() in self._events.keys():
listeners = self._events[event.get_type()]
for listener in listeners:
listener(event)
def add_event_listener(self, event_type, listener):
"""Add an event listener for an event type"""
# Add listener to the event type
if not self.has_listener(event_type, listener):
listeners = self._events.get(event_type, [])
listeners.append(listener)
self._events[event_type] = listeners
def remove_event_listener(self, event_type, listener):
"""Remove event listener. """
# Remove the listener from the event type
if self.has_listener(event_type, listener):
listeners = self._events[event_type]
if len(listeners) == 1:
# Only this listener remains so remove the key
del self._events[event_type]
else:
# Update listeners chain
listeners.remove(listener)
self._events[event_type] = listeners

109
watcher/common/messaging/messaging_core.py Normal file
View File

@ -0,0 +1,109 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from oslo_config import cfg
from watcher.common.messaging.events.event_dispatcher import \
EventDispatcher
from watcher.common.messaging.messaging_handler import \
MessagingHandler
from watcher.common.rpc import RequestContextSerializer
from watcher.objects.base import WatcherObjectSerializer
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
CONF = cfg.CONF
WATCHER_MESSAGING_OPTS = [
cfg.StrOpt('notifier_driver',
default='messaging', help='The name of the driver used by'
' oslo messaging'),
cfg.StrOpt('executor',
default='eventlet', help='The name of a message executor, for'
'example: eventlet, blocking'),
cfg.StrOpt('protocol',
default='rabbit', help='The protocol used by the message'
' broker, for example rabbit'),
cfg.StrOpt('user',
default='guest', help='The username used by the message '
'broker'),
cfg.StrOpt('password',
default='guest', help='The password of user used by the '
'message broker'),
cfg.StrOpt('host',
default='localhost', help='The host where the message broker'
'is installed'),
cfg.StrOpt('port',
default='5672', help='The port used bythe message broker'),
cfg.StrOpt('virtual_host',
default='', help='The virtual host used by the message '
'broker')
]
CONF = cfg.CONF
opt_group = cfg.OptGroup(name='watcher_messaging',
title='Options for the messaging core')
CONF.register_group(opt_group)
CONF.register_opts(WATCHER_MESSAGING_OPTS, opt_group)
class MessagingCore(EventDispatcher):
API_VERSION = '1.0'
def __init__(self, publisher_id, topic_control, topic_status):
EventDispatcher.__init__(self)
self.serializer = RequestContextSerializer(WatcherObjectSerializer())
self.publisher_id = publisher_id
self.topic_control = self.build_topic(topic_control)
self.topic_status = self.build_topic(topic_status)
def build_topic(self, topic_name):
return MessagingHandler(self.publisher_id, topic_name, self,
self.API_VERSION, self.serializer)
def connect(self):
LOG.debug("connecting to rabbitMQ broker")
self.topic_control.start()
self.topic_status.start()
def disconnect(self):
LOG.debug("Disconnect to rabbitMQ broker")
self.topic_control.stop()
self.topic_status.stop()
def publish_control(self, event, payload):
return self.topic_control.publish_event(event, payload)
def publish_status(self, event, payload, request_id=None):
return self.topic_status.publish_event(event, payload, request_id)
def get_version(self):
return self.API_VERSION
def check_api_version(self, context):
api_manager_version = self.client.call(
context.to_dict(), 'check_api_version',
api_version=self.API_VERSION)
return api_manager_version
def response(self, evt, ctx, message):
payload = {
'request_id': ctx['request_id'],
'msg': message
}
self.publish_status(evt, payload)

107
watcher/common/messaging/messaging_handler.py Normal file
View File

@ -0,0 +1,107 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import eventlet
from oslo_config import cfg
import oslo_messaging as om
from threading import Thread
from watcher.common.messaging.utils.transport_url_builder import \
TransportUrlBuilder
from watcher.common.rpc import JsonPayloadSerializer
from watcher.common.rpc import RequestContextSerializer
from watcher.openstack.common import log
eventlet.monkey_patch()
LOG = log.getLogger(__name__)
CONF = cfg.CONF
class MessagingHandler(Thread):
def __init__(self, publisher_id, topic_watcher, endpoint, version,
serializer=None):
Thread.__init__(self)
self.__server = None
self.__notifier = None
self.__endpoints = []
self.__topics = []
self._publisher_id = publisher_id
self._topic_watcher = topic_watcher
self.__endpoints.append(endpoint)
self.__version = version
self.__serializer = serializer
def add_endpoint(self, endpoint):
self.__endpoints.append(endpoint)
def remove_endpoint(self, endpoint):
if endpoint in self.__endpoints:
self.__endpoints.remove(endpoint)
def build_notifier(self):
serializer = RequestContextSerializer(JsonPayloadSerializer())
return om.Notifier(
self.transport,
driver=CONF.watcher_messaging.notifier_driver,
publisher_id=self._publisher_id,
topic=self._topic_watcher,
serializer=serializer)
def build_server(self, targets):
return om.get_rpc_server(self.transport, targets,
self.__endpoints,
executor=CONF.
watcher_messaging.executor,
serializer=self.__serializer)
def __build_transport_url(self):
return TransportUrlBuilder().url
def __config(self):
try:
self.transport = om.get_transport(
cfg.CONF,
url=self.__build_transport_url())
self.__notifier = self.build_notifier()
if 0 < len(self.__endpoints):
targets = om.Target(
topic=self._topic_watcher,
server=CONF.watcher_messaging.host,
version=self.__version)
self.__server = self.build_server(targets)
else:
LOG.warn("you have no defined endpoint, \
so you can only publish events")
except Exception as e:
LOG.error("configure : %s" % str(e.message))
def run(self):
LOG.debug("configure MessagingHandler for %s" % self._topic_watcher)
self.__config()
if len(self.__endpoints) > 0:
LOG.debug("Starting up server")
self.__server.start()
def stop(self):
LOG.debug('Stop up server')
self.__server.wait()
self.__server.stop()
def publish_event(self, event_type, payload, request_id=None):
self.__notifier.info({'version_api': self.__version,
'request_id': request_id},
{'event_id': event_type}, payload)

50
watcher/common/messaging/notification_handler.py Normal file
View File

@ -0,0 +1,50 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import eventlet
from oslo import messaging
from watcher.common.messaging.utils.observable import \
Observable
from watcher.openstack.common import log
eventlet.monkey_patch()
LOG = log.getLogger(__name__)
class NotificationHandler(Observable):
def __init__(self, publisher_id):
Observable.__init__(self)
self.publisher_id = publisher_id
def info(self, ctx, publisher_id, event_type, payload, metadata):
if publisher_id == self.publisher_id:
self.set_changed()
self.notify(ctx, publisher_id, event_type, metadata, payload)
return messaging.NotificationResult.HANDLED
def warn(self, ctx, publisher_id, event_type, payload, metadata):
if publisher_id == self.publisher_id:
self.set_changed()
self.notify(ctx, publisher_id, event_type, metadata, payload)
return messaging.NotificationResult.HANDLED
def error(self, ctx, publisher_id, event_type, payload, metadata):
if publisher_id == self.publisher_id:
self.set_changed()
self.notify(ctx, publisher_id, event_type, metadata, payload)
return messaging.NotificationResult.HANDLED

0
watcher/common/messaging/utils/__init__.py Normal file
View File

62
watcher/common/messaging/utils/observable.py Normal file
View File

@ -0,0 +1,62 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from watcher.common.messaging.utils.synchronization import \
Synchronization
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
class Observable(Synchronization):
def __init__(self):
self.__observers = []
self.changed = 0
Synchronization.__init__(self)
def set_changed(self):
self.changed = 1
def clear_changed(self):
self.changed = 0
def has_changed(self):
return self.changed
def register_observer(self, observer):
if observer not in self.__observers:
self.__observers.append(observer)
def unregister_observer(self, observer):
try:
self.__observers.remove(observer)
except ValueError:
pass
def notify(self, ctx=None, publisherid=None, event_type=None,
metadata=None, payload=None, modifier=None):
self.mutex.acquire()
try:
if not self.changed:
return
for observer in self.__observers:
if modifier != observer:
observer.update(self, ctx, metadata, publisherid,
event_type, payload)
self.clear_changed()
finally:
self.mutex.release()

22
watcher/common/messaging/utils/synchronization.py Normal file
View File

@ -0,0 +1,22 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import threading
class Synchronization(object):
def __init__(self):
self.mutex = threading.RLock()

35
watcher/common/messaging/utils/transport_url_builder.py Normal file
View File

@ -0,0 +1,35 @@
# -*- encoding: utf-8 -*-
# Copyright (c) 2015 b<>com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from oslo_config import cfg
from watcher.openstack.common import log
LOG = log.getLogger(__name__)
CONF = cfg.CONF
class TransportUrlBuilder(object):
@property
def url(self):
return "%s://%s:%s@%s:%s/%s" % (
CONF.watcher_messaging.protocol,
CONF.watcher_messaging.user,
CONF.watcher_messaging.password,
CONF.watcher_messaging.host,
CONF.watcher_messaging.port,
CONF.watcher_messaging.virtual_host
)

66
watcher/common/paths.py Normal file
View File

@ -0,0 +1,66 @@
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
# Copyright 2012 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import os
from oslo_config import cfg
PATH_OPTS = [
cfg.StrOpt('pybasedir',
default=os.path.abspath(os.path.join(os.path.dirname(__file__),
'../')),
help='Directory where the watcher python module is installed.'),
cfg.StrOpt('bindir',
default='$pybasedir/bin',
help='Directory where watcher binaries are installed.'),
cfg.StrOpt('state_path',
default='$pybasedir',
help="Top-level directory for maintaining watcher's state."),
]
CONF = cfg.CONF
CONF.register_opts(PATH_OPTS)
def basedir_def(*args):
"""Return an uninterpolated path relative to $pybasedir."""
return os.path.join('$pybasedir', *args)
def bindir_def(*args):
"""Return an uninterpolated path relative to $bindir."""
return os.path.join('$bindir', *args)
def state_path_def(*args):
"""Return an uninterpolated path relative to $state_path."""
return os.path.join('$state_path', *args)
def basedir_rel(*args):
"""Return a path relative to $pybasedir."""
return os.path.join(CONF.pybasedir, *args)
def bindir_rel(*args):
"""Return a path relative to $bindir."""
return os.path.join(CONF.bindir, *args)
def state_path_rel(*args):
"""Return a path relative to $state_path."""
return os.path.join(CONF.state_path, *args)

69
watcher/common/policy.py Normal file
View File

@ -0,0 +1,69 @@
# Copyright (c) 2011 OpenStack Foundation
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Policy Engine For Watcher."""
from oslo_concurrency import lockutils
from oslo_config import cfg
from watcher.openstack.common import policy
_ENFORCER = None
CONF = cfg.CONF
@lockutils.synchronized('policy_enforcer', 'watcher-')
def init_enforcer(policy_file=None, rules=None,
default_rule=None, use_conf=True):
"""Synchronously initializes the policy enforcer
:param policy_file: Custom policy file to use, if none is specified,
`CONF.policy_file` will be used.
:param rules: Default dictionary / Rules to use. It will be
considered just in the first instantiation.
:param default_rule: Default rule to use, CONF.default_rule will
be used if none is specified.
:param use_conf: Whether to load rules from config file.
"""
global _ENFORCER
if _ENFORCER:
return
_ENFORCER = policy.Enforcer(policy_file=policy_file,
rules=rules,
default_rule=default_rule,
use_conf=use_conf)
def get_enforcer():
"""Provides access to the single instance of Policy enforcer."""
if not _ENFORCER:
init_enforcer()
return _ENFORCER
def enforce(rule, target, creds, do_raise=False, exc=None, *args, **kwargs):
"""A shortcut for policy.Enforcer.enforce()
Checks authorization of a rule against the target and credentials.
"""
enforcer = get_enforcer()
return enforcer.enforce(rule, target, creds, do_raise=do_raise,
exc=exc, *args, **kwargs)

Some files were not shown because too many files have changed in this diff Show More