stackforge/billingstack
Code Issues Proposed changes

Retire stackforge/billingstack

Browse Source
This commit is contained in:
Monty Taylor
2015-10-17 16:02:34 -04:00
parent c378381306
commit d6f0ac4234
246 changed files with 5 additions and 24003 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.coveragerc
View File

@@ -1,7 +0,0 @@
[run]
branch = True
source = billingstack
omit = billingstack/tests/*,billingstack/openstack/*
[report]
ignore-errors = True

58
.gitignore vendored
View File

@@ -1,58 +0,0 @@
*.py[cod]
# C extensions
*.so
# Packages
*.egg
*.egg-info
dist
build
eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg
lib
lib64
# Installer logs
pip-log.txt
# Unit test / coverage reports
.coverage
.tox
nosetests.xml
.testrepository
# Translations
*.mo
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
.venv
.codeintel
doc/source/api/*
doc/build/*
AUTHORS
TAGS
ChangeLog
# Project specific
etc/billingstack/*.ini
etc/billingstack/*.conf
billingstack/versioninfo
*.sqlite
billingstack-screenrc
status
logs
.ropeproject
*.sublime-project
*.sublime-workspace

4
.gitreview
View File

@@ -1,4 +0,0 @@
[gerrit]
host=review.openstack.org
port=29418
project=stackforge/billingstack.git

42
.pylintrc
View File

@@ -1,42 +0,0 @@
# The format of this file isn't really documented; just use --generate-rcfile
[MASTER]
# Add <file or directory> to the black list. It should be a base name, not a
# path. You may set this option multiple times.
ignore=test
[Messages Control]
# NOTE(justinsb): We might want to have a 2nd strict pylintrc in future
# C0111: Don't require docstrings on every method
# W0511: TODOs in code comments are fine.
# W0142: *args and **kwargs are fine.
# W0622: Redefining id is fine.
disable=C0111,W0511,W0142,W0622
[Basic]
# Variable names can be 1 to 31 characters long, with lowercase and underscores
variable-rgx=[a-z_][a-z0-9_]{0,30}$
# Argument names can be 2 to 31 characters long, with lowercase and underscores
argument-rgx=[a-z_][a-z0-9_]{1,30}$
# Method names should be at least 3 characters long
# and be lowecased with underscores
method-rgx=([a-z_][a-z0-9_]{2,50}|setUp|tearDown)$
# Module names matching billingstack-* are ok (files in bin/)
module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+)|(billingstack-[a-z0-9_-]+))$
# Don't require docstrings on tests.
no-docstring-rgx=((__.*__)|([tT]est.*)|setUp|tearDown)$
[Design]
max-public-methods=100
min-public-methods=0
max-args=6
[Variables]
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid to define new builtins when possible.
# _ is used by our localization
additional-builtins=_

4
.testr.conf
View File

@@ -1,4 +0,0 @@
[DEFAULT]
test_command=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} ${PYTHON:-python} -m subunit.run discover -t ./ ./ $LISTOPT $IDOPTION
test_id_option=--load-list $IDFILE
test_list_option=--list

253
HACKING.rst
View File

@@ -1,253 +0,0 @@
BillingStack Style Commandments
===============================
- Step 1: Read http://www.python.org/dev/peps/pep-0008/
- Step 2: Read http://www.python.org/dev/peps/pep-0008/ again
- Step 3: Read on
General
-------
- Put two newlines between top-level code (funcs, classes, etc)
- Put one newline between methods in classes and anywhere else
- Do not write "except:", use "except Exception:" at the very least
- Include your name with TODOs as in "#TODO(termie)"
- Do not name anything the same name as a built-in or reserved word
- Use the "is not" operator when testing for unequal identities. Example::
if not X is Y: # BAD, intended behavior is ambiguous
pass
if X is not Y: # OKAY, intuitive
pass
- Use the "not in" operator for evaluating membership in a collection. Example::
if not X in Y: # BAD, intended behavior is ambiguous
pass
if X not in Y: # OKAY, intuitive
pass
if not (X in Y or X in Z): # OKAY, still better than all those 'not's
pass
Imports
-------
- Do not make relative imports
- Order your imports by the full module path
- Organize your imports according to the following template
Example::
# vim: tabstop=4 shiftwidth=4 softtabstop=4
{{stdlib imports in human alphabetical order}}
\n
{{third-party lib imports in human alphabetical order}}
\n
{{billingstack imports in human alphabetical order}}
\n
\n
{{begin your code}}
Human Alphabetical Order Examples
---------------------------------
Example::
import httplib
import logging
import random
import StringIO
import time
import unittest
import eventlet
import webob.exc
from billingstack.api import v1
from billingstack.central import rpc_api
from billingstack.rater import rpc_api
Docstrings
----------
Docstrings are required for all functions and methods.
Docstrings should ONLY use triple-double-quotes (``"""``)
Single-line docstrings should NEVER have extraneous whitespace
between enclosing triple-double-quotes.
**INCORRECT** ::
""" There is some whitespace between the enclosing quotes :( """
**CORRECT** ::
"""There is no whitespace between the enclosing quotes :)"""
Docstrings that span more than one line should look like this:
Example::
"""
Start the docstring on the line following the opening triple-double-quote
If you are going to describe parameters and return values, use Sphinx, the
appropriate syntax is as follows.
:param foo: the foo parameter
:param bar: the bar parameter
:returns: return_type -- description of the return value
:returns: description of the return value
:raises: AttributeError, KeyError
"""
**DO NOT** leave an extra newline before the closing triple-double-quote.
Dictionaries/Lists
------------------
If a dictionary (dict) or list object is longer than 80 characters, its items
should be split with newlines. Embedded iterables should have their items
indented. Additionally, the last item in the dictionary should have a trailing
comma. This increases readability and simplifies future diffs.
Example::
my_dictionary = {
"image": {
"name": "Just a Snapshot",
"size": 2749573,
"properties": {
"user_id": 12,
"arch": "x86_64",
},
"things": [
"thing_one",
"thing_two",
],
"status": "ACTIVE",
},
}
Calling Methods
---------------
Calls to methods 80 characters or longer should format each argument with
newlines. This is not a requirement, but a guideline::
unnecessarily_long_function_name('string one',
'string two',
kwarg1=constants.ACTIVE,
kwarg2=['a', 'b', 'c'])
Rather than constructing parameters inline, it is better to break things up::
list_of_strings = [
'what_a_long_string',
'not as long',
]
dict_of_numbers = {
'one': 1,
'two': 2,
'twenty four': 24,
}
object_one.call_a_method('string three',
'string four',
kwarg1=list_of_strings,
kwarg2=dict_of_numbers)
Internationalization (i18n) Strings
-----------------------------------
In order to support multiple languages, we have a mechanism to support
automatic translations of exception and log strings.
Example::
msg = _("An error occurred")
raise HTTPBadRequest(explanation=msg)
If you have a variable to place within the string, first internationalize the
template string then do the replacement.
Example::
msg = _("Missing parameter: %s") % ("flavor",)
LOG.error(msg)
If you have multiple variables to place in the string, use keyword parameters.
This helps our translators reorder parameters when needed.
Example::
msg = _("The server with id %(s_id)s has no key %(m_key)s")
LOG.error(msg % {"s_id": "1234", "m_key": "imageId"})
Creating Unit Tests
-------------------
For every new feature, unit tests should be created that both test and
(implicitly) document the usage of said feature. If submitting a patch for a
bug that had no unit test, a new passing unit test should be added. If a
submitted bug fix does have a unit test, be sure to add a new one that fails
without the patch and passes with the patch.
Commit Messages
---------------
Using a common format for commit messages will help keep our git history
readable. Follow these guidelines:
First, provide a brief summary of 50 characters or less. Summaries
of greater then 72 characters will be rejected by the gate.
The first line of the commit message should provide an accurate
description of the change, not just a reference to a bug or
blueprint. It must be followed by a single blank line.
Following your brief summary, provide a more detailed description of
the patch, manually wrapping the text at 72 characters. This
description should provide enough detail that one does not have to
refer to external resources to determine its high-level functionality.
Once you use 'git review', two lines will be appended to the commit
message: a blank line followed by a 'Change-Id'. This is important
to correlate this commit with a specific review in Gerrit, and it
should not be modified.
For further information on constructing high quality commit messages,
and how to split up commits into a series of changes, consult the
project wiki:
http://wiki.openstack.org/GitCommitMessages
openstack-common
----------------
A number of modules from openstack-common are imported into the project.
These modules are "incubating" in openstack-common and are kept in sync
with the help of openstack-common's update.py script. See:
http://wiki.openstack.org/CommonLibrary#Incubation
The copy of the code should never be directly modified here. Please
always update openstack-common first and then run the script to copy
the changes across.
Logging
-------
Use __name__ as the name of your logger and name your module-level logger
objects 'LOG'::
LOG = logging.getLogger(__name__)

175
LICENSE
View File

@@ -1,175 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

11
MANIFEST.in
View File

@@ -1,11 +0,0 @@
include AUTHORS
include ChangeLog
include billingstack/versioninfo
include *.txt *.ini *.cfg *.rst *.md
include etc/billingstack/*.sample
include etc/billingstack/policy.json
exclude .gitignore
exclude .gitreview
exclude *.sublime-project
global-exclude *.pyc

11
README.rst
View File

@@ -1,8 +1,7 @@
BillingStack This project is no longer maintained.
============
Site: www.billingstack.org The contents of this repository are still available in the Git source code
management system. To see the contents of this repository before it reached
its end of life, please check out the previous commit with
"git checkout HEAD^1".
Docs: http://billingstack.rtfd.org
Github: http://github.com/stackforge/billingstack
Bugs: http://launchpad.net/billingstack

59
billingstack.sublime-project
View File

@@ -1,59 +0,0 @@
{
"folders":
[
{
"file_exclude_patterns":
[
"*.pyc",
"*.pyo",
"*.exe",
"*.dll",
"*.obj",
"*.o",
"*.a",
"*.lib",
"*.so",
"*.dylib",
"*.ncb",
"*.sdf",
"*.suo",
"*.pdb",
"*.idb",
".DS_Store",
"*.class",
"*.psd",
"*.db",
".vagrant",
".noseids"
],
"folder_exclude_patterns":
[
".svn",
".git",
".hg",
"CVS",
"*.egg",
"*.egg-info",
".tox",
"venv",
".venv",
"doc/build",
"doc/source/api"
],
"path": "."
}
],
"settings":
{
"default_line_ending": "unix",
"detect_indentation": false,
"ensure_newline_at_eof_on_save": true,
"rulers":
[
79
],
"tab_size": 4,
"translate_tabs_to_spaces": true,
"trim_trailing_white_space_on_save": true
}
}

15
billingstack/__init__.py
View File

@@ -1,15 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

31
billingstack/api/__init__.py
View File

@@ -1,31 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Copyright © 2013 Woorea Solutions, S.L
#
# Author: Luis Gervaso <luis@woorea.es>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# Copied: Moniker
from oslo.config import cfg
API_SERVICE_OPTS = [
cfg.IntOpt('api_port', default=9091,
help='The port for the billing API server'),
cfg.IntOpt('api_listen', default='0.0.0.0', help='Bind to address'),
cfg.StrOpt('auth_strategy', default='noauth',
help='The strategy to use for auth. Supports noauth or '
'keystone'),
]
cfg.CONF.register_opts(API_SERVICE_OPTS, 'service:api')

91
billingstack/api/app.py
View File

@@ -1,91 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import logging
import os
import pecan
from oslo.config import cfg
from wsgiref import simple_server
from billingstack import service
from billingstack.api import hooks
from billingstack.openstack.common import log
cfg.CONF.import_opt('state_path', 'billingstack.paths')
LOG = log.getLogger(__name__)
def get_config():
conf = {
'app': {
'root': 'billingstack.api.v2.controllers.root.RootController',
'modules': ['designate.api.v2'],
}
}
return pecan.configuration.conf_from_dict(conf)
def setup_app(pecan_config=None, extra_hooks=None):
app_hooks = [
hooks.NoAuthHook()
]
if extra_hooks:
app_hooks.extend(extra_hooks)
pecan_config = pecan_config or get_config()
pecan.configuration.set_config(dict(pecan_config), overwrite=True)
app = pecan.make_app(
pecan_config.app.root,
debug=cfg.CONF.debug,
hooks=app_hooks,
force_canonical=getattr(pecan_config.app, 'force_canonical', True)
)
return app
class VersionSelectorApplication(object):
def __init__(self):
self.v2 = setup_app()
def __call__(self, environ, start_response):
return self.v2(environ, start_response)
def start():
service.prepare_service()
root = VersionSelectorApplication()
host = cfg.CONF['service:api'].api_listen
port = cfg.CONF['service:api'].api_port
srv = simple_server.make_server(host, port, root)
LOG.info('Starting server in PID %s' % os.getpid())
LOG.info("Configuration:")
cfg.CONF.log_opt_values(LOG, logging.INFO)
if host == '0.0.0.0':
LOG.info('serving on 0.0.0.0:%s, view at http://127.0.0.1:%s' %
(port, port))
else:
LOG.info("serving on http://%s:%s" % (host, port))
srv.serve_forever()

158
billingstack/api/base.py
View File

@@ -1,158 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import pecan.rest
from wsme.types import Base, Enum, UserType, text, Unset, wsproperty
from oslo.config import cfg
from billingstack.openstack.common import log
LOG = log.getLogger(__name__)
cfg.CONF.register_opts([
cfg.StrOpt('cors_allowed_origin', default='*', help='Allowed CORS Origin'),
cfg.IntOpt('cors_max_age', default=3600)])
CORS_ALLOW_HEADERS = [
'origin',
'authorization',
'accept',
'content-type',
'x-requested-with'
]
class RestController(pecan.rest.RestController):
def _handle_patch(self, method, remainder):
return self._handle_post(method, remainder)
class Property(UserType):
"""
A Property that just passes the value around...
"""
def tonativetype(self, value):
return value
def fromnativetype(self, value):
return value
property_type = Property()
def _query_to_criterion(query, storage_func=None, **kw):
"""
Iterate over the query checking against the valid signatures (later).
:param query: A list of queries.
:param storage_func: The name of the storage function to very against.
"""
translation = {
'customer': 'customer_id'
}
criterion = {}
for q in query:
key = translation.get(q.field, q.field)
criterion[key] = q.as_dict()
criterion.update(kw)
return criterion
operation_kind = Enum(str, 'lt', 'le', 'eq', 'ne', 'ge', 'gt')
class Query(Base):
"""
Query filter.
"""
_op = None # provide a default
def get_op(self):
return self._op or 'eq'
def set_op(self, value):
self._op = value
field = text
"The name of the field to test"
#op = wsme.wsattr(operation_kind, default='eq')
# this ^ doesn't seem to work.
op = wsproperty(operation_kind, get_op, set_op)
"The comparison operator. Defaults to 'eq'."
value = text
"The value to compare against the stored data"
def __repr__(self):
# for LOG calls
return '<Query %r %s %r>' % (self.field, self.op, self.value)
@classmethod
def sample(cls):
return cls(field='resource_id',
op='eq',
value='bd9431c1-8d69-4ad3-803a-8d4a6b89fd36',
)
def as_dict(self):
return {
'op': self.op,
'field': self.field,
'value': self.value
}
class ModelBase(Base):
def as_dict(self):
"""
Return this model as a dict
"""
data = {}
for attr in self._wsme_attributes:
value = attr.__get__(self, self.__class__)
if value is not Unset:
if isinstance(value, Base) and hasattr(value, "as_dict"):
value = value.as_dict()
data[attr.name] = value
return data
def to_db(self):
"""
Returns this Model object as it's DB form
Example
'currency' vs 'currency_name'
"""
return self.as_dict()
@classmethod
def from_db(cls, values):
"""
Return a class of this object from values in the from_db
"""
return cls(**values)

40
billingstack/api/hooks.py
View File

@@ -1,40 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import hooks
from billingstack.openstack.common.context import RequestContext
class NoAuthHook(hooks.PecanHook):
"""
Simple auth - all requests will be is_admin=True
"""
def merchant_id(self, path):
"""
Get merchant id from url
"""
parts = [p for p in path.split('/') if p]
try:
index = parts.index('merchants') + 1
return parts[index]
except ValueError:
return
except IndexError:
return
def before(self, state):
merchant_id = self.merchant_id(state.request.path_url)
state.request.ctxt = RequestContext(tenant=merchant_id, is_admin=True)

0
billingstack/api/templates/error.html
View File

9
billingstack/api/templates/index.html
View File

@@ -1,9 +0,0 @@
<html>
<head>
<title>BillingStack Diagnostics</title>
</head>
<body>
<h1>Diagnostics</h1>
<p>Here you'll find some basic information about your BillingStack server</p>
</body>
</html>

64
billingstack/api/utils.py
View File

@@ -1,64 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# Copied: http://flask.pocoo.org/snippets/56/
from datetime import timedelta
from flask import make_response, request, current_app
import functools
def crossdomain(origin=None, methods=None, headers=None,
max_age=21600, attach_to_all=True,
automatic_options=True):
if methods is not None:
methods = ', '.join(sorted(x.upper() for x in methods))
if headers is not None and not isinstance(headers, basestring):
headers = ', '.join(x.upper() for x in headers)
if not isinstance(origin, basestring):
origin = ', '.join(origin)
if isinstance(max_age, timedelta):
max_age = max_age.total_seconds()
def get_methods():
if methods is not None:
return methods
options_resp = current_app.make_default_options_response()
return options_resp.headers['allow']
def decorator(f):
def wrapped_function(*args, **kw):
if automatic_options and request.method == 'OPTIONS':
resp = current_app.make_default_options_response()
else:
resp = make_response(f(*args, **kw))
if not attach_to_all and request.method != 'OPTIONS':
return resp
h = resp.headers
h['Access-Control-Allow-Origin'] = origin
h['Access-Control-Allow-Credentials'] = 'true'
h['Access-Control-Allow-Methods'] = get_methods()
h['Access-Control-Max-Age'] = str(max_age)
if headers is not None:
h['Access-Control-Allow-Headers'] = headers
return resp
f.provide_automatic_options = False
f.required_methods = ['OPTIONS']
return functools.update_wrapper(wrapped_function, f)
return decorator

18
billingstack/api/v2/__init__.py
View File

@@ -1,18 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
cfg.CONF.import_opt('state_path', 'billingstack.paths')

15
billingstack/api/v2/controllers/__init__.py
View File

@@ -1,15 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

67
billingstack/api/v2/controllers/currency.py
View File

@@ -1,67 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.central.rpcapi import central_api
class CurrencyController(RestController):
def __init__(self, id_):
self.id_ = id_
@wsme_pecan.wsexpose(models.Currency)
def get_all(self):
row = central_api.get_currency(request.ctxt, self.id_)
return models.Currency.from_db(row)
@wsme.validate(models.Currency)
@wsme_pecan.wsexpose(models.Currency, body=models.Currency)
def patch(self, body):
row = central_api.update_currency(request.ctxt, self.id_, body.to_db())
return models.Currency.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
central_api.delete_currency(request.ctxt, self.id_)
class CurrenciesController(RestController):
@expose()
def _lookup(self, currency_id, *remainder):
return CurrencyController(currency_id), remainder
@wsme.validate(models.Currency)
@wsme_pecan.wsexpose(models.Currency, body=models.Currency,
status_code=202)
def post(self, body):
row = central_api.create_currency(request.ctxt, body.to_db())
return models.Currency.from_db(row)
@wsme_pecan.wsexpose([models.Currency], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(q)
rows = central_api.list_currencies(
request.ctxt, criterion=criterion)
return map(models.Currency.from_db, rows)

74
billingstack/api/v2/controllers/customer.py
View File

@@ -1,74 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.api.v2.controllers.payment import PaymentMethodsController
from billingstack.central.rpcapi import central_api
class CustomerController(RestController):
payment_methods = PaymentMethodsController()
def __init__(self, id_):
self.id_ = id_
request.context['customer_id'] = id_
@wsme_pecan.wsexpose(models.Customer)
def get_all(self):
row = central_api.get_customer(request.ctxt, self.id_)
return models.Customer.from_db(row)
@wsme.validate(models.Customer)
@wsme_pecan.wsexpose(models.Customer, body=models.Customer)
def patch(self, body):
row = central_api.update_customer(request.ctxt, self.id_, body.to_db())
return models.Customer.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
central_api.delete_customer(request.ctxt, self.id_)
class CustomersController(RestController):
@expose()
def _lookup(self, customer_id, *remainder):
return CustomerController(customer_id), remainder
@wsme.validate(models.Customer)
@wsme_pecan.wsexpose(models.Customer, body=models.Customer,
status_code=202)
def post(self, body):
row = central_api.create_customer(
request.ctxt,
request.context['merchant_id'],
body.to_db())
return models.Customer.from_db(row)
@wsme_pecan.wsexpose([models.Customer], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(q)
rows = central_api.list_customers(
request.ctxt, criterion=criterion)
return map(models.Customer.from_db, rows)

73
billingstack/api/v2/controllers/invoice.py
View File

@@ -1,73 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.biller.rpcapi import biller_api
class InvoiceController(RestController):
def __init__(self, id_):
self.id_ = id_
request.context['invoice_id'] = id_
@wsme_pecan.wsexpose(models.Invoice)
def get_all(self):
row = biller_api.get_invoice(request.ctxt, self.id_)
return models.Invoice.from_db(row)
@wsme.validate(models.Invoice)
@wsme_pecan.wsexpose(models.Invoice, body=models.Invoice)
def patch(self, body):
row = biller_api.update_invoice(request.ctxt, self.id_, body.to_db())
return models.Invoice.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
biller_api.delete_invoice(request.ctxt, self.id_)
class InvoicesController(RestController):
@expose()
def _lookup(self, invoice_id, *remainder):
return InvoiceController(invoice_id), remainder
@wsme.validate(models.Invoice)
@wsme_pecan.wsexpose(models.Invoice, body=models.Invoice, status_code=202)
def post(self, body):
row = biller_api.create_invoice(
request.ctxt,
request.context['merchant_id'],
body.to_db())
return models.Invoice.from_db(row)
@wsme_pecan.wsexpose([models.Invoice], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(
q,
merchant_id=request.context['merchant_id'])
rows = biller_api.list_invoices(
request.ctxt, criterion=criterion)
return map(models.Invoice.from_db, rows)

68
billingstack/api/v2/controllers/invoice_state.py
View File

@@ -1,68 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.biller.rpcapi import biller_api
class InvoiceStateController(RestController):
def __init__(self, id_):
self.id_ = id_
@wsme_pecan.wsexpose(models.InvoiceState)
def get_all(self):
row = biller_api.get_invoice_state(request.ctxt, self.id_)
return models.InvoiceState.from_db(row)
@wsme.validate(models.InvoiceState)
@wsme_pecan.wsexpose(models.InvoiceState, body=models.InvoiceState)
def patch(self, body):
row = biller_api.update_invoice_state(
request.ctxt, self.id_, body.to_db())
return models.InvoiceState.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
biller_api.delete_invoice_state(request.ctxt, self.id_)
class InvoiceStatesController(RestController):
@expose()
def _lookup(self, invoice_state_id, *remainder):
return InvoiceStateController(invoice_state_id), remainder
@wsme.validate(models.InvoiceState)
@wsme_pecan.wsexpose(models.InvoiceState, body=models.InvoiceState,
status_code=202)
def post(self, body):
row = biller_api.create_invoice_state(request.ctxt, body.to_db())
return models.InvoiceState.from_db(row)
@wsme_pecan.wsexpose([models.InvoiceState], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(q)
rows = biller_api.list_invoice_states(
request.ctxt, criterion=criterion)
return map(models.InvoiceState.from_db, rows)

67
billingstack/api/v2/controllers/language.py
View File

@@ -1,67 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.central.rpcapi import central_api
class LanguageController(RestController):
def __init__(self, id_):
self.id_ = id_
@wsme_pecan.wsexpose(models.Language)
def get_all(self):
row = central_api.get_language(request.ctxt, self.id_)
return models.Language.from_db(row)
@wsme.validate(models.InvoiceState)
@wsme_pecan.wsexpose(models.Language, body=models.Language)
def patch(self, body):
row = central_api.update_language(request.ctxt, self.id_, body.to_db())
return models.Language.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
central_api.delete_language(request.ctxt, self.id_)
class LanguagesController(RestController):
@expose()
def _lookup(self, language_id, *remainder):
return LanguageController(language_id), remainder
@wsme.validate(models.InvoiceState)
@wsme_pecan.wsexpose(models.Language, body=models.Language,
status_code=202)
def post(self, body):
row = central_api.create_language(request.ctxt, body.to_db())
return models.Language.from_db(row)
@wsme_pecan.wsexpose([models.Language], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(q)
rows = central_api.list_languages(
request.ctxt, criterion=criterion)
return map(models.Language.from_db, rows)

85
billingstack/api/v2/controllers/merchant.py
View File

@@ -1,85 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.central.rpcapi import central_api
from billingstack.api.v2.controllers.customer import CustomersController
from billingstack.api.v2.controllers.payment import PGConfigsController
from billingstack.api.v2.controllers.plan import PlansController
from billingstack.api.v2.controllers.product import ProductsController
from billingstack.api.v2.controllers.subscription import \
SubscriptionsController
from billingstack.api.v2.controllers.invoice import InvoicesController
from billingstack.api.v2.controllers.usage import UsagesController
class MerchantController(RestController):
customers = CustomersController()
payment_gateway_configurations = PGConfigsController()
plans = PlansController()
products = ProductsController()
subscriptions = SubscriptionsController()
invoices = InvoicesController()
usage = UsagesController()
def __init__(self, id_):
self.id_ = id_
request.context['merchant_id'] = id_
@wsme_pecan.wsexpose(models.Merchant)
def get_all(self):
row = central_api.get_merchant(request.ctxt, self.id_)
return models.Merchant.from_db(row)
@wsme.validate(models.InvoiceState)
@wsme_pecan.wsexpose(models.Merchant, body=models.Merchant)
def patch(self, body):
row = central_api.update_merchant(request.ctxt, self.id_, body.to_db())
return models.Merchant.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
central_api.delete_merchant(request.ctxt, self.id_)
class MerchantsController(RestController):
@expose()
def _lookup(self, merchant_id, *remainder):
return MerchantController(merchant_id), remainder
@wsme.validate(models.Merchant)
@wsme_pecan.wsexpose(models.Merchant, body=models.Merchant,
status_code=202)
def post(self, body):
row = central_api.create_merchant(request.ctxt, body.to_db())
return models.Merchant.from_db(row)
@wsme_pecan.wsexpose([models.Merchant], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(q)
rows = central_api.list_merchants(
request.ctxt, criterion=criterion)
return map(models.Merchant.from_db, rows)

141
billingstack/api/v2/controllers/payment.py
View File

@@ -1,141 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.collector.rpcapi import collector_api
class PGProviders(RestController):
@wsme_pecan.wsexpose([models.PGProvider], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(q)
rows = collector_api.list_pg_providers(
request.ctxt, criterion=criterion)
return map(models.PGProvider.from_db, rows)
class PGConfigController(RestController):
def __init__(self, id_):
self.id_ = id_
@wsme_pecan.wsexpose(models.PGConfig)
def get_all(self):
row = collector_api.get_pg_config(request.ctxt, self.id_)
return models.PGConfig.from_db(row)
@wsme.validate(models.PGConfig)
@wsme_pecan.wsexpose(models.PGConfig, body=models.PGConfig)
def patch(self, body):
row = collector_api.update_pg_config(
request.ctxt,
self.id_,
body.to_db())
return models.PGConfig.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
collector_api.delete_pg_config(request.ctxt, self.id_)
class PGConfigsController(RestController):
@expose()
def _lookup(self, method_id, *remainder):
return PGConfigController(method_id), remainder
@wsme.validate(models.PGConfig)
@wsme_pecan.wsexpose(models.PGConfig, body=models.PGConfig,
status_code=202)
def post(self, body):
values = body.to_db()
values['merchant_id'] = request.context['merchant_id']
row = collector_api.create_pg_config(
request.ctxt,
values)
return models.PGConfig.from_db(row)
@wsme_pecan.wsexpose([models.PGConfig], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(
q, merchant_id=request.context['merchant_id'])
rows = collector_api.list_pg_configs(
request.ctxt, criterion=criterion)
return map(models.PGConfig.from_db, rows)
class PaymentMethodController(RestController):
def __init__(self, id_):
self.id_ = id_
request.context['payment_method_id'] = id_
@wsme_pecan.wsexpose(models.PaymentMethod)
def get_all(self):
row = collector_api.get_payment_method(request.ctxt, self.id_)
return models.PaymentMethod.from_db(row)
@wsme.validate(models.PaymentMethod)
@wsme_pecan.wsexpose(models.PaymentMethod, body=models.PaymentMethod)
def patch(self, body):
row = collector_api.update_payment_method(
request.ctxt,
self.id_,
body.to_db())
return models.PaymentMethod.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
collector_api.delete_payment_method(request.ctxt, self.id_)
class PaymentMethodsController(RestController):
@expose()
def _lookup(self, method_id, *remainder):
return PaymentMethodController(method_id), remainder
@wsme.validate(models.PaymentMethod)
@wsme_pecan.wsexpose(models.PaymentMethod, body=models.PaymentMethod,
status_code=202)
def post(self, body):
values = body.to_db()
values['customer_id'] = request.context['customer_id']
row = collector_api.create_payment_method(request.ctxt, values)
return models.PaymentMethod.from_db(row)
@wsme_pecan.wsexpose([models.PaymentMethod], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(
q, merchant_id=request.context['merchant_id'],
customer_id=request.context['customer_id'])
rows = collector_api.list_payment_methods(
request.ctxt, criterion=criterion)
return map(models.PaymentMethod.from_db, rows)

116
billingstack/api/v2/controllers/plan.py
View File

@@ -1,116 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.central.rpcapi import central_api
class ItemController(RestController):
def __init__(self, id_):
self.id_ = id_
@wsme.validate(models.PlanItem)
@wsme_pecan.wsexpose(models.PlanItem, body=models.PlanItem)
def put(self, body):
values = {
'plan_id': request.context['plan_id'],
'product_id': self.id_
}
row = central_api.create_plan_item(request.ctxt, values)
return models.PlanItem.from_db(row)
@wsme.validate(models.PlanItem)
@wsme_pecan.wsexpose(models.PlanItem, body=models.PlanItem)
def patch(self, body):
row = central_api.update_plan_item(
request.ctxt,
request.context['plan_id'],
self.id_,
body.to_db())
return models.PlanItem.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self, id_):
central_api.delete_plan_item(
request.ctxt,
request.context['plan_id'],
id_)
class ItemsController(RestController):
@expose()
def _lookup(self, id_, *remainder):
return ItemController(id_), remainder
class PlanController(RestController):
items = ItemsController()
def __init__(self, id_):
self.id_ = id_
request.context['plan_id'] = id_
@wsme_pecan.wsexpose(models.Plan)
def get_all(self):
row = central_api.get_plan(request.ctxt, self.id_)
return models.Plan.from_db(row)
@wsme.validate(models.Plan)
@wsme_pecan.wsexpose(models.Plan, body=models.Plan)
def patch(self, body):
row = central_api.update_plan(request.ctxt, self.id_, body.to_db())
return models.Plan.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
central_api.delete_plan(request.ctxt, self.id_)
class PlansController(RestController):
@expose()
def _lookup(self, plan_id, *remainder):
return PlanController(plan_id), remainder
@wsme.validate(models.Plan)
@wsme_pecan.wsexpose(models.Plan, body=models.Plan, status_code=202)
def post(self, body):
row = central_api.create_plan(
request.ctxt,
request.context['merchant_id'],
body.to_db())
return models.Plan.from_db(row)
@wsme_pecan.wsexpose([models.Plan], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(
q,
merchant_id=request.context['merchant_id'])
rows = central_api.list_plans(
request.ctxt, criterion=criterion)
return map(models.Plan.from_db, rows)

74
billingstack/api/v2/controllers/product.py
View File

@@ -1,74 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.central.rpcapi import central_api
class ProductController(RestController):
def __init__(self, id_):
self.id_ = id_
request.context['product_id'] = id_
@wsme_pecan.wsexpose(models.Product)
def get_all(self):
row = central_api.get_product(request.ctxt, self.id_)
return models.Product.from_db(row)
@wsme.validate(models.Product)
@wsme_pecan.wsexpose(models.Product, body=models.Product)
def patch(self, body):
row = central_api.update_product(request.ctxt, self.id_, body.to_db())
return models.Product.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
central_api.delete_product(request.ctxt, self.id_)
class ProductsController(RestController):
@expose()
def _lookup(self, product_id, *remainder):
return ProductController(product_id), remainder
@wsme.validate(models.Product)
@wsme_pecan.wsexpose(models.Product, body=models.Product,
status_code=202)
def post(self, body):
row = central_api.create_product(
request.ctxt,
request.context['merchant_id'],
body.to_db())
return models.Product.from_db(row)
@wsme_pecan.wsexpose([models.Product], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(
q,
merchant_id=request.context['merchant_id'])
rows = central_api.list_products(
request.ctxt, criterion=criterion)
return map(models.Product.from_db, rows)

42
billingstack/api/v2/controllers/root.py
View File

@@ -1,42 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from billingstack.openstack.common import log
from billingstack.api.v2.controllers.currency import CurrenciesController
from billingstack.api.v2.controllers.language import LanguagesController
from billingstack.api.v2.controllers.merchant import MerchantsController
from billingstack.api.v2.controllers.invoice_state import \
InvoiceStatesController
from billingstack.api.v2.controllers.payment import PGProviders
LOG = log.getLogger(__name__)
class V2Controller(object):
# Central
currencies = CurrenciesController()
languages = LanguagesController()
merchants = MerchantsController()
# Biller
invoice_states = InvoiceStatesController()
# Collector
payment_gateway_providers = PGProviders()
class RootController(object):
v2 = V2Controller()

75
billingstack/api/v2/controllers/subscription.py
View File

@@ -1,75 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.central.rpcapi import central_api
class SubscriptionController(RestController):
def __init__(self, id_):
self.id_ = id_
request.context['subscription_id'] = id_
@wsme_pecan.wsexpose(models.Subscription)
def get_all(self):
row = central_api.get_subscription(request.ctxt, self.id_)
return models.Subscription.from_db(row)
@wsme.validate(models.Subscription)
@wsme_pecan.wsexpose(models.Subscription, body=models.Subscription)
def patch(self, body):
row = central_api.update_subscription(request.ctxt, self.id_,
body.to_db())
return models.Subscription.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
central_api.delete_subscription(request.ctxt, self.id_)
class SubscriptionsController(RestController):
@expose()
def _lookup(self, subscription_id, *remainder):
return SubscriptionController(subscription_id), remainder
@wsme.validate(models.Subscription)
@wsme_pecan.wsexpose(models.Subscription, body=models.Subscription,
status_code=202)
def post(self, body):
row = central_api.create_subscription(
request.ctxt,
request.context['merchant_id'],
body.to_db())
return models.Subscription.from_db(row)
@wsme_pecan.wsexpose([models.Subscription], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(
q,
merchant_id=request.context['merchant_id'])
rows = central_api.list_subscriptions(
request.ctxt, criterion=criterion)
return map(models.Subscription.from_db, rows)

73
billingstack/api/v2/controllers/usage.py
View File

@@ -1,73 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from pecan import expose, request
import wsme
import wsmeext.pecan as wsme_pecan
from billingstack.api.base import Query, _query_to_criterion, RestController
from billingstack.api.v2 import models
from billingstack.rater.rpcapi import rater_api
class UsageController(RestController):
def __init__(self, id_):
self.id_ = id_
request.context['usage_id'] = id_
@wsme_pecan.wsexpose(models.Usage)
def get_all(self):
row = rater_api.get_usage(request.ctxt, self.id_)
return models.Usage.from_db(row)
@wsme.validate(models.Usage)
@wsme_pecan.wsexpose(models.Usage, body=models.Usage)
def patch(self, body):
row = rater_api.update_usage(request.ctxt, self.id_, body.to_db())
return models.Usage.from_db(row)
@wsme_pecan.wsexpose(None, status_code=204)
def delete(self):
rater_api.delete_usage(request.ctxt, self.id_)
class UsagesController(RestController):
@expose()
def _lookup(self, usage_id, *remainder):
return UsageController(usage_id), remainder
@wsme.validate(models.Usage)
@wsme_pecan.wsexpose(models.Usage, body=models.Usage, status_code=202)
def post(self, body):
row = rater_api.create_usage(
request.ctxt,
request.context['merchant_id'],
body.to_db())
return models.Usage.from_db(row)
@wsme_pecan.wsexpose([models.Usage], [Query])
def get_all(self, q=[]):
criterion = _query_to_criterion(
q,
merchant_id=request.context['merchant_id'])
rows = rater_api.list_usages(
request.ctxt, criterion=criterion)
return map(models.Usage.from_db, rows)

221
billingstack/api/v2/models.py
View File

@@ -1,221 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from wsme.types import text, DictType
from datetime import datetime
from billingstack.api.base import ModelBase, property_type
from billingstack.openstack.common import log
LOG = log.getLogger(__name__)
class Base(ModelBase):
id = text
class DescribedBase(Base):
name = text
title = text
description = text
def change_suffixes(data, keys, shorten=True, suffix='_name'):
"""
Loop thro the keys foreach key setting for example
'currency_name' > 'currency'
"""
for key in keys:
if shorten:
new, old = key, key + suffix
else:
new, old = key + suffix, key
if old in data:
if new in data:
raise RuntimeError("Can't override old key with new key")
data[new] = data.pop(old)
class Currency(DescribedBase):
pass
class Language(DescribedBase):
pass
class InvoiceState(DescribedBase):
pass
class PGProvider(DescribedBase):
def __init__(self, **kw):
#kw['methods'] = [PGMethod.from_db(m) for m in kw.get('methods', [])]
super(PGProvider, self).__init__(**kw)
methods = [DictType(key_type=text, value_type=property_type)]
properties = DictType(key_type=text, value_type=property_type)
class ContactInfo(Base):
id = text
first_name = text
last_name = text
company = text
address1 = text
address2 = text
address3 = text
locality = text
region = text
country_name = text
postal_code = text
phone = text
email = text
website = text
class PlanItem(ModelBase):
name = text
title = text
description = text
plan_id = text
product_id = text
pricing = [DictType(key_type=text, value_type=property_type)]
class Plan(DescribedBase):
def __init__(self, **kw):
if 'items' in kw:
kw['items'] = map(PlanItem.from_db, kw.pop('items'))
super(Plan, self).__init__(**kw)
items = [PlanItem]
properties = DictType(key_type=text, value_type=property_type)
class Product(DescribedBase):
properties = DictType(key_type=text, value_type=property_type)
pricing = [DictType(key_type=text, value_type=property_type)]
class InvoiceLine(Base):
description = text
price = float
quantity = float
sub_total = float
invoice_id = text
class Invoice(Base):
identifier = text
sub_total = float
tax_percentage = float
tax_total = float
total = float
class Subscription(Base):
billing_day = int
resource_id = text
resource_type = text
plan_id = text
customer_id = text
payment_method_id = text
class Usage(Base):
measure = text
start_timestamp = datetime
end_timestamp = datetime
price = float
total = float
value = float
merchant_id = text
product_id = text
subscription_id = text
class PGConfig(Base):
name = text
title = text
merchant_id = text
provider_id = text
state = text
properties = DictType(key_type=text, value_type=property_type)
class PaymentMethod(Base):
name = text
identifier = text
expires = text
merchant_id = text
customer_id = text
provider_config_id = text
state = text
properties = DictType(key_type=text, value_type=property_type)
class Account(Base):
_keys = ['currency', 'language']
currency = text
language = text
name = text
class Merchant(Account):
default_gateway = text
def to_db(self):
values = self.as_dict()
change_suffixes(values, self._keys, shorten=False)
return values
@classmethod
def from_db(cls, values):
change_suffixes(values, cls._keys)
return cls(**values)
class Customer(Account):
merchant_id = text
contact_info = [ContactInfo]
def __init__(self, **kw):
infos = kw.get('contact_info', {})
kw['contact_info'] = [ContactInfo.from_db(i) for i in infos]
super(Customer, self).__init__(**kw)
def to_db(self):
values = self.as_dict()
change_suffixes(values, self._keys, shorten=False)
return values
@classmethod
def from_db(cls, values):
change_suffixes(values, cls._keys)
return cls(**values)

27
billingstack/biller/__init__.py
View File

@@ -1,27 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
cfg.CONF.register_group(cfg.OptGroup(
name='service:biller', title="Configuration for Biller Service"
))
cfg.CONF.register_opts([
cfg.IntOpt('workers', default=None,
help='Number of worker processes to spawn'),
cfg.StrOpt('storage-driver', default='sqlalchemy',
help='The storage driver to use'),
], group='service:biller')

94
billingstack/biller/rpcapi.py
View File

@@ -1,94 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
from billingstack.openstack.common.rpc import proxy
rpcapi_opts = [
cfg.StrOpt('biller_topic', default='biller',
help='the topic biller nodes listen on')
]
cfg.CONF.register_opts(rpcapi_opts)
class BillerAPI(proxy.RpcProxy):
BASE_RPC_VERSION = '1.0'
def __init__(self):
super(BillerAPI, self).__init__(
topic=cfg.CONF.biller_topic,
default_version=self.BASE_RPC_VERSION)
# Invoice States
def create_invoice_state(self, ctxt, values):
return self.call(ctxt, self.make_msg('create_invoice_state',
values=values))
def list_invoice_states(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_invoice_states',
criterion=criterion))
def get_invoice_state(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_invoice_state', id_=id_))
def update_invoice_state(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_invoice_state',
id_=id_, values=values))
def delete_invoice_state(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_invoice_state', id_=id_))
# Invoices
def create_invoice(self, ctxt, merchant_id, values):
return self.call(ctxt, self.make_msg('create_invoice',
merchant_id=merchant_id, values=values))
def list_invoices(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_invoices',
criterion=criterion))
def get_invoice(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_invoice', id_=id_))
def update_invoice(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_invoice', id_=id_,
values=values))
def delete_invoice(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_invoice', id_=id_))
# Invoice lines
def create_invoice_line(self, ctxt, invoice_id, values):
return self.call(ctxt, self.make_msg('create_invoice_line',
invoice_id=invoice_id, values=values))
def list_invoice_lines(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_invoice_lines',
criterion=criterion))
def get_invoice_line(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_invoice_line', id_=id_))
def update_invoice_line(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_invoice_line', id_=id_,
values=values))
def delete_invoice_line(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_invoice_line', id_=id_))
biller_api = BillerAPI()

105
billingstack/biller/service.py
View File

@@ -1,105 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import sys
from oslo.config import cfg
from billingstack.openstack.common import log as logging
from billingstack.openstack.common import service as os_service
from billingstack.openstack.common.rpc import service as rpc_service
from billingstack.storage.utils import get_connection
from billingstack import service as bs_service
cfg.CONF.import_opt('biller_topic', 'billingstack.biller.rpcapi')
cfg.CONF.import_opt('host', 'billingstack.netconf')
cfg.CONF.import_opt('state_path', 'billingstack.paths')
LOG = logging.getLogger(__name__)
class Service(rpc_service.Service):
"""
Biller service
"""
def __init__(self, *args, **kwargs):
kwargs.update(
host=cfg.CONF.host,
topic=cfg.CONF.biller_topic,
)
super(Service, self).__init__(*args, **kwargs)
def start(self):
self.storage_conn = get_connection('biller')
super(Service, self).start()
def wait(self):
super(Service, self).wait()
self.conn.consumer_thread.wait()
def create_invoice_state(self, ctxt, values):
return self.storage_conn.create_invoice_state(ctxt, values)
def list_invoice_states(self, ctxt, **kw):
return self.storage_conn.list_invoice_states(ctxt, **kw)
def get_invoice_state(self, ctxt, id_):
return self.storage_conn.get_invoice_state(ctxt, id_)
def update_invoice_state(self, ctxt, id_, values):
return self.storage_conn.update_invoice_state(ctxt, id_, values)
def delete_invoice_state(self, ctxt, id_):
return self.storage_conn.delete_invoice_state(ctxt, id_)
def create_invoice(self, ctxt, merchant_id, values):
return self.storage_conn.create_invoice_state(
ctxt, merchant_id, values)
def list_invoices(self, ctxt, **kw):
return self.storage_conn.list_invoices(ctxt, **kw)
def get_invoice(self, ctxt, id_):
return self.storage_conn.get_invoice(ctxt, id_)
def update_invoice(self, ctxt, id_, values):
return self.storage_conn.update_invoice(ctxt, id_, values)
def delete_invoice(self, ctxt, id_):
return self.storage_conn.delete_invoice(ctxt, id_)
def create_invoice_line(self, ctxt, invoice_id, values):
return self.storage_conn.create_invoice_line_state(
ctxt, invoice_id, values)
def list_invoice_lines(self, ctxt, **kw):
return self.storage_conn.list_invoice_lines(ctxt, **kw)
def get_invoice_line(self, ctxt, id_):
return self.storage_conn.get_invoice_line(ctxt, id_)
def update_invoice_line(self, ctxt, id_, values):
return self.storage_conn.update_invoice_line(ctxt, id_, values)
def delete_invoice_line(self, ctxt, id_):
return self.storage_conn.delete_invoice_line(ctxt, id_)
def launch():
bs_service.prepare_service(sys.argv)
launcher = os_service.launch(Service(),
cfg.CONF['service:biller'].workers)
launcher.wait()

26
billingstack/biller/storage/__init__.py
View File

@@ -1,26 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from billingstack.storage import base
class StorageEngine(base.StorageEngine):
"""Base class for the biller storage"""
__plugin_ns__ = 'billingstack.biller.storage'
class Connection(base.Connection):
"""Define the base API for biller storage"""

246
billingstack/biller/storage/impl_sqlalchemy.py
View File

@@ -1,246 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
A Usage plugin using sqlalchemy...
"""
from oslo.config import cfg
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, ForeignKey
from sqlalchemy import DateTime, Float, Unicode
from sqlalchemy.orm import relationship
from billingstack.openstack.common import log as logging
from billingstack.sqlalchemy.types import UUID
from billingstack.sqlalchemy import api, model_base, session
from billingstack.biller.storage import Connection, StorageEngine
from billingstack.central import rpcapi as central_api
# DB SCHEMA
BASE = declarative_base(cls=model_base.ModelBase)
LOG = logging.getLogger(__name__)
cfg.CONF.register_group(cfg.OptGroup(
name='biller:sqlalchemy', title='Config for biller sqlalchemy plugin'))
cfg.CONF.register_opts(session.SQLOPTS, group='biller:sqlalchemy')
class InvoiceState(BASE):
"""
A State representing the currented state a Invoice is in
Example:
Completed, Failed
"""
name = Column(Unicode(60), nullable=False, primary_key=True)
title = Column(Unicode(100), nullable=False)
description = Column(Unicode(255))
class Invoice(BASE, model_base.BaseMixin):
"""
An invoice
"""
identifier = Column(Unicode(255), nullable=False)
due = Column(DateTime, )
sub_total = Column(Float)
tax_percentage = Column(Float)
tax_total = Column(Float)
total = Column(Float)
customer_id = Column(UUID, nullable=False)
line_items = relationship('InvoiceLine', backref='invoice_lines')
state = relationship('InvoiceState', backref='invoices')
state_id = Column(Unicode(60), ForeignKey('invoice_state.name'),
nullable=False)
# Keep track of the currency and merchant
currency_name = Column(Unicode(10), nullable=False)
merchant_id = Column(UUID, nullable=False)
class InvoiceLine(BASE, model_base.BaseMixin):
"""
A Line item in which makes up the Invoice
"""
description = Column(Unicode(255))
price = Column(Float)
quantity = Column(Float)
sub_total = Column(Float)
invoice_id = Column(UUID, ForeignKey('invoice.id', ondelete='CASCADE',
onupdate='CASCADE'), nullable=False)
class SQLAlchemyEngine(StorageEngine):
__plugin_name__ = 'sqlalchemy'
def get_connection(self):
return Connection()
class Connection(Connection, api.HelpersMixin):
def __init__(self):
self.setup('biller:sqlalchemy')
def base(self):
return BASE
# Invoice States
def create_invoice_state(self, ctxt, values):
"""
Add a supported invoice_state to the database
"""
row = InvoiceState(**values)
self._save(row)
return dict(row)
def list_invoice_states(self, ctxt, **kw):
rows = self._list(InvoiceState, **kw)
return map(dict, rows)
def get_invoice_state(self, ctxt, id_):
row = self._get_id_or_name(InvoiceState, id_)
return dict(row)
def update_invoice_state(self, ctxt, id_, values):
row = self._update(InvoiceState, id_, values, by_name=True)
return dict(row)
def delete_invoice_state(self, ctxt, id_):
self._delete(InvoiceState, id_, by_name=True)
# Invoices
def _invoice(self, row):
invoice = dict(row)
return invoice
def create_invoice(self, ctxt, merchant_id, values):
"""
Add a new Invoice
:param merchant_id: The Merchant
:param values: Values describing the new Invoice
"""
merchant = central_api.get_merchant(merchant_id)
invoice = Invoice(**values)
invoice.merchant = merchant
self._save(invoice)
return self._invoice(invoice)
def list_invoices(self, ctxt, **kw):
"""
List Invoices
"""
rows = self._list(Invoice, **kw)
return map(self._invoice, rows)
def get_invoice(self, ctxt, id_):
"""
Get a Invoice
:param id_: The Invoice ID
"""
row = self._get(Invoice, id_)
return self.invoice(row)
def update_invoice(self, ctxt, id_, values):
"""
Update a Invoice
:param id_: The Invoice ID
:param values: Values to update with
"""
row = self._get(Invoice, id_)
row.update(values)
self._save(row)
return self._invoice(row)
def delete_invoice(self, ctxt, id_):
"""
Delete a Invoice
:param id_: Invoice ID
"""
self._delete(Invoice, id_)
# Invoices Items
def _invoice_line(self, row):
line = dict(row)
return line
def create_invoice_items(self, ctxt, invoice_id, values):
"""
Add a new Invoice
:param invoice_id: The Invoice
:param values: Values describing the new Invoice Line
"""
invoice = self._get(Invoice, invoice_id)
line = InvoiceLine(**values)
line.invoice = invoice
self._save(line)
return self._invoice_line(line)
def list_invoice_lines(self, ctxt, **kw):
"""
List Invoice Lines
"""
rows = self._list(InvoiceLine, **kw)
return map(self._invoice_line, rows)
def get_invoice_line(self, ctxt, id_):
"""
Get a Invoice Line
:param id_: The Invoice Line ID
"""
row = self._get(InvoiceLine, id_)
return self._invoice_line(row)
def update_invoice_line(self, ctxt, id_, values):
"""
Update a Invoice Line
:param id_: The Invoice ID
:param values: Values to update with
"""
row = self._get(InvoiceLine, id_)
row.update(values)
self._save(row)
return self._invoice_line(row)
def delete_invoice_line(self, ctxt, id_):
"""
Delete a Invoice Line
:param id_: Invoice Line ID
"""
self._delete(InvoiceLine, id_)

28
billingstack/central/__init__.py
View File

@@ -1,28 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
cfg.CONF.register_group(cfg.OptGroup(
name='service:central', title="Configuration for Central Service"
))
cfg.CONF.register_opts([
cfg.IntOpt('workers', default=None,
help='Number of worker processes to spawn'),
cfg.StrOpt('storage-driver', default='sqlalchemy',
help='The storage driver to use'),
], group='service:central')

0
billingstack/central/flows/__init__.py
View File

43
billingstack/central/flows/merchant.py
View File

@@ -1,43 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2013 Hewlett-Packard Development Company, L.P.
#
# Author: Endre Karlson <endre.karlson@hp.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in co68mpliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from taskflow.patterns import linear_flow
from billingstack import tasks
from billingstack.openstack.common import log
ACTION = 'merchant:create'
LOG = log.getLogger(__name__)
class EntryCreateTask(tasks.RootTask):
def __init__(self, storage, **kw):
super(EntryCreateTask, self).__init__(**kw)
self.storage = storage
def execute(self, ctxt, values):
return self.storage.create_merchant(ctxt, values)
def create_flow(storage):
flow = linear_flow.Flow(ACTION)
entry_task = EntryCreateTask(storage, provides='merchant', prefix=ACTION)
flow.add(entry_task)
return flow

211
billingstack/central/rpcapi.py
View File

@@ -1,211 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
from billingstack.openstack.common.rpc import proxy
rpcapi_opts = [
cfg.StrOpt('central_topic', default='central',
help='the topic central nodes listen on')
]
cfg.CONF.register_opts(rpcapi_opts)
class CentralAPI(proxy.RpcProxy):
BASE_RPC_VERSION = '1.0'
def __init__(self):
super(CentralAPI, self).__init__(
topic=cfg.CONF.central_topic,
default_version=self.BASE_RPC_VERSION)
# Currency
def create_currency(self, ctxt, values):
return self.call(ctxt, self.make_msg('create_currency', values=values))
def list_currencies(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_currencies',
criterion=criterion))
def get_currency(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_currency',
id_=id_))
def update_currency(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_currency',
id_=id_, values=values))
def delete_currency(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_currency',
id_=id_))
# Language
def create_language(self, ctxt, values):
return self.call(ctxt, self.make_msg('create_language', values=values))
def list_languages(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_languages',
criterion=criterion))
def get_language(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_language', id_=id_))
def update_language(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_language',
id_=id_, values=values))
def delete_language(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_language', id_=id_))
# Contact Info
def create_contact_info(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('create_contact_info', id_=id_,
values=values))
def get_contact_info(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_contact_info', id_))
def update_contact_info(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_contact_info', id_=id_,
values=values))
def delete_contact_info(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_contact_info', id_=id_))
# Merchant
def create_merchant(self, ctxt, values):
return self.call(ctxt, self.make_msg('create_merchant', values=values))
def list_merchants(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_merchants',
criterion=criterion))
def get_merchant(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_merchant', id_=id_))
def update_merchant(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_merchant',
id_=id_, values=values))
def delete_merchant(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_merchant',
id_=id_))
# Customer
def create_customer(self, ctxt, merchant_id, values):
return self.call(ctxt, self.make_msg('create_customer',
merchant_id=merchant_id, values=values))
def list_customers(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_customers',
criterion=criterion))
def get_customer(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_customer', id_=id_))
def update_customer(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_customer',
id_=id_, values=values))
def delete_customer(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_customer', id_=id_))
# Plans
def create_plan(self, ctxt, merchant_id, values):
return self.call(ctxt, self.make_msg('create_plan',
merchant_id=merchant_id, values=values))
def list_plans(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_plans',
criterion=criterion))
def get_plan(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_plan', id_=id_))
def update_plan(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_plan', id_=id_,
values=values))
def delete_plan(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_plan', id_=id_))
def get_plan_by_subscription(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_plan_by_subscription',
id_=id_))
# PlanItems
def create_plan_item(self, ctxt, values):
return self.call(ctxt, self.make_msg('create_plan_item',
values=values))
def list_plan_items(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_plan_items',
criterion=criterion))
def get_plan_item(self, ctxt, plan_id, product_id):
return self.call(ctxt, self.make_msg('get_plan_item',
plan_id=plan_id, product_id=product_id))
def update_plan_item(self, ctxt, plan_id, product_id, values):
return self.call(ctxt, self.make_msg('update_plan_item',
plan_id=plan_id, product_id=product_id,
values=values))
def delete_plan_item(self, ctxt, plan_id, product_id):
return self.call(ctxt, self.make_msg('delete_plan_item',
plan_id=plan_id, product_id=product_id))
# Products
def create_product(self, ctxt, merchant_id, values):
return self.call(ctxt, self.make_msg('create_product',
merchant_id=merchant_id, values=values))
def list_products(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_products',
criterion=criterion))
def get_product(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_product', id_=id_))
def update_product(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_product', id_=id_,
values=values))
def delete_product(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_product', id_=id_))
# Subscriptions
def create_subscription(self, ctxt, values):
return self.call(ctxt, self.make_msg('create_subscription',
values=values))
def list_subscriptions(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_subscriptions',
criterion=criterion))
def get_subscription(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_subscription', id_=id_))
def update_subscription(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_subscription', id_=id_,
values=values))
def delete_subscription(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_subscription', id_=id_))
central_api = CentralAPI()

215
billingstack/central/service.py
View File

@@ -1,215 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import sys
from oslo.config import cfg
from taskflow.engines import run as run_flow
from billingstack.openstack.common import log as logging
from billingstack.openstack.common.rpc import service as rpc_service
from billingstack.openstack.common import service as os_service
from billingstack.central.flows import merchant
from billingstack.storage.utils import get_connection
from billingstack import service as bs_service
cfg.CONF.import_opt('central_topic', 'billingstack.central.rpcapi')
cfg.CONF.import_opt('host', 'billingstack.netconf')
cfg.CONF.import_opt('state_path', 'billingstack.paths')
LOG = logging.getLogger(__name__)
class Service(rpc_service.Service):
def __init__(self, *args, **kwargs):
kwargs.update(
host=cfg.CONF.host,
topic=cfg.CONF.central_topic,
)
super(Service, self).__init__(*args, **kwargs)
def start(self):
self.storage_conn = get_connection('central')
super(Service, self).start()
def wait(self):
super(Service, self).wait()
self.conn.consumer_thread.wait()
# Currency
def create_currency(self, ctxt, values):
return self.storage_conn.create_currency(ctxt, values)
def list_currencies(self, ctxt, **kw):
return self.storage_conn.list_currencies(ctxt, **kw)
def get_currency(self, ctxt, id_):
return self.storage_conn.get_currency(ctxt, id_)
def update_currency(self, ctxt, id_, values):
return self.storage_conn.update_currency(ctxt, id_, values)
def delete_currency(self, ctxt, id_):
return self.storage_conn.delete_currency(ctxt, id_)
# Language
def create_language(self, ctxt, values):
return self.storage_conn.create_language(ctxt, values)
def list_languages(self, ctxt, **kw):
return self.storage_conn.list_languages(ctxt, **kw)
def get_language(self, ctxt, id_):
return self.storage_conn.get_language(ctxt, id_)
def update_language(self, ctxt, id_, values):
return self.storage_conn.update_language(ctxt, id_, values)
def delete_language(self, ctxt, id_):
return self.storage_conn.delete_language(ctxt, id_)
# Contact Info
def create_contact_info(self, ctxt, obj, values, cls=None,
rel_attr='contact_info'):
return self.storage_conn.create_contact_info(ctxt, values)
def get_contact_info(self, ctxt, id_):
return self.storage_conn.get_contact_info(ctxt, id_)
def update_contact_info(self, ctxt, id_, values):
return self.storage_conn.update_contact_info(ctxt, values)
def delete_contact_info(self, ctxt, id_):
return self.storage_conn.delete_contact_info(ctxt, id_)
# PGP
def list_pg_providers(self, ctxt, **kw):
return self.storage_conn.list_pg_providers(ctxt, **kw)
def get_pg_provider(self, ctxt, pgp_id):
return self.storage_conn.get_pg_provider(ctxt, pgp_id)
# Merchant
def create_merchant(self, ctxt, values):
flow = merchant.create_flow(self.storage_conn)
result = run_flow(flow, engine_conf="parallel",
store={'values': values, 'ctxt': ctxt})
return result['merchant']
def list_merchants(self, ctxt, **kw):
return self.storage_conn.list_merchants(ctxt, **kw)
def get_merchant(self, ctxt, id_):
return self.storage_conn.get_merchant(ctxt, id_)
def update_merchant(self, ctxt, id_, values):
return self.storage_conn.update_merchant(ctxt, id_, values)
def delete_merchant(self, ctxt, id_):
return self.storage_conn.delete_merchant(ctxt, id_)
# Customer
def create_customer(self, ctxt, merchant_id, values):
return self.storage_conn.create_customer(ctxt, merchant_id, values)
def list_customers(self, ctxt, **kw):
return self.storage_conn.list_customers(ctxt, **kw)
def get_customer(self, ctxt, id_):
return self.storage_conn.get_customer(ctxt, id_)
def update_customer(self, ctxt, id_, values):
return self.storage_conn.update_customer(ctxt, id_, values)
def delete_customer(self, ctxt, id_):
return self.storage_conn.delete_customer(ctxt, id_)
# Plans
def create_plan(self, ctxt, merchant_id, values):
return self.storage_conn.create_plan(ctxt, merchant_id, values)
def list_plans(self, ctxt, **kw):
return self.storage_conn.list_plans(ctxt, **kw)
def get_plan(self, ctxt, id_):
return self.storage_conn.get_plan(ctxt, id_)
def update_plan(self, ctxt, id_, values):
return self.storage_conn.update_plan(ctxt, id_, values)
def delete_plan(self, ctxt, id_):
return self.storage_conn.delete_plan(ctxt, id_)
def get_plan_by_subscription(self, ctxt, id_):
return self.storage_conn.get_plan_by_subscription(ctxt, id_)
# PlanItems
def create_plan_item(self, ctxt, values):
return self.storage_conn.create_plan_item(ctxt, values)
def list_plan_items(self, ctxt, **kw):
return self.storage_conn.list_plan_items(ctxt, **kw)
def get_plan_item(self, ctxt, plan_id, product_id):
return self.storage_conn.get_plan_item(ctxt, plan_id, product_id)
def update_plan_item(self, ctxt, plan_id, product_id, values):
return self.storage_conn.update_plan_item(
ctxt, plan_id, product_id, values)
def delete_plan_item(self, ctxt, plan_id, product_id):
return self.storage_conn.delete_plan_item(ctxt, plan_id, product_id)
# Products
def create_product(self, ctxt, merchant_id, values):
return self.storage_conn.create_product(ctxt, merchant_id, values)
def list_products(self, ctxt, **kw):
return self.storage_conn.list_products(ctxt, **kw)
def get_product(self, ctxt, id_):
return self.storage_conn.get_product(ctxt, id_)
def update_product(self, ctxt, id_, values):
return self.storage_conn.update_product(ctxt, id_, values)
def delete_product(self, ctxt, id_):
return self.storage_conn.delete_product(ctxt, id_)
# Subscriptions
def create_subscription(self, ctxt, values):
return self.storage_conn.create_subscription(ctxt, values)
def list_subscriptions(self, ctxt, **kw):
return self.storage_conn.list_subscriptions(ctxt, **kw)
def get_subscription(self, ctxt, id_):
return self.storage_conn.get_subscription(ctxt, id_)
def update_subscription(self, ctxt, id_, values):
return self.storage_conn.update_subscription(ctxt, id_, values)
def delete_subscription(self, ctxt, id_):
return self.storage_conn.delete_subscription(ctxt, id_)
def launch():
bs_service.prepare_service(sys.argv)
launcher = os_service.launch(Service(),
cfg.CONF['service:central'].workers)
launcher.wait()

31
billingstack/central/storage/__init__.py
View File

@@ -1,31 +0,0 @@
# Copyright 2012 Managed I.T.
#
# Author: Kiall Mac Innes <kiall@managedit.ie>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# Copied: Moniker
from billingstack.openstack.common import log as logging
from billingstack.storage import base
LOG = logging.getLogger(__name__)
class StorageEngine(base.StorageEngine):
__plugin_type__ = 'central'
__plugin_ns__ = 'billingstack.central.storage'
class Connection(base.Connection):
pass

502
billingstack/central/storage/impl_sqlalchemy/__init__.py
View File

@@ -1,502 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from sqlalchemy.orm import exc
from oslo.config import cfg
from billingstack.openstack.common import log as logging
from billingstack import exceptions
from billingstack import utils as common_utils
from billingstack.sqlalchemy import utils as db_utils, api
from billingstack.sqlalchemy.session import SQLOPTS
from billingstack.central.storage import Connection, StorageEngine
from billingstack.central.storage.impl_sqlalchemy import models
LOG = logging.getLogger(__name__)
cfg.CONF.register_group(cfg.OptGroup(
name='central:sqlalchemy', title="Configuration for SQLAlchemy Storage"
))
cfg.CONF.register_opts(SQLOPTS, group='central:sqlalchemy')
class SQLAlchemyEngine(StorageEngine):
__plugin_name__ = 'sqlalchemy'
def get_connection(self):
return Connection(self.name)
class Connection(Connection, api.HelpersMixin):
"""
SQLAlchemy connection
"""
def __init__(self, config_group):
self.setup(config_group)
def base(self):
return models.BASE
def set_properties(self, obj, properties, cls=None, rel_attr='properties',
purge=False):
"""
Set's a dict with key values on a relation on the row
:param obj: Either a row object or a id to use in connection with cls
:param properties: Key and Value dict with props to set. 1 row item.
:param cls: The class to use if obj isn't a row to query.
:param rel_attr: The relation attribute name to get the class to use
:param purge: Purge entries that doesn't exist in existing but in DB
"""
row = self._get_row(obj, cls=cls)
existing = self._kv_rows(row[rel_attr])
for key, value in properties.items():
values = {'name': key, 'value': value}
if key not in existing:
rel_row = self._make_rel_row(row, rel_attr, values)
row[rel_attr].append(rel_row)
else:
existing[key].update(values)
if purge:
for key in existing:
if not key in properties:
row[rel_attr].remove(existing[key])
# Currency
def create_currency(self, ctxt, values):
"""
Add a supported currency to the database
"""
data = common_utils.get_currency(values['name'])
row = models.Currency(**data)
self._save(row)
return dict(row)
def list_currencies(self, ctxt, **kw):
rows = self._list(models.Currency, **kw)
return map(dict, rows)
def get_currency(self, ctxt, id_):
row = self._get_id_or_name(models.Currency, id_)
return dict(row)
def update_currency(self, ctxt, id_, values):
row = self._update(models.Currency, id_, values, by_name=True)
return dict(row)
def delete_currency(self, ctxt, id_):
self._delete(models.Currency, id_, by_name=True)
# Language
def create_language(self, ctxt, values):
"""
Add a supported language to the database
"""
data = common_utils.get_language(values['name'])
row = models.Language(**data)
self._save(row)
return dict(row)
def list_languages(self, ctxt, **kw):
rows = self._list(models.Language, **kw)
return map(dict, rows)
def get_language(self, ctxt, id_):
row = self._get_id_or_name(models.Language, id_)
return dict(row)
def update_language(self, ctxt, id_, values):
row = self._update(models.Language, id_, values, by_name=True)
return dict(row)
def delete_language(self, ctxt, id_):
self._delete(models.Language, id_, by_name=True)
# ContactInfo
def create_contact_info(self, ctxt, obj, values, cls=None,
rel_attr='contact_info'):
"""
:param entity: The object to add the contact_info to
:param values: The values to add
"""
row = self._get_row(obj, cls=cls)
rel_row = self._make_rel_row(obj, rel_attr, values)
local, remote = db_utils.get_prop_names(row)
if rel_attr in remote:
if isinstance(row[rel_attr], list):
row[rel_attr].append(rel_row)
else:
row[rel_attr] = rel_row
else:
msg = 'Attempted to set non-relation %s' % rel_attr
raise exceptions.BadRequest(msg)
if cls:
self._save(rel_row)
return dict(rel_row)
else:
return rel_row
def get_contact_info(self, ctxt, id_):
self._get(models.ContactInfo, id_)
def update_contact_info(self, ctxt, id_, values):
return self._update(models.ContactInfo, id_, values)
def delete_contact_info(self, ctxt, id_):
self._delete(models.ContactInfo, id_)
# Merchant
def create_merchant(self, ctxt, values):
row = models.Merchant(**values)
self._save(row)
return dict(row)
def list_merchants(self, ctxt, **kw):
rows = self._list(models.Merchant, **kw)
return map(dict, rows)
def get_merchant(self, ctxt, id_):
row = self._get(models.Merchant, id_)
return dict(row)
def update_merchant(self, ctxt, id_, values):
row = self._update(models.Merchant, id_, values)
return dict(row)
def delete_merchant(self, ctxt, id_):
self._delete(models.Merchant, id_)
# Customer
def _customer(self, row):
data = dict(row)
data['contact_info'] = [dict(i) for i in row.contact_info]
data['default_info'] = dict(row.default_info) if row.default_info\
else {}
return data
def create_customer(self, ctxt, merchant_id, values):
merchant = self._get(models.Merchant, merchant_id)
contact_info = values.pop('contact_info', None)
customer = models.Customer(**values)
merchant.customers.append(customer)
if contact_info:
info_row = self.create_contact_info(ctxt, customer, contact_info)
customer.default_info = info_row
self._save(customer)
return self._customer(customer)
def list_customers(self, ctxt, **kw):
rows = self._list(models.Customer, **kw)
return map(dict, rows)
def get_customer(self, ctxt, id_):
row = self._get(models.Customer, id_)
return self._customer(row)
def update_customer(self, ctxt, id_, values):
row = self._update(models.Customer, id_, values)
return self._customer(row)
def delete_customer(self, ctxt, id_):
return self._delete(models.Customer, id_)
def _entity(self, row):
"""
Helper to serialize a entity like a Product or a Plan
:param row: The Row.
"""
entity = dict(row)
if hasattr(row, 'properties'):
entity['properties'] = self._kv_rows(
row.properties, func=lambda i: i['value'])
if hasattr(row, 'pricing'):
entity['pricing'] = row.pricing or []
return entity
# Plan
def _plan(self, row):
plan = self._entity(row)
plan['items'] = map(self._plan_item, row.plan_items) if row.plan_items\
else []
return plan
def create_plan(self, ctxt, merchant_id, values):
"""
Add a new Plan
:param merchant_id: The Merchant
:param values: Values describing the new Plan
"""
merchant = self._get(models.Merchant, merchant_id)
properties = values.pop('properties', {})
plan = models.Plan(**values)
plan.merchant = merchant
self.set_properties(plan, properties)
self._save(plan)
return self._plan(plan)
def list_plans(self, ctxt, **kw):
"""
List Plan
:param merchant_id: The Merchant to list it for
"""
rows = self._list(models.Plan, **kw)
return map(self._plan, rows)
def get_plan(self, ctxt, id_):
"""
Get a Plan
:param id_: The Plan ID
"""
row = self._get(models.Plan, id_)
return self._plan(row)
def update_plan(self, ctxt, id_, values):
"""
Update a Plan
:param id_: The Plan ID
:param values: Values to update with
"""
properties = values.pop('properties', {})
row = self._get(models.Plan, id_)
row.update(values)
self.set_properties(row, properties)
self._save(row)
return self._plan(row)
def delete_plan(self, ctxt, id_):
"""
Delete a Plan
:param id_: Plan ID
"""
self._delete(models.Plan, id_)
def get_plan_by_subscription(self, ctxt, subscription_id):
q = self.session.query(models.Plan).join(models.Subscription)\
.filter(models.Subscription.id == subscription_id)
try:
row = q.one()
except exc.NoResultFound:
msg = 'Couldn\'t find any Plan for subscription %s' % \
subscription_id
raise exceptions.NotFound(msg)
return self._plan(row)
# PlanItemw
def _plan_item(self, row):
entity = self._entity(row)
entity['name'] = row.product.name
entity['title'] = row.title or row.product.title
entity['description'] = row.description or row.product.description
return entity
def create_plan_item(self, ctxt, values):
row = models.PlanItem(**values)
self._save(row)
return self._entity(row)
def list_plan_items(self, ctxt, **kw):
return self._list(models.PlanItem, **kw)
def get_plan_item(self, ctxt, plan_id, product_id, criterion={}):
criterion.update({'plan_id': plan_id, 'product_id': product_id})
row = self._get(models.PlanItem, criterion=criterion)
return self._entity(row)
def update_plan_item(self, ctxt, plan_id, product_id, values):
criterion = {'plan_id': plan_id, 'product_id': product_id}
row = self._get(models.PlanItem, criterion=criterion)
row.update(values)
self._save(row)
return self._entity(row)
def delete_plan_item(self, ctxt, plan_id, product_id):
"""
Remove a Product from a Plan by deleting the PlanItem.
:param plan_id: The Plan's ID.
:param product_id: The Product's ID.
"""
query = self.session.query(models.PlanItem).\
filter_by(plan_id=plan_id, product_id=product_id)
count = query.delete()
if count == 0:
msg = 'Couldn\'t match plan_id %s or product_id %s' % (
plan_id, product_id)
raise exceptions.NotFound(msg)
# Products
def _product(self, row):
product = self._entity(row)
return product
def create_product(self, ctxt, merchant_id, values):
"""
Add a new Product
:param merchant_id: The Merchant
:param values: Values describing the new Product
"""
values = values.copy()
merchant = self._get(models.Merchant, merchant_id)
properties = values.pop('properties', {})
product = models.Product(**values)
product.merchant = merchant
self.set_properties(product, properties)
self._save(product)
return self._product(product)
def list_products(self, ctxt, **kw):
"""
List Products
:param merchant_id: The Merchant to list it for
"""
rows = self._list(models.Product, **kw)
return map(self._product, rows)
def get_product(self, ctxt, id_):
"""
Get a Product
:param id_: The Product ID
"""
row = self._get(models.Product, id_)
return self._product(row)
def update_product(self, ctxt, id_, values):
"""
Update a Product
:param id_: The Product ID
:param values: Values to update with
"""
values = values.copy()
properties = values.pop('properties', {})
row = self._get(models.Product, id_)
row.update(values)
self.set_properties(row, properties)
self._save(row)
return self._product(row)
def delete_product(self, ctxt, id_):
"""
Delete a Product
:param id_: Product ID
"""
self._delete(models.Product, id_)
# Subscriptions
def _subscription(self, row):
subscription = dict(row)
return subscription
def create_subscription(self, ctxt, values):
"""
Add a new Subscription
:param merchant_id: The Merchant
:param values: Values describing the new Subscription
"""
subscription = models.Subscription(**values)
self._save(subscription)
return self._subscription(subscription)
def list_subscriptions(self, ctxt, criterion=None, **kw):
"""
List Subscriptions
:param merchant_id: The Merchant to list it for
"""
query = self.session.query(models.Subscription)
# NOTE: Filter needs to be joined for merchant_id
query = db_utils.filter_merchant_by_join(
query, models.Customer, criterion)
rows = self._list(
query=query,
cls=models.Subscription,
criterion=criterion,
**kw)
return map(self._subscription, rows)
def get_subscription(self, ctxt, id_):
"""
Get a Subscription
:param id_: The Subscription ID
"""
row = self._get(models.Subscription, id_)
return self._subscription(row)
def update_subscription(self, ctxt, id_, values):
"""
Update a Subscription
:param id_: The Subscription ID
:param values: Values to update with
"""
row = self._get(models.Subscription, id_)
row.update(values)
self._save(row)
return self._subscription(row)
def delete_subscription(self, ctxt, id_):
"""
Delete a Subscription
:param id_: Subscription ID
"""
self._delete(models.Subscription, id_)

94
billingstack/central/storage/impl_sqlalchemy/migration/README.md
View File

@@ -1,94 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
#
# Copyright 2012 New Dream Network, LLC (DreamHost)
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# @author Mark McClain (DreamHost)
The migrations in the alembic/versions contain the changes needed to migrate
from older billingstack releases to newer versions. A migration occurs by executing
a script that details the changes needed to upgrade/downgrade the database. The
migration scripts are ordered so that multiple scripts can run sequentially to
update the database. The scripts are executed by billingstack's migration wrapper
which uses the Alembic library to manage the migration. billingstack supports
migration from Folsom or later.
If you are a deployer or developer and want to migrate from Folsom to Grizzly
or later you must first add version tracking to the database:
$ billingstack-db-manage -config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini stamp folsom
You can then upgrade to the latest database version via:
$ billingstack-db-manage --config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini upgrade head
To check the current database version:
$ billingstack-db-manage --config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini current
To create a script to run the migration offline:
$ billingstack-db-manage --config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini upgrade head --sql
To run the offline migration between specific migration versions:
$ billingstack-db-manage --config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini upgrade \
<start version>:<end version> --sql
Upgrade the database incrementally:
$ billingstack-db-manage --config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini upgrade --delta <# of revs>
Downgrade the database by a certain number of revisions:
$ billingstack-db-manage --config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini downgrade --delta <# of revs>
DEVELOPERS:
A database migration script is required when you submit a change to billingstack
that alters the database model definition. The migration script is a special
python file that includes code to update/downgrade the database to match the
changes in the model definition. Alembic will execute these scripts in order to
provide a linear migration path between revision. The billingstack-db-manage command
can be used to generate migration template for you to complete. The operations
in the template are those supported by the Alembic migration library.
$ billingstack-db-manage --config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini revision \
-m "description of revision" \
--autogenerate
This generates a prepopulated template with the changes needed to match the
database state with the models. You should inspect the autogenerated template
to ensure that the proper models have been altered.
In rare circumstances, you may want to start with an empty migration template
and manually author the changes necessary for an upgrade/downgrade. You can
create a blank file via:
$ billingstack-db-manage --config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini revision \
-m "description of revision"
The migration timeline should remain linear so that there is a clear path when
upgrading/downgrading. To verify that the timeline does branch, you can run
this command:
$ billingstack-db-manage --config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini check_migration
If the migration path does branch, you can find the branch point via:
$ billingstack-db-manage --config-file /path/to/quantum.conf \
--config-file /path/to/plugin/config.ini history

0
billingstack/central/storage/impl_sqlalchemy/migration/__init__.py
View File

52
billingstack/central/storage/impl_sqlalchemy/migration/alembic.ini
View File

@@ -1,52 +0,0 @@
# A generic, single database configuration.
[alembic]
# path to migration scripts
script_location = %(here)s/alembic
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# default to an empty string because the Quantum migration cli will
# extract the correct value and set it programatically before alemic is fully
# invoked.
sqlalchemy.url =
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

0
billingstack/central/storage/impl_sqlalchemy/migration/alembic_migrations/__init__.py
View File

91
billingstack/central/storage/impl_sqlalchemy/migration/alembic_migrations/env.py
View File

@@ -1,91 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
#
# Copyright 2012 New Dream Network, LLC (DreamHost)
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# @author: Mark McClain, DreamHost
# Copied: Quantum
from logging.config import fileConfig
from alembic import context
from sqlalchemy import create_engine, pool
from billingstack.central.storage.impl_sqlalchemy.models import ModelBase
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
billingstack_config = config.billingstack_config
# Interpret the config file for Python logging.
# This line sets up loggers basically.
fileConfig(config.config_file_name)
# set the target for 'autogenerate' support
target_metadata = ModelBase.metadata
def run_migrations_offline():
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
context.configure(url=billingstack_config['central:sqlalchemy']
.database_connection)
with context.begin_transaction():
context.run_migrations(options=build_options())
def run_migrations_online():
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
engine = create_engine(
billingstack_config['central:sqlalchemy'].database_connection,
poolclass=pool.NullPool)
connection = engine.connect()
context.configure(
connection=connection,
target_metadata=target_metadata
)
try:
with context.begin_transaction():
context.run_migrations(options=build_options())
finally:
connection.close()
def build_options():
return {}
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

40
billingstack/central/storage/impl_sqlalchemy/migration/alembic_migrations/script.py.mako
View File

@@ -1,40 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
#
# Copyright ${create_date.year} OpenStack LLC
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision}
Create Date: ${create_date}
"""
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
def upgrade(options=None):
${upgrades if upgrades else "pass"}
def downgrade(config=None):
${downgrades if downgrades else "pass"}

3
billingstack/central/storage/impl_sqlalchemy/migration/alembic_migrations/versions/README
View File

@@ -1,3 +0,0 @@
This directory contains the migration scripts for the billingstack project. Please
see the README in billinstack/db/migration on how to use and generate new
migrations.

125
billingstack/central/storage/impl_sqlalchemy/migration/cli.py
View File

@@ -1,125 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
#
# Copyright 2012 New Dream Network, LLC (DreamHost)
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# @author: Mark McClain, DreamHost
# Copied: Quantum
import os
from alembic import command as alembic_command
from alembic import config as alembic_config
from alembic import util as alembic_util
from oslo.config import cfg
from billingstack.openstack.common.gettextutils import _
_db_opts = [
cfg.StrOpt('database_connection',
default='',
help=_('URL to database')),
]
CONF = cfg.ConfigOpts()
CONF.register_opts(_db_opts, 'central:sqlalchemy')
def do_alembic_command(config, cmd, *args, **kwargs):
try:
getattr(alembic_command, cmd)(config, *args, **kwargs)
except alembic_util.CommandError, e:
alembic_util.err(str(e))
def do_check_migration(config, cmd):
do_alembic_command(config, 'branches')
def do_upgrade_downgrade(config, cmd):
if not CONF.command.revision and not CONF.command.delta:
raise SystemExit(_('You must provide a revision or relative delta'))
revision = CONF.command.revision
if CONF.command.delta:
sign = '+' if CONF.command.name == 'upgrade' else '-'
revision = sign + str(CONF.command.delta)
else:
revision = CONF.command.revision
do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
def do_stamp(config, cmd):
do_alembic_command(config, cmd,
CONF.command.revision,
sql=CONF.command.sql)
def do_revision(config, cmd):
do_alembic_command(config, cmd,
message=CONF.command.message,
autogenerate=CONF.command.autogenerate,
sql=CONF.command.sql)
def add_command_parsers(subparsers):
for name in ['current', 'history', 'branches']:
parser = subparsers.add_parser(name)
parser.set_defaults(func=do_alembic_command)
parser = subparsers.add_parser('check_migration')
parser.set_defaults(func=do_check_migration)
for name in ['upgrade', 'downgrade']:
parser = subparsers.add_parser(name)
parser.add_argument('--delta', type=int)
parser.add_argument('--sql', action='store_true')
parser.add_argument('revision', nargs='?')
parser.set_defaults(func=do_upgrade_downgrade)
parser = subparsers.add_parser('stamp')
parser.add_argument('--sql', action='store_true')
parser.add_argument('revision')
parser.set_defaults(func=do_stamp)
parser = subparsers.add_parser('revision')
parser.add_argument('-m', '--message')
parser.add_argument('--autogenerate', action='store_true')
parser.add_argument('--sql', action='store_true')
parser.set_defaults(func=do_revision)
command_opt = cfg.SubCommandOpt('command',
title='Command',
help=_('Available commands'),
handler=add_command_parsers)
CONF.register_cli_opt(command_opt)
def main():
config = alembic_config.Config(
os.path.join(os.path.dirname(__file__), 'alembic.ini')
)
config.set_main_option(
'script_location',
'billingstack.central.storage'
'.impl_sqlalchemy.migration:alembic_migrations')
# attach the Quantum conf to the Alembic conf
config.billingstack_config = CONF
CONF()
CONF.command.func(config, CONF.command.name)

228
billingstack/central/storage/impl_sqlalchemy/models.py
View File

@@ -1,228 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from sqlalchemy import Column, ForeignKey, UniqueConstraint
from sqlalchemy import Integer, Unicode
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base, declared_attr
from billingstack import utils
from billingstack.openstack.common import log as logging
from billingstack.sqlalchemy.types import JSON, UUID
from billingstack.sqlalchemy.model_base import (
ModelBase, BaseMixin, PropertyMixin)
LOG = logging.getLogger(__name__)
BASE = declarative_base(cls=ModelBase)
class Currency(BASE):
"""
Allowed currency
"""
name = Column(Unicode(10), nullable=False, primary_key=True)
title = Column(Unicode(100), nullable=False)
class Language(BASE):
"""
A Language
"""
name = Column(Unicode(10), nullable=False, primary_key=True)
title = Column(Unicode(100), nullable=False)
class ContactInfo(BASE, BaseMixin):
"""
Contact Information about an entity like a User, Customer etc...
"""
@declared_attr
def __mapper_args__(cls):
name = unicode(utils.capital_to_underscore(cls.__name__))
return {"polymorphic_on": "info_type", "polymorphic_identity": name}
info_type = Column(Unicode(20), nullable=False)
first_name = Column(Unicode(100))
last_name = Column(Unicode(100))
company = Column(Unicode(100))
address1 = Column(Unicode(255))
address2 = Column(Unicode(255))
address3 = Column(Unicode(255))
locality = Column(Unicode(60))
region = Column(Unicode(60))
country_name = Column(Unicode(100))
postal_code = Column(Unicode(40))
phone = Column(Unicode(100))
email = Column(Unicode(100))
website = Column(Unicode(100))
class CustomerInfo(ContactInfo):
id = Column(UUID, ForeignKey("contact_info.id",
onupdate='CASCADE', ondelete='CASCADE'),
primary_key=True)
customer_id = Column(UUID, ForeignKey('customer.id'), nullable=False)
class Merchant(BASE, BaseMixin):
"""
A Merchant is like a Account in Recurly
"""
name = Column(Unicode(60), nullable=False)
title = Column(Unicode(60))
customers = relationship('Customer', backref='merchant')
plans = relationship('Plan', backref='merchant')
products = relationship('Product', backref='merchant')
currency = relationship('Currency', uselist=False, backref='merchants')
currency_name = Column(Unicode(10), ForeignKey('currency.name'),
nullable=False)
language = relationship('Language', uselist=False, backref='merchants')
language_name = Column(Unicode(10), ForeignKey('language.name'),
nullable=False)
class Customer(BASE, BaseMixin):
"""
A Customer is linked to a Merchant and can have Users related to it
"""
name = Column(Unicode(60), nullable=False)
title = Column(Unicode(60))
merchant_id = Column(UUID, ForeignKey('merchant.id', ondelete='CASCADE'),
nullable=False)
contact_info = relationship(
'CustomerInfo',
backref='customer',
primaryjoin='Customer.id == CustomerInfo.customer_id',
lazy='joined')
default_info = relationship(
'CustomerInfo',
primaryjoin='Customer.default_info_id == CustomerInfo.id',
uselist=False,
post_update=True)
default_info_id = Column(
UUID,
ForeignKey('customer_info.id', use_alter=True,
onupdate='CASCADE', name='default_info'))
currency = relationship('Currency', uselist=False, backref='customers')
currency_name = Column(Unicode(10), ForeignKey('currency.name'))
language = relationship('Language', uselist=False, backref='customers')
language_name = Column(Unicode(10), ForeignKey('language.name'))
class Plan(BASE, BaseMixin):
"""
A Product collection like a "Virtual Web Cluster" with 10 servers
"""
name = Column(Unicode(60), nullable=False)
title = Column(Unicode(100))
description = Column(Unicode(255))
#provider = Column(Unicode(255), nullable=False)
plan_items = relationship('PlanItem', backref='plan')
merchant_id = Column(UUID, ForeignKey('merchant.id',
ondelete='CASCADE'), nullable=False)
class PlanProperty(BASE, PropertyMixin):
__table_args__ = (UniqueConstraint('name', 'plan_id', name='plan'),)
plan = relationship('Plan', backref='properties', lazy='joined')
plan_id = Column(
UUID,
ForeignKey('plan.id',
ondelete='CASCADE',
onupdate='CASCADE'))
class PlanItem(BASE, BaseMixin):
__table_args__ = (UniqueConstraint('plan_id', 'product_id', name='item'),)
title = Column(Unicode(100))
description = Column(Unicode(255))
pricing = Column(JSON)
plan_id = Column(UUID, ForeignKey('plan.id', ondelete='CASCADE'),
onupdate='CASCADE', primary_key=True)
product = relationship('Product', backref='plan_items', uselist=False)
product_id = Column(UUID, ForeignKey('product.id', onupdate='CASCADE'),
primary_key=True)
class Product(BASE, BaseMixin):
"""
A sellable Product, like vCPU hours, bandwidth units
"""
name = Column(Unicode(60), nullable=False)
title = Column(Unicode(100))
description = Column(Unicode(255))
pricing = Column(JSON)
merchant_id = Column(UUID, ForeignKey('merchant.id', ondelete='CASCADE'),
nullable=False)
class ProductProperty(BASE, PropertyMixin):
"""
A Metadata row for something like Product or PlanItem
"""
__table_args__ = (UniqueConstraint('name', 'product_id', name='product'),)
product = relationship('Product', backref='properties', lazy='joined')
product_id = Column(
UUID,
ForeignKey('product.id',
ondelete='CASCADE',
onupdate='CASCADE'))
class Subscription(BASE, BaseMixin):
"""
The thing that ties together stuff that is to be billed
In other words a Plan which is a collection of Products or a Product.
"""
billing_day = Column(Integer)
resource_id = Column(Unicode(255), nullable=False)
resource_type = Column(Unicode(255), nullable=True)
plan = relationship('Plan', backref='subscriptions', uselist=False)
plan_id = Column(UUID, ForeignKey('plan.id', ondelete='CASCADE'),
nullable=False)
customer = relationship('Customer', backref='subscriptions')
customer_id = Column(UUID, ForeignKey('customer.id', ondelete='CASCADE'),
nullable=False)
payment_method_id = Column(UUID)

27
billingstack/collector/__init__.py
View File

@@ -1,27 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
cfg.CONF.register_group(cfg.OptGroup(
name='service:collector', title="Configuration for collector Service"
))
cfg.CONF.register_opts([
cfg.IntOpt('workers', default=None,
help='Number of worker processes to spawn'),
cfg.StrOpt('storage-driver', default='sqlalchemy',
help='The storage driver to use'),
], group='service:collector')

17
billingstack/collector/flows/__init__.py
View File

@@ -1,17 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2013 Hewlett-Packard Development Company, L.P.
#
# Author: Endre Karlson <endre.karlson@hp.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

97
billingstack/collector/flows/gateway_configuration.py
View File

@@ -1,97 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2013 Hewlett-Packard Development Company, L.P.
#
# Author: Endre Karlson <endre.karlson@hp.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from taskflow.patterns import linear_flow
from billingstack import exceptions
from billingstack import tasks
from billingstack.collector import states
from billingstack.openstack.common import log
from billingstack.payment_gateway import get_provider
ACTION = 'gateway_configuration:create'
LOG = log.getLogger(__name__)
class EntryCreateTask(tasks.RootTask):
def __init__(self, storage, **kw):
super(EntryCreateTask, self).__init__(**kw)
self.storage = storage
def execute(self, ctxt, values):
values['state'] = states.VERIFYING
return self.storage.create_pg_config(ctxt, values)
class PrerequirementsTask(tasks.RootTask):
"""
Fetch provider information for use in the next task.
"""
def __init__(self, storage, **kw):
super(PrerequirementsTask, self).__init__(**kw)
self.storage = storage
def execute(self, ctxt, gateway_config):
return self.storage.get_pg_provider(
ctxt, gateway_config['provider_id'])
class BackendVerifyTask(tasks.RootTask):
"""
This is the verification task that runs in a threaded flow.
1. Load the Provider Plugin via entrypoints
2. Instantiate the Plugin with the Config
3. Execute verify_config call
4. Update storage accordingly
"""
def __init__(self, storage, **kw):
super(BackendVerifyTask, self).__init__(**kw)
self.storage = storage
def execute(self, ctxt, gateway_config, gateway_provider):
gateway_provider_cls = get_provider(gateway_provider['name'])
gateway_provider_obj = gateway_provider_cls(gateway_config)
try:
gateway_provider_obj.verify_config()
except exceptions.ConfigurationError:
self.storage.update_pg_config(
ctxt, gateway_config['id'], {'state': states.INVALID})
raise
self.storage.update_pg_config(
ctxt, gateway_config['id'], {'state': states.ACTIVE})
def create_flow(storage):
flow = linear_flow.Flow(ACTION + ':initial')
entry_task = EntryCreateTask(
storage, provides='gateway_config', prefix=ACTION)
flow.add(entry_task)
backend_flow = linear_flow.Flow(ACTION + ':backend')
prereq_task = PrerequirementsTask(
storage, provides='gateway_provider', prefix=ACTION)
backend_flow.add(prereq_task)
backend_flow.add(BackendVerifyTask(storage, prefix=ACTION))
flow.add(backend_flow)
return flow

103
billingstack/collector/flows/payment_method.py
View File

@@ -1,103 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2013 Hewlett-Packard Development Company, L.P.
#
# Author: Endre Karlson <endre.karlson@hp.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from taskflow.patterns import linear_flow
from billingstack import exceptions
from billingstack import tasks
from billingstack.collector import states
from billingstack.openstack.common import log
from billingstack.payment_gateway import get_provider
ACTION = 'payment_method:create'
LOG = log.getLogger(__name__)
class EntryCreateTask(tasks.RootTask):
"""
Create the initial entry in the database
"""
def __init__(self, storage, **kw):
super(EntryCreateTask, self).__init__(**kw)
self.storage = storage
def execute(self, ctxt, values):
values['state'] = states.PENDING
return self.storage.create_payment_method(ctxt, values)
class PrerequirementsTask(tasks.RootTask):
"""
Task to get the config and the provider from the catalog / database.
"""
def __init__(self, storage, **kw):
super(PrerequirementsTask, self).__init__(**kw)
self.storage = storage
def execute(self, ctxt, values):
data = {}
data['gateway_config'] = self.storage.get_pg_config(
ctxt, values['provider_config_id'])
data['gateway_provider'] = self.storage.get_pg_provider(
ctxt, data['gateway_config']['provider_id'])
return data
class BackendCreateTask(tasks.RootTask):
def __init__(self, storage, **kw):
super(BackendCreateTask, self).__init__(**kw)
self.storage = storage
def execute(self, ctxt, payment_method, gateway_config, gateway_provider):
gateway_provider_cls = get_provider(gateway_provider['name'])
gateway_provider_obj = gateway_provider_cls(gateway_config)
try:
gateway_provider_obj.create_payment_method(
payment_method['customer_id'],
payment_method)
except exceptions.BadRequest:
self.storage.update_payment_method(
ctxt, payment_method['id'], {'status': states.INVALID})
raise
def create_flow(storage):
"""
The flow for the service to start
"""
flow = linear_flow.Flow(ACTION + ':initial')
entry_task = EntryCreateTask(storage, provides='payment_method',
prefix=ACTION)
flow.add(entry_task)
backend_flow = linear_flow.Flow(ACTION + ':backend')
prereq_task = PrerequirementsTask(
storage,
provides=set([
'gateway_config',
'gateway_provider']),
prefix=ACTION)
backend_flow.add(prereq_task)
backend_flow.add(BackendCreateTask(storage, prefix=ACTION))
flow.add(backend_flow)
return flow

94
billingstack/collector/rpcapi.py
View File

@@ -1,94 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
from billingstack.openstack.common.rpc import proxy
rpcapi_opts = [
cfg.StrOpt('collector_topic', default='collector',
help='the topic collector nodes listen on')
]
cfg.CONF.register_opts(rpcapi_opts)
class CollectorAPI(proxy.RpcProxy):
BASE_RPC_VERSION = '1.0'
def __init__(self):
super(CollectorAPI, self).__init__(
topic=cfg.CONF.collector_topic,
default_version=self.BASE_RPC_VERSION)
# PGP
def list_pg_providers(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_pg_providers',
criterion=criterion))
def get_pg_provider(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_pg_provider', id_=id_))
# PGM
def list_pg_methods(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_pg_methods',
criterion=criterion))
def get_pg_method(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_pg_method', id_=id_))
def delete_pg_method(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_pg_method', id_=id_))
# PGC
def create_pg_config(self, ctxt, values):
return self.call(ctxt, self.make_msg('create_pg_config',
values=values))
def list_pg_configs(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_pg_configs',
criterion=criterion))
def get_pg_config(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_pg_config', id_=id_))
def update_pg_config(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_pg_config', id_=id_,
values=values))
def delete_pg_config(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_pg_config', id_=id_))
# PaymentMethod
def create_payment_method(self, ctxt, values):
return self.call(ctxt, self.make_msg('create_payment_method',
values=values))
def list_payment_methods(self, ctxt, criterion=None):
return self.call(ctxt, self.make_msg('list_payment_methods',
criterion=criterion))
def get_payment_method(self, ctxt, id_):
return self.call(ctxt, self.make_msg('get_payment_method', id_=id_))
def update_payment_method(self, ctxt, id_, values):
return self.call(ctxt, self.make_msg('update_payment_method', id_=id_,
values=values))
def delete_payment_method(self, ctxt, id_):
return self.call(ctxt, self.make_msg('delete_payment_method', id_=id_))
collector_api = CollectorAPI()

108
billingstack/collector/service.py
View File

@@ -1,108 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
A service that does calls towards the PGP web endpoint or so
"""
import sys
from oslo.config import cfg
from taskflow.engines import run as run_flow
from billingstack.openstack.common import log as logging
from billingstack.openstack.common.rpc import service as rpc_service
from billingstack.openstack.common import service as os_service
from billingstack.storage.utils import get_connection
from billingstack.central.rpcapi import CentralAPI
from billingstack import service as bs_service
from billingstack.collector.flows import (
gateway_configuration, payment_method)
cfg.CONF.import_opt('host', 'billingstack.netconf')
cfg.CONF.import_opt('collector_topic', 'billingstack.collector.rpcapi')
cfg.CONF.import_opt('state_path', 'billingstack.paths')
LOG = logging.getLogger(__name__)
class Service(rpc_service.Service):
def __init__(self, *args, **kwargs):
kwargs.update(
host=cfg.CONF.host,
topic=cfg.CONF.collector_topic,
)
super(Service, self).__init__(*args, **kwargs)
# Get a storage connection
self.central_api = CentralAPI()
def start(self):
self.storage_conn = get_connection('collector')
super(Service, self).start()
def wait(self):
super(Service, self).wait()
self.conn.consumer_thread.wait()
# PGP
def list_pg_providers(self, ctxt, **kw):
return self.storage_conn.list_pg_providers(ctxt, **kw)
# PGC
def create_pg_config(self, ctxt, values):
flow = gateway_configuration.create_flow(self.storage_conn)
results = run_flow(flow, store={'values': values, 'ctxt': ctxt})
return results['gateway_config']
def list_pg_configs(self, ctxt, **kw):
return self.storage_conn.list_pg_configs(ctxt, **kw)
def get_pg_config(self, ctxt, id_):
return self.storage_conn.get_pg_config(ctxt, id_)
def update_pg_config(self, ctxt, id_, values):
return self.storage_conn.update_pg_config(ctxt, id_, values)
def delete_pg_config(self, ctxt, id_):
return self.storage_conn.delete_pg_config(ctxt, id_)
# PM
def create_payment_method(self, ctxt, values):
flow = payment_method.create_flow(self.storage_conn)
results = run_flow(flow, store={'values': values, 'ctxt': ctxt})
return results['payment_method']
def list_payment_methods(self, ctxt, **kw):
return self.storage_conn.list_payment_methods(ctxt, **kw)
def get_payment_method(self, ctxt, id_, **kw):
return self.storage_conn.get_payment_method(ctxt, id_)
def update_payment_method(self, ctxt, id_, values):
return self.storage_conn.update_payment_method(ctxt, id_, values)
def delete_payment_method(self, ctxt, id_):
return self.storage_conn.delete_payment_method(ctxt, id_)
def launch():
bs_service.prepare_service(sys.argv)
launcher = os_service.launch(Service(),
cfg.CONF['service:collector'].workers)
launcher.wait()

21
billingstack/collector/states.py
View File

@@ -1,21 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2013 Hewlett-Packard Development Company, L.P.
#
# Author: Endre Karlson <endre.karlson@hp.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
PENDING = u'PENDING'
VERIFYING = u'VERIFYING'
ACTIVE = u'ACTIVE'
INVALID = u'INVALID'

108
billingstack/collector/storage/__init__.py
View File

@@ -1,108 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from billingstack.storage import base
class StorageEngine(base.StorageEngine):
"""Base class for the collector storage"""
__plugin_ns__ = 'billingstack.collector.storage'
class Connection(base.Connection):
"""Define the base API for collector storage"""
def pg_provider_register(self):
"""
Register a Provider and it's Methods
"""
raise NotImplementedError
def list_pg_providers(self, ctxt, **kw):
"""
List available PG Providers
"""
raise NotImplementedError
def get_pg_provider(self, ctxt, id_):
"""
Get a PaymentGateway Provider
"""
raise NotImplementedError
def pg_provider_deregister(self, ctxt, id_):
"""
De-register a PaymentGateway Provider (Plugin) and all it's methods
"""
raise NotImplementedError
def create_pg_config(self, ctxt, values):
"""
Create a PaymentGateway Configuration
"""
raise NotImplementedError
def list_pg_configs(self, ctxt, **kw):
"""
List PaymentGateway Configurations
"""
raise NotImplementedError
def get_pg_config(self, ctxt, id_):
"""
Get a PaymentGateway Configuration
"""
raise NotImplementedError
def update_pg_config(self, ctxt, id_, values):
"""
Update a PaymentGateway Configuration
"""
raise NotImplementedError
def delete_pg_config(self, ctxt, id_):
"""
Delete a PaymentGateway Configuration
"""
raise NotImplementedError
def create_payment_method(self, ctxt, values):
"""
Configure a PaymentMethod like a CreditCard
"""
raise NotImplementedError
def list_payment_methods(self, ctxt, criterion=None, **kw):
"""
List a Customer's PaymentMethods
"""
raise NotImplementedError
def get_payment_method(self, ctxt, id_, **kw):
"""
Get a Customer's PaymentMethod
"""
raise NotImplementedError
def update_payment_method(self, ctxt, id_, values):
"""
Update a Customer's PaymentMethod
"""
raise NotImplementedError
def delete_payment_method(self, ctxt, id_):
"""
Delete a Customer's PaymentMethod
"""
raise NotImplementedError

263
billingstack/collector/storage/impl_sqlalchemy.py
View File

@@ -1,263 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
from sqlalchemy import Column, ForeignKey
from sqlalchemy import Unicode
from sqlalchemy.orm import exc, relationship
from sqlalchemy.ext.declarative import declarative_base
from billingstack.collector import states
from billingstack.collector.storage import Connection, StorageEngine
from billingstack.openstack.common import log as logging
from billingstack.sqlalchemy.types import JSON, UUID
from billingstack.sqlalchemy import api, model_base, session, utils
LOG = logging.getLogger(__name__)
BASE = declarative_base(cls=model_base.ModelBase)
cfg.CONF.register_group(cfg.OptGroup(
name='collector:sqlalchemy',
title='Config for collector sqlalchemy plugin'))
cfg.CONF.register_opts(session.SQLOPTS, group='collector:sqlalchemy')
class PGProvider(BASE, model_base.BaseMixin):
"""
A Payment Gateway - The thing that processes a Payment Method
This is registered either by the Admin or by the PaymentGateway plugin
"""
__tablename__ = 'pg_provider'
name = Column(Unicode(60), nullable=False)
title = Column(Unicode(100))
description = Column(Unicode(255))
properties = Column(JSON)
methods = relationship(
'PGMethod',
backref='provider',
lazy='joined')
def method_map(self):
return self.attrs_map(['provider_methods'])
class PGMethod(BASE, model_base.BaseMixin):
"""
This represents a PaymentGatewayProviders method with some information
like name, type etc to describe what is in other settings known as a
"CreditCard"
Example:
A Visa card: {"type": "creditcard", "visa"}
"""
__tablename__ = 'pg_method'
name = Column(Unicode(100), nullable=False)
title = Column(Unicode(100))
description = Column(Unicode(255))
type = Column(Unicode(100), nullable=False)
properties = Column(JSON)
# NOTE: This is so a PGMethod can be "owned" by a Provider, meaning that
# other Providers should not be able to use it.
provider_id = Column(UUID, ForeignKey(
'pg_provider.id',
ondelete='CASCADE',
onupdate='CASCADE'))
@staticmethod
def make_key(data):
return '%(type)s:%(name)s' % data
def key(self):
return self.make_key(self)
class PGConfig(BASE, model_base.BaseMixin):
"""
A Merchant's configuration of a PaymentGateway like api keys, url and more
"""
__tablename__ = 'pg_config'
name = Column(Unicode(100), nullable=False)
title = Column(Unicode(100))
properties = Column(JSON)
# Link to the Merchant
merchant_id = Column(UUID, nullable=False)
provider = relationship('PGProvider',
backref='merchant_configurations')
provider_id = Column(UUID, ForeignKey('pg_provider.id',
onupdate='CASCADE'),
nullable=False)
state = Column(Unicode(20), default=states.PENDING)
class PaymentMethod(BASE, model_base.BaseMixin):
name = Column(Unicode(255), nullable=False)
identifier = Column(Unicode(255), nullable=False)
expires = Column(Unicode(255))
properties = Column(JSON)
customer_id = Column(UUID, nullable=False)
provider_config = relationship('PGConfig', backref='payment_methods',
lazy='joined')
provider_config_id = Column(UUID, ForeignKey('pg_config.id',
onupdate='CASCADE'), nullable=False)
state = Column(Unicode(20), default=states.PENDING)
class SQLAlchemyEngine(StorageEngine):
__plugin_name__ = 'sqlalchemy'
def get_connection(self):
return Connection()
class Connection(Connection, api.HelpersMixin):
def __init__(self):
self.setup('collector:sqlalchemy')
def base(self):
return BASE
# Payment Gateway Providers
def pg_provider_register(self, ctxt, values):
values = values.copy()
methods = values.pop('methods', [])
query = self.session.query(PGProvider)\
.filter_by(name=values['name'])
try:
provider = query.one()
except exc.NoResultFound:
provider = PGProvider()
provider.update(values)
self._set_provider_methods(ctxt, provider, methods)
self._save(provider)
return self._dict(provider, extra=['methods'])
def list_pg_providers(self, ctxt, **kw):
rows = self._list(PGProvider, **kw)
return [self._dict(r, extra=['methods']) for r in rows]
def get_pg_provider(self, ctxt, id_, **kw):
row = self._get(PGProvider, id_)
return self._dict(row, extra=['methods'])
def pg_provider_deregister(self, ctxt, id_):
self._delete(PGProvider, id_)
def _get_provider_methods(self, provider):
"""
Used internally to form a "Map" of the Providers methods
"""
methods = {}
for m in provider.methods:
methods[m.key()] = m
return methods
def _set_provider_methods(self, ctxt, provider, config_methods):
"""Helper method for setting the Methods for a Provider"""
existing = self._get_provider_methods(provider)
for method in config_methods:
self._set_method(provider, method, existing)
def _set_method(self, provider, method, existing):
key = PGMethod.make_key(method)
if key in existing:
existing[key].update(method)
else:
row = PGMethod(**method)
provider.methods.append(row)
# Payment Gateway Configuration
def create_pg_config(self, ctxt, values):
row = PGConfig(**values)
self._save(row)
return dict(row)
def list_pg_configs(self, ctxt, **kw):
rows = self._list(PGConfig, **kw)
return map(dict, rows)
def get_pg_config(self, ctxt, id_, **kw):
row = self._get(PGConfig, id_, **kw)
return dict(row)
def update_pg_config(self, ctxt, id_, values):
row = self._update(PGConfig, id_, values)
return dict(row)
def delete_pg_config(self, ctxt, id_):
self._delete(PGConfig, id_)
# PaymentMethod
def create_payment_method(self, ctxt, values):
row = PaymentMethod(**values)
self._save(row)
return self._dict(row)
def list_payment_methods(self, ctxt, criterion=None, **kw):
query = self.session.query(PaymentMethod)
# NOTE: Filter needs to be joined for merchant_id
query = utils.filter_merchant_by_join(
query, PGConfig, criterion)
rows = self._list(
cls=PaymentMethod,
query=query,
criterion=criterion,
**kw)
return [self._dict(row) for row in rows]
def get_payment_method(self, ctxt, id_, **kw):
row = self._get_id_or_name(PaymentMethod, id_)
return self._dict(row)
def update_payment_method(self, ctxt, id_, values):
row = self._update(PaymentMethod, id_, values)
return self._dict(row)
def delete_payment_method(self, ctxt, id_):
self._delete(PaymentMethod, id_)

31
billingstack/conf.py
View File

@@ -1,31 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import os
from oslo.config import cfg
from billingstack.openstack.common import rpc
cfg.CONF.register_opts([
cfg.StrOpt('pybasedir',
default=os.path.abspath(os.path.join(os.path.dirname(__file__),
'../')),
help='Directory where the nova python module is installed'),
cfg.StrOpt('state-path', default='$pybasedir',
help='Top-level directory for maintaining state')
])
rpc.set_defaults(control_exchange='billingstack')

89
billingstack/exceptions.py
View File

@@ -1,89 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2012 Managed I.T.
#
# Author: Kiall Mac Innes <kiall@managedit.ie>
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import re
class Base(Exception):
error_code = 500
message_tmpl = None
def __init__(self, message='', *args, **kw):
self.message = message % kw if self.message_tmpl else message
self.errors = kw.pop('errors', None)
super(Base, self).__init__(self.message)
@property
def error_type(self):
name = "_".join(l.lower() for l in re.findall('[A-Z][^A-Z]*',
self.__class__.__name__))
name = re.sub('_+remote$', '', name)
return name
def __str__(self):
return self.message
def get_message(self):
"""
Return the exception message or None
"""
if unicode(self):
return unicode(self)
else:
return None
class NotImplemented(Base, NotImplementedError):
pass
class ConfigurationError(Base):
pass
class BadRequest(Base):
error_code = 400
class InvalidObject(BadRequest):
pass
class InvalidSortKey(BadRequest):
pass
class InvalidQueryField(BadRequest):
pass
class InvalidOperator(BadRequest):
pass
class Forbidden(Base):
pass
class Duplicate(Base):
error_code = 409
class NotFound(Base):
error_code = 404

33
billingstack/manage/__init__.py
View File

@@ -1,33 +0,0 @@
# Copyright 2012 Managed I.T.
#
# Author: Kiall Mac Innes <kiall@managedit.ie>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# Copied: Moniker
from oslo.config import cfg
from cliff.app import App
from cliff.commandmanager import CommandManager
from billingstack.version import version_info as version
cfg.CONF.import_opt('state_path', 'billingstack.paths')
class Shell(App):
def __init__(self):
super(Shell, self).__init__(
description='BillingStack Management CLI',
version=version.version_string(),
command_manager=CommandManager('billingstack.manage')
)

86
billingstack/manage/base.py
View File

@@ -1,86 +0,0 @@
# Copyright 2012 Managed I.T.
#
# Author: Kiall Mac Innes <kiall@managedit.ie>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# Copied: Moniker
from cliff.command import Command as CliffCommand
from cliff.lister import Lister
from cliff.show import ShowOne
from billingstack import utils
class Command(CliffCommand):
def run(self, parsed_args):
#self.context = billingstackContext.get_admin_context()
return super(Command, self).run(parsed_args)
def execute(self, parsed_args):
"""
Execute something, this is since we overload self.take_action()
in order to format the data
This method __NEEDS__ to be overloaded!
:param parsed_args: The parsed args that are given by take_action()
"""
raise NotImplementedError
def post_execute(self, data):
"""
Format the results locally if needed, by default we just return data
:param data: Whatever is returned by self.execute()
"""
return data
def setup(self, parsed_args):
pass
def take_action(self, parsed_args):
# TODO: Common Exception Handling Here
self.setup(parsed_args)
results = self.execute(parsed_args)
return self.post_execute(results)
class ListCommand(Command, Lister):
def post_execute(self, results):
if len(results) > 0:
columns = utils.get_columns(results)
data = [utils.get_item_properties(i, columns) for i in results]
return columns, data
else:
return [], ()
class GetCommand(Command, ShowOne):
def post_execute(self, results):
return results.keys(), results.values()
class CreateCommand(Command, ShowOne):
def post_execute(self, results):
return results.keys(), results.values()
class UpdateCommand(Command, ShowOne):
def post_execute(self, results):
return results.keys(), results.values()
class DeleteCommand(Command):
pass

34
billingstack/manage/database.py
View File

@@ -1,34 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
from billingstack.openstack.common import log
from billingstack.manage.base import Command
from billingstack.storage.utils import get_connection
LOG = log.getLogger(__name__)
cfg.CONF.import_opt('state_path', 'billingstack.paths')
class DatabaseCommand(Command):
"""
A Command that uses a storage connection to do some stuff
"""
def get_connection(self, service):
return get_connection(service)

42
billingstack/manage/provider.py
View File

@@ -1,42 +0,0 @@
# -*- encoding: utf-8 -*-
#
# Author: Endre Karlson <endre.karlson@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from billingstack.openstack.common.context import get_admin_context
from billingstack.payment_gateway import register_providers
from billingstack.manage.base import ListCommand
from billingstack.manage.database import DatabaseCommand
class ProvidersRegister(DatabaseCommand):
"""
Register Payment Gateway Providers
"""
def execute(self, parsed_args):
context = get_admin_context()
register_providers(context)
class ProvidersList(DatabaseCommand, ListCommand):
def execute(self, parsed_args):
context = get_admin_context()
conn = self.get_connection('collector')
data = conn.list_pg_providers(context)
for p in data:
keys = ['type', 'name']
methods = [":".join([m[k] for k in keys]) for m in p['methods']]
p['methods'] = ", ".join(methods)
return data

59
billingstack/netconf.py
View File

@@ -1,59 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
# Copyright 2012 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import socket
from oslo.config import cfg
CONF = cfg.CONF
def _get_my_ip():
"""
Returns the actual ip of the local machine.
This code figures out what source address would be used if some traffic
were to be sent out to some well known address on the Internet. In this
case, a Google DNS server is used, but the specific address does not
matter much. No traffic is actually sent.
"""
try:
csock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
csock.connect(('8.8.8.8', 80))
(addr, port) = csock.getsockname()
csock.close()
return addr
except socket.error:
return "127.0.0.1"
netconf_opts = [
cfg.StrOpt('my_ip',
default=_get_my_ip(),
help='ip address of this host'),
cfg.StrOpt('host',
default=socket.getfqdn(),
help='Name of this node. This can be an opaque identifier. '
'It is not necessarily a hostname, FQDN, or IP address. '
'However, the node name must be valid within '
'an AMQP key, and if using ZeroMQ, a valid '
'hostname, FQDN, or IP address')
]
CONF.register_opts(netconf_opts)

0
billingstack/openstack/__init__.py
View File

0
billingstack/openstack/common/__init__.py
View File

86
billingstack/openstack/common/context.py
View File

@@ -1,86 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Simple class that stores security context information in the web request.
Projects should subclass this class if they wish to enhance the request
context or provide additional information in their specific WSGI pipeline.
"""
import itertools
from billingstack.openstack.common import uuidutils
def generate_request_id():
return 'req-%s' % uuidutils.generate_uuid()
class RequestContext(object):
"""Helper class to represent useful information about a request context.
Stores information about the security context under which the user
accesses the system, as well as additional request information.
"""
def __init__(self, auth_token=None, user=None, tenant=None, is_admin=False,
read_only=False, show_deleted=False, request_id=None,
instance_uuid=None):
self.auth_token = auth_token
self.user = user
self.tenant = tenant
self.is_admin = is_admin
self.read_only = read_only
self.show_deleted = show_deleted
self.instance_uuid = instance_uuid
if not request_id:
request_id = generate_request_id()
self.request_id = request_id
def to_dict(self):
return {'user': self.user,
'tenant': self.tenant,
'is_admin': self.is_admin,
'read_only': self.read_only,
'show_deleted': self.show_deleted,
'auth_token': self.auth_token,
'request_id': self.request_id,
'instance_uuid': self.instance_uuid}
def get_admin_context(show_deleted=False):
context = RequestContext(None,
tenant=None,
is_admin=True,
show_deleted=show_deleted)
return context
def get_context_from_function_and_args(function, args, kwargs):
"""Find an arg of type RequestContext and return it.
This is useful in a couple of decorators where we don't
know much about the function we're wrapping.
"""
for arg in itertools.chain(kwargs.values(), args):
if isinstance(arg, RequestContext):
return arg
return None

0
billingstack/openstack/common/crypto/__init__.py
View File

179
billingstack/openstack/common/crypto/utils.py
View File

@@ -1,179 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import base64
from Crypto.Hash import HMAC
from Crypto import Random
from billingstack.openstack.common.gettextutils import _ # noqa
from billingstack.openstack.common import importutils
class CryptoutilsException(Exception):
"""Generic Exception for Crypto utilities."""
message = _("An unknown error occurred in crypto utils.")
class CipherBlockLengthTooBig(CryptoutilsException):
"""The block size is too big."""
def __init__(self, requested, permitted):
msg = _("Block size of %(given)d is too big, max = %(maximum)d")
message = msg % {'given': requested, 'maximum': permitted}
super(CryptoutilsException, self).__init__(message)
class HKDFOutputLengthTooLong(CryptoutilsException):
"""The amount of Key Material asked is too much."""
def __init__(self, requested, permitted):
msg = _("Length of %(given)d is too long, max = %(maximum)d")
message = msg % {'given': requested, 'maximum': permitted}
super(CryptoutilsException, self).__init__(message)
class HKDF(object):
"""An HMAC-based Key Derivation Function implementation (RFC5869)
This class creates an object that allows to use HKDF to derive keys.
"""
def __init__(self, hashtype='SHA256'):
self.hashfn = importutils.import_module('Crypto.Hash.' + hashtype)
self.max_okm_length = 255 * self.hashfn.digest_size
def extract(self, ikm, salt=None):
"""An extract function that can be used to derive a robust key given
weak Input Key Material (IKM) which could be a password.
Returns a pseudorandom key (of HashLen octets)
:param ikm: input keying material (ex a password)
:param salt: optional salt value (a non-secret random value)
"""
if salt is None:
salt = '\x00' * self.hashfn.digest_size
return HMAC.new(salt, ikm, self.hashfn).digest()
def expand(self, prk, info, length):
"""An expand function that will return arbitrary length output that can
be used as keys.
Returns a buffer usable as key material.
:param prk: a pseudorandom key of at least HashLen octets
:param info: optional string (can be a zero-length string)
:param length: length of output keying material (<= 255 * HashLen)
"""
if length > self.max_okm_length:
raise HKDFOutputLengthTooLong(length, self.max_okm_length)
N = (length + self.hashfn.digest_size - 1) / self.hashfn.digest_size
okm = ""
tmp = ""
for block in range(1, N + 1):
tmp = HMAC.new(prk, tmp + info + chr(block), self.hashfn).digest()
okm += tmp
return okm[:length]
MAX_CB_SIZE = 256
class SymmetricCrypto(object):
"""Symmetric Key Crypto object.
This class creates a Symmetric Key Crypto object that can be used
to encrypt, decrypt, or sign arbitrary data.
:param enctype: Encryption Cipher name (default: AES)
:param hashtype: Hash/HMAC type name (default: SHA256)
"""
def __init__(self, enctype='AES', hashtype='SHA256'):
self.cipher = importutils.import_module('Crypto.Cipher.' + enctype)
self.hashfn = importutils.import_module('Crypto.Hash.' + hashtype)
def new_key(self, size):
return Random.new().read(size)
def encrypt(self, key, msg, b64encode=True):
"""Encrypt the provided msg and returns the cyphertext optionally
base64 encoded.
Uses AES-128-CBC with a Random IV by default.
The plaintext is padded to reach blocksize length.
The last byte of the block is the length of the padding.
The length of the padding does not include the length byte itself.
:param key: The Encryption key.
:param msg: the plain text.
:returns encblock: a block of encrypted data.
"""
iv = Random.new().read(self.cipher.block_size)
cipher = self.cipher.new(key, self.cipher.MODE_CBC, iv)
# CBC mode requires a fixed block size. Append padding and length of
# padding.
if self.cipher.block_size > MAX_CB_SIZE:
raise CipherBlockLengthTooBig(self.cipher.block_size, MAX_CB_SIZE)
r = len(msg) % self.cipher.block_size
padlen = self.cipher.block_size - r - 1
msg += '\x00' * padlen
msg += chr(padlen)
enc = iv + cipher.encrypt(msg)
if b64encode:
enc = base64.b64encode(enc)
return enc
def decrypt(self, key, msg, b64decode=True):
"""Decrypts the provided ciphertext, optionally base 64 encoded, and
returns the plaintext message, after padding is removed.
Uses AES-128-CBC with an IV by default.
:param key: The Encryption key.
:param msg: the ciphetext, the first block is the IV
"""
if b64decode:
msg = base64.b64decode(msg)
iv = msg[:self.cipher.block_size]
cipher = self.cipher.new(key, self.cipher.MODE_CBC, iv)
padded = cipher.decrypt(msg[self.cipher.block_size:])
l = ord(padded[-1]) + 1
plain = padded[:-l]
return plain
def sign(self, key, msg, b64encode=True):
"""Signs a message string and returns a base64 encoded signature.
Uses HMAC-SHA-256 by default.
:param key: The Signing key.
:param msg: the message to sign.
"""
h = HMAC.new(key, msg, self.hashfn)
out = h.digest()
if b64encode:
out = base64.b64encode(out)
return out

16
billingstack/openstack/common/db/__init__.py
View File

@@ -1,16 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2012 Cloudscaling Group, Inc
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

106
billingstack/openstack/common/db/api.py
View File

@@ -1,106 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2013 Rackspace Hosting
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Multiple DB API backend support.
Supported configuration options:
The following two parameters are in the 'database' group:
`backend`: DB backend name or full module path to DB backend module.
`use_tpool`: Enable thread pooling of DB API calls.
A DB backend module should implement a method named 'get_backend' which
takes no arguments. The method can return any object that implements DB
API methods.
*NOTE*: There are bugs in eventlet when using tpool combined with
threading locks. The python logging module happens to use such locks. To
work around this issue, be sure to specify thread=False with
eventlet.monkey_patch().
A bug for eventlet has been filed here:
https://bitbucket.org/eventlet/eventlet/issue/137/
"""
import functools
from oslo.config import cfg
from billingstack.openstack.common import importutils
from billingstack.openstack.common import lockutils
db_opts = [
cfg.StrOpt('backend',
default='sqlalchemy',
deprecated_name='db_backend',
deprecated_group='DEFAULT',
help='The backend to use for db'),
cfg.BoolOpt('use_tpool',
default=False,
deprecated_name='dbapi_use_tpool',
deprecated_group='DEFAULT',
help='Enable the experimental use of thread pooling for '
'all DB API calls')
]
CONF = cfg.CONF
CONF.register_opts(db_opts, 'database')
class DBAPI(object):
def __init__(self, backend_mapping=None):
if backend_mapping is None:
backend_mapping = {}
self.__backend = None
self.__backend_mapping = backend_mapping
@lockutils.synchronized('dbapi_backend', 'billingstack-')
def __get_backend(self):
"""Get the actual backend. May be a module or an instance of
a class. Doesn't matter to us. We do this synchronized as it's
possible multiple greenthreads started very quickly trying to do
DB calls and eventlet can switch threads before self.__backend gets
assigned.
"""
if self.__backend:
# Another thread assigned it
return self.__backend
backend_name = CONF.database.backend
self.__use_tpool = CONF.database.use_tpool
if self.__use_tpool:
from eventlet import tpool
self.__tpool = tpool
# Import the untranslated name if we don't have a
# mapping.
backend_path = self.__backend_mapping.get(backend_name,
backend_name)
backend_mod = importutils.import_module(backend_path)
self.__backend = backend_mod.get_backend()
return self.__backend
def __getattr__(self, key):
backend = self.__backend or self.__get_backend()
attr = getattr(backend, key)
if not self.__use_tpool or not hasattr(attr, '__call__'):
return attr
def tpool_wrapper(*args, **kwargs):
return self.__tpool.execute(attr, *args, **kwargs)
functools.update_wrapper(tpool_wrapper, attr)
return tpool_wrapper

51
billingstack/openstack/common/db/exception.py
View File

@@ -1,51 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""DB related custom exceptions."""
from billingstack.openstack.common.gettextutils import _ # noqa
class DBError(Exception):
"""Wraps an implementation specific exception."""
def __init__(self, inner_exception=None):
self.inner_exception = inner_exception
super(DBError, self).__init__(str(inner_exception))
class DBDuplicateEntry(DBError):
"""Wraps an implementation specific exception."""
def __init__(self, columns=[], inner_exception=None):
self.columns = columns
super(DBDuplicateEntry, self).__init__(inner_exception)
class DBDeadlock(DBError):
def __init__(self, inner_exception=None):
super(DBDeadlock, self).__init__(inner_exception)
class DBInvalidUnicodeParameter(Exception):
message = _("Invalid Parameter: "
"Unicode is not supported by the current database.")
class DbMigrationError(DBError):
"""Wraps migration specific exception."""
def __init__(self, message=None):
super(DbMigrationError, self).__init__(str(message))

16
billingstack/openstack/common/db/sqlalchemy/__init__.py
View File

@@ -1,16 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2012 Cloudscaling Group, Inc
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

103
billingstack/openstack/common/db/sqlalchemy/models.py
View File

@@ -1,103 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2011 X.commerce, a business unit of eBay Inc.
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# Copyright 2011 Piston Cloud Computing, Inc.
# Copyright 2012 Cloudscaling Group, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
SQLAlchemy models.
"""
from sqlalchemy import Column, Integer
from sqlalchemy import DateTime
from sqlalchemy.orm import object_mapper
from billingstack.openstack.common.db.sqlalchemy.session import get_session
from billingstack.openstack.common import timeutils
class ModelBase(object):
"""Base class for models."""
__table_initialized__ = False
created_at = Column(DateTime, default=timeutils.utcnow)
updated_at = Column(DateTime, onupdate=timeutils.utcnow)
metadata = None
def save(self, session=None):
"""Save this object."""
if not session:
session = get_session()
# NOTE(boris-42): This part of code should be look like:
# sesssion.add(self)
# session.flush()
# But there is a bug in sqlalchemy and eventlet that
# raises NoneType exception if there is no running
# transaction and rollback is called. As long as
# sqlalchemy has this bug we have to create transaction
# explicity.
with session.begin(subtransactions=True):
session.add(self)
session.flush()
def __setitem__(self, key, value):
setattr(self, key, value)
def __getitem__(self, key):
return getattr(self, key)
def get(self, key, default=None):
return getattr(self, key, default)
def __iter__(self):
columns = dict(object_mapper(self).columns).keys()
# NOTE(russellb): Allow models to specify other keys that can be looked
# up, beyond the actual db columns. An example would be the 'name'
# property for an Instance.
if hasattr(self, '_extra_keys'):
columns.extend(self._extra_keys())
self._i = iter(columns)
return self
def next(self):
n = self._i.next()
return n, getattr(self, n)
def update(self, values):
"""Make the model object behave like a dict."""
for k, v in values.iteritems():
setattr(self, k, v)
def iteritems(self):
"""Make the model object behave like a dict.
Includes attributes from joins."""
local = dict(self)
joined = dict([(k, v) for k, v in self.__dict__.iteritems()
if not k[0] == '_'])
local.update(joined)
return local.iteritems()
class SoftDeleteMixin(object):
deleted_at = Column(DateTime)
deleted = Column(Integer, default=0)
def soft_delete(self, session=None):
"""Mark this object as deleted."""
self.deleted = self.id
self.deleted_at = timeutils.utcnow()
self.save(session=session)

132
billingstack/openstack/common/db/sqlalchemy/utils.py
View File

@@ -1,132 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# Copyright 2010-2011 OpenStack LLC.
# Copyright 2012 Justin Santa Barbara
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Implementation of paginate query."""
import sqlalchemy
from openstack.common.gettextutils import _
from openstack.common import log as logging
LOG = logging.getLogger(__name__)
class InvalidSortKey(Exception):
message = _("Sort key supplied was not valid.")
# copy from glance/db/sqlalchemy/api.py
def paginate_query(query, model, limit, sort_keys, marker=None,
sort_dir=None, sort_dirs=None):
"""Returns a query with sorting / pagination criteria added.
Pagination works by requiring a unique sort_key, specified by sort_keys.
(If sort_keys is not unique, then we risk looping through values.)
We use the last row in the previous page as the 'marker' for pagination.
So we must return values that follow the passed marker in the order.
With a single-valued sort_key, this would be easy: sort_key > X.
With a compound-values sort_key, (k1, k2, k3) we must do this to repeat
the lexicographical ordering:
(k1 > X1) or (k1 == X1 && k2 > X2) or (k1 == X1 && k2 == X2 && k3 > X3)
We also have to cope with different sort_directions.
Typically, the id of the last row is used as the client-facing pagination
marker, then the actual marker object must be fetched from the db and
passed in to us as marker.
:param query: the query object to which we should add paging/sorting
:param model: the ORM model class
:param limit: maximum number of items to return
:param sort_keys: array of attributes by which results should be sorted
:param marker: the last item of the previous page; we returns the next
results after this value.
:param sort_dir: direction in which results should be sorted (asc, desc)
:param sort_dirs: per-column array of sort_dirs, corresponding to sort_keys
:rtype: sqlalchemy.orm.query.Query
:return: The query with sorting/pagination added.
"""
if 'id' not in sort_keys:
# TODO(justinsb): If this ever gives a false-positive, check
# the actual primary key, rather than assuming its id
LOG.warn(_('Id not in sort_keys; is sort_keys unique?'))
assert(not (sort_dir and sort_dirs))
# Default the sort direction to ascending
if sort_dirs is None and sort_dir is None:
sort_dir = 'asc'
# Ensure a per-column sort direction
if sort_dirs is None:
sort_dirs = [sort_dir for _sort_key in sort_keys]
assert(len(sort_dirs) == len(sort_keys))
# Add sorting
for current_sort_key, current_sort_dir in zip(sort_keys, sort_dirs):
sort_dir_func = {
'asc': sqlalchemy.asc,
'desc': sqlalchemy.desc,
}[current_sort_dir]
try:
sort_key_attr = getattr(model, current_sort_key)
except AttributeError:
raise InvalidSortKey()
query = query.order_by(sort_dir_func(sort_key_attr))
# Add pagination
if marker is not None:
marker_values = []
for sort_key in sort_keys:
v = getattr(marker, sort_key)
marker_values.append(v)
# Build up an array of sort criteria as in the docstring
criteria_list = []
for i in xrange(0, len(sort_keys)):
crit_attrs = []
for j in xrange(0, i):
model_attr = getattr(model, sort_keys[j])
crit_attrs.append((model_attr == marker_values[j]))
model_attr = getattr(model, sort_keys[i])
if sort_dirs[i] == 'desc':
crit_attrs.append((model_attr < marker_values[i]))
elif sort_dirs[i] == 'asc':
crit_attrs.append((model_attr > marker_values[i]))
else:
raise ValueError(_("Unknown sort direction, "
"must be 'desc' or 'asc'"))
criteria = sqlalchemy.sql.and_(*crit_attrs)
criteria_list.append(criteria)
f = sqlalchemy.sql.or_(*criteria_list)
query = query.filter(f)
if limit is not None:
query = query.limit(limit)
return query

146
billingstack/openstack/common/eventlet_backdoor.py
View File

@@ -1,146 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2012 OpenStack Foundation.
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from __future__ import print_function
import errno
import gc
import os
import pprint
import socket
import sys
import traceback
import eventlet
import eventlet.backdoor
import greenlet
from oslo.config import cfg
from billingstack.openstack.common.gettextutils import _ # noqa
from billingstack.openstack.common import log as logging
help_for_backdoor_port = (
"Acceptable values are 0, <port>, and <start>:<end>, where 0 results "
"in listening on a random tcp port number; <port> results in listening "
"on the specified port number (and not enabling backdoor if that port "
"is in use); and <start>:<end> results in listening on the smallest "
"unused port number within the specified range of port numbers. The "
"chosen port is displayed in the service's log file.")
eventlet_backdoor_opts = [
cfg.StrOpt('backdoor_port',
default=None,
help="Enable eventlet backdoor. %s" % help_for_backdoor_port)
]
CONF = cfg.CONF
CONF.register_opts(eventlet_backdoor_opts)
LOG = logging.getLogger(__name__)
class EventletBackdoorConfigValueError(Exception):
def __init__(self, port_range, help_msg, ex):
msg = ('Invalid backdoor_port configuration %(range)s: %(ex)s. '
'%(help)s' %
{'range': port_range, 'ex': ex, 'help': help_msg})
super(EventletBackdoorConfigValueError, self).__init__(msg)
self.port_range = port_range
def _dont_use_this():
print("Don't use this, just disconnect instead")
def _find_objects(t):
return filter(lambda o: isinstance(o, t), gc.get_objects())
def _print_greenthreads():
for i, gt in enumerate(_find_objects(greenlet.greenlet)):
print(i, gt)
traceback.print_stack(gt.gr_frame)
print()
def _print_nativethreads():
for threadId, stack in sys._current_frames().items():
print(threadId)
traceback.print_stack(stack)
print()
def _parse_port_range(port_range):
if ':' not in port_range:
start, end = port_range, port_range
else:
start, end = port_range.split(':', 1)
try:
start, end = int(start), int(end)
if end < start:
raise ValueError
return start, end
except ValueError as ex:
raise EventletBackdoorConfigValueError(port_range, ex,
help_for_backdoor_port)
def _listen(host, start_port, end_port, listen_func):
try_port = start_port
while True:
try:
return listen_func((host, try_port))
except socket.error as exc:
if (exc.errno != errno.EADDRINUSE or
try_port >= end_port):
raise
try_port += 1
def initialize_if_enabled():
backdoor_locals = {
'exit': _dont_use_this, # So we don't exit the entire process
'quit': _dont_use_this, # So we don't exit the entire process
'fo': _find_objects,
'pgt': _print_greenthreads,
'pnt': _print_nativethreads,
}
if CONF.backdoor_port is None:
return None
start_port, end_port = _parse_port_range(str(CONF.backdoor_port))
# NOTE(johannes): The standard sys.displayhook will print the value of
# the last expression and set it to __builtin__._, which overwrites
# the __builtin__._ that gettext sets. Let's switch to using pprint
# since it won't interact poorly with gettext, and it's easier to
# read the output too.
def displayhook(val):
if val is not None:
pprint.pprint(val)
sys.displayhook = displayhook
sock = _listen('localhost', start_port, end_port, eventlet.listen)
# In the case of backdoor port being zero, a port number is assigned by
# listen(). In any case, pull the port number out here.
port = sock.getsockname()[1]
LOG.info(_('Eventlet backdoor listening on %(port)s for process %(pid)d') %
{'port': port, 'pid': os.getpid()})
eventlet.spawn_n(eventlet.backdoor.backdoor_server, sock,
locals=backdoor_locals)
return port

139
billingstack/openstack/common/exception.py
View File

@@ -1,139 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Exceptions common to OpenStack projects
"""
import logging
from billingstack.openstack.common.gettextutils import _ # noqa
_FATAL_EXCEPTION_FORMAT_ERRORS = False
class Error(Exception):
def __init__(self, message=None):
super(Error, self).__init__(message)
class ApiError(Error):
def __init__(self, message='Unknown', code='Unknown'):
self.api_message = message
self.code = code
super(ApiError, self).__init__('%s: %s' % (code, message))
class NotFound(Error):
pass
class UnknownScheme(Error):
msg_fmt = "Unknown scheme '%s' found in URI"
def __init__(self, scheme):
msg = self.msg_fmt % scheme
super(UnknownScheme, self).__init__(msg)
class BadStoreUri(Error):
msg_fmt = "The Store URI %s was malformed. Reason: %s"
def __init__(self, uri, reason):
msg = self.msg_fmt % (uri, reason)
super(BadStoreUri, self).__init__(msg)
class Duplicate(Error):
pass
class NotAuthorized(Error):
pass
class NotEmpty(Error):
pass
class Invalid(Error):
pass
class BadInputError(Exception):
"""Error resulting from a client sending bad input to a server"""
pass
class MissingArgumentError(Error):
pass
class DatabaseMigrationError(Error):
pass
class ClientConnectionError(Exception):
"""Error resulting from a client connecting to a server"""
pass
def wrap_exception(f):
def _wrap(*args, **kw):
try:
return f(*args, **kw)
except Exception as e:
if not isinstance(e, Error):
logging.exception(_('Uncaught exception'))
raise Error(str(e))
raise
_wrap.func_name = f.func_name
return _wrap
class OpenstackException(Exception):
"""Base Exception class.
To correctly use this class, inherit from it and define
a 'msg_fmt' property. That message will get printf'd
with the keyword arguments provided to the constructor.
"""
msg_fmt = "An unknown exception occurred"
def __init__(self, **kwargs):
try:
self._error_string = self.msg_fmt % kwargs
except Exception:
if _FATAL_EXCEPTION_FORMAT_ERRORS:
raise
else:
# at least get the core message out if something happened
self._error_string = self.msg_fmt
def __str__(self):
return self._error_string
class MalformedRequestBody(OpenstackException):
msg_fmt = "Malformed message body: %(reason)s"
class InvalidContentType(OpenstackException):
msg_fmt = "Invalid content type %(content_type)s"

101
billingstack/openstack/common/excutils.py
View File

@@ -1,101 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack Foundation.
# Copyright 2012, Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Exception related utilities.
"""
import logging
import sys
import time
import traceback
import six
from billingstack.openstack.common.gettextutils import _ # noqa
class save_and_reraise_exception(object):
"""Save current exception, run some code and then re-raise.
In some cases the exception context can be cleared, resulting in None
being attempted to be re-raised after an exception handler is run. This
can happen when eventlet switches greenthreads or when running an
exception handler, code raises and catches an exception. In both
cases the exception context will be cleared.
To work around this, we save the exception state, run handler code, and
then re-raise the original exception. If another exception occurs, the
saved exception is logged and the new exception is re-raised.
In some cases the caller may not want to re-raise the exception, and
for those circumstances this context provides a reraise flag that
can be used to suppress the exception. For example:
except Exception:
with save_and_reraise_exception() as ctxt:
decide_if_need_reraise()
if not should_be_reraised:
ctxt.reraise = False
"""
def __init__(self):
self.reraise = True
def __enter__(self):
self.type_, self.value, self.tb, = sys.exc_info()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is not None:
logging.error(_('Original exception being dropped: %s'),
traceback.format_exception(self.type_,
self.value,
self.tb))
return False
if self.reraise:
six.reraise(self.type_, self.value, self.tb)
def forever_retry_uncaught_exceptions(infunc):
def inner_func(*args, **kwargs):
last_log_time = 0
last_exc_message = None
exc_count = 0
while True:
try:
return infunc(*args, **kwargs)
except Exception as exc:
this_exc_message = six.u(str(exc))
if this_exc_message == last_exc_message:
exc_count += 1
else:
exc_count = 1
# Do not log any more frequently than once a minute unless
# the exception message changes
cur_time = int(time.time())
if (cur_time - last_log_time > 60 or
this_exc_message != last_exc_message):
logging.exception(
_('Unexpected exception occurred %d time(s)... '
'retrying.') % exc_count)
last_log_time = cur_time
last_exc_message = this_exc_message
exc_count = 0
# This should be a very rare event. In case it isn't, do
# a sleep.
time.sleep(1)
return inner_func

139
billingstack/openstack/common/fileutils.py
View File

@@ -1,139 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import contextlib
import errno
import os
import tempfile
from billingstack.openstack.common import excutils
from billingstack.openstack.common.gettextutils import _ # noqa
from billingstack.openstack.common import log as logging
LOG = logging.getLogger(__name__)
_FILE_CACHE = {}
def ensure_tree(path):
"""Create a directory (and any ancestor directories required)
:param path: Directory to create
"""
try:
os.makedirs(path)
except OSError as exc:
if exc.errno == errno.EEXIST:
if not os.path.isdir(path):
raise
else:
raise
def read_cached_file(filename, force_reload=False):
"""Read from a file if it has been modified.
:param force_reload: Whether to reload the file.
:returns: A tuple with a boolean specifying if the data is fresh
or not.
"""
global _FILE_CACHE
if force_reload and filename in _FILE_CACHE:
del _FILE_CACHE[filename]
reloaded = False
mtime = os.path.getmtime(filename)
cache_info = _FILE_CACHE.setdefault(filename, {})
if not cache_info or mtime > cache_info.get('mtime', 0):
LOG.debug(_("Reloading cached file %s") % filename)
with open(filename) as fap:
cache_info['data'] = fap.read()
cache_info['mtime'] = mtime
reloaded = True
return (reloaded, cache_info['data'])
def delete_if_exists(path, remove=os.unlink):
"""Delete a file, but ignore file not found error.
:param path: File to delete
:param remove: Optional function to remove passed path
"""
try:
remove(path)
except OSError as e:
if e.errno != errno.ENOENT:
raise
@contextlib.contextmanager
def remove_path_on_error(path, remove=delete_if_exists):
"""Protect code that wants to operate on PATH atomically.
Any exception will cause PATH to be removed.
:param path: File to work with
:param remove: Optional function to remove passed path
"""
try:
yield
except Exception:
with excutils.save_and_reraise_exception():
remove(path)
def file_open(*args, **kwargs):
"""Open file
see built-in file() documentation for more details
Note: The reason this is kept in a separate module is to easily
be able to provide a stub module that doesn't alter system
state at all (for unit tests)
"""
return file(*args, **kwargs)
def write_to_tempfile(content, path=None, suffix='', prefix='tmp'):
"""Create temporary file or use existing file.
This util is needed for creating temporary file with
specified content, suffix and prefix. If path is not None,
it will be used for writing content. If the path doesn't
exist it'll be created.
:param content: content for temporary file.
:param path: same as parameter 'dir' for mkstemp
:param suffix: same as parameter 'suffix' for mkstemp
:param prefix: same as parameter 'prefix' for mkstemp
For example: it can be used in database tests for creating
configuration files.
"""
if path:
ensure_tree(path)
(fd, path) = tempfile.mkstemp(suffix=suffix, dir=path, prefix=prefix)
try:
os.write(fd, content)
finally:
os.close(fd)
return path

373
billingstack/openstack/common/gettextutils.py
View File

@@ -1,373 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2012 Red Hat, Inc.
# Copyright 2013 IBM Corp.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
gettext for openstack-common modules.
Usual usage in an openstack.common module:
from billingstack.openstack.common.gettextutils import _
"""
import copy
import gettext
import logging
import os
import re
try:
import UserString as _userString
except ImportError:
import collections as _userString
from babel import localedata
import six
_localedir = os.environ.get('billingstack'.upper() + '_LOCALEDIR')
_t = gettext.translation('billingstack', localedir=_localedir, fallback=True)
_AVAILABLE_LANGUAGES = {}
USE_LAZY = False
def enable_lazy():
"""Convenience function for configuring _() to use lazy gettext
Call this at the start of execution to enable the gettextutils._
function to use lazy gettext functionality. This is useful if
your project is importing _ directly instead of using the
gettextutils.install() way of importing the _ function.
"""
global USE_LAZY
USE_LAZY = True
def _(msg):
if USE_LAZY:
return Message(msg, 'billingstack')
else:
if six.PY3:
return _t.gettext(msg)
return _t.ugettext(msg)
def install(domain, lazy=False):
"""Install a _() function using the given translation domain.
Given a translation domain, install a _() function using gettext's
install() function.
The main difference from gettext.install() is that we allow
overriding the default localedir (e.g. /usr/share/locale) using
a translation-domain-specific environment variable (e.g.
NOVA_LOCALEDIR).
:param domain: the translation domain
:param lazy: indicates whether or not to install the lazy _() function.
The lazy _() introduces a way to do deferred translation
of messages by installing a _ that builds Message objects,
instead of strings, which can then be lazily translated into
any available locale.
"""
if lazy:
# NOTE(mrodden): Lazy gettext functionality.
#
# The following introduces a deferred way to do translations on
# messages in OpenStack. We override the standard _() function
# and % (format string) operation to build Message objects that can
# later be translated when we have more information.
#
# Also included below is an example LocaleHandler that translates
# Messages to an associated locale, effectively allowing many logs,
# each with their own locale.
def _lazy_gettext(msg):
"""Create and return a Message object.
Lazy gettext function for a given domain, it is a factory method
for a project/module to get a lazy gettext function for its own
translation domain (i.e. nova, glance, cinder, etc.)
Message encapsulates a string so that we can translate
it later when needed.
"""
return Message(msg, domain)
from six import moves
moves.builtins.__dict__['_'] = _lazy_gettext
else:
localedir = '%s_LOCALEDIR' % domain.upper()
if six.PY3:
gettext.install(domain,
localedir=os.environ.get(localedir))
else:
gettext.install(domain,
localedir=os.environ.get(localedir),
unicode=True)
class Message(_userString.UserString, object):
"""Class used to encapsulate translatable messages."""
def __init__(self, msg, domain):
# _msg is the gettext msgid and should never change
self._msg = msg
self._left_extra_msg = ''
self._right_extra_msg = ''
self._locale = None
self.params = None
self.domain = domain
@property
def data(self):
# NOTE(mrodden): this should always resolve to a unicode string
# that best represents the state of the message currently
localedir = os.environ.get(self.domain.upper() + '_LOCALEDIR')
if self.locale:
lang = gettext.translation(self.domain,
localedir=localedir,
languages=[self.locale],
fallback=True)
else:
# use system locale for translations
lang = gettext.translation(self.domain,
localedir=localedir,
fallback=True)
if six.PY3:
ugettext = lang.gettext
else:
ugettext = lang.ugettext
full_msg = (self._left_extra_msg +
ugettext(self._msg) +
self._right_extra_msg)
if self.params is not None:
full_msg = full_msg % self.params
return six.text_type(full_msg)
@property
def locale(self):
return self._locale
@locale.setter
def locale(self, value):
self._locale = value
if not self.params:
return
# This Message object may have been constructed with one or more
# Message objects as substitution parameters, given as a single
# Message, or a tuple or Map containing some, so when setting the
# locale for this Message we need to set it for those Messages too.
if isinstance(self.params, Message):
self.params.locale = value
return
if isinstance(self.params, tuple):
for param in self.params:
if isinstance(param, Message):
param.locale = value
return
if isinstance(self.params, dict):
for param in self.params.values():
if isinstance(param, Message):
param.locale = value
def _save_dictionary_parameter(self, dict_param):
full_msg = self.data
# look for %(blah) fields in string;
# ignore %% and deal with the
# case where % is first character on the line
keys = re.findall('(?:[^%]|^)?%\((\w*)\)[a-z]', full_msg)
# if we don't find any %(blah) blocks but have a %s
if not keys and re.findall('(?:[^%]|^)%[a-z]', full_msg):
# apparently the full dictionary is the parameter
params = copy.deepcopy(dict_param)
else:
params = {}
for key in keys:
try:
params[key] = copy.deepcopy(dict_param[key])
except TypeError:
# cast uncopyable thing to unicode string
params[key] = six.text_type(dict_param[key])
return params
def _save_parameters(self, other):
# we check for None later to see if
# we actually have parameters to inject,
# so encapsulate if our parameter is actually None
if other is None:
self.params = (other, )
elif isinstance(other, dict):
self.params = self._save_dictionary_parameter(other)
else:
# fallback to casting to unicode,
# this will handle the problematic python code-like
# objects that cannot be deep-copied
try:
self.params = copy.deepcopy(other)
except TypeError:
self.params = six.text_type(other)
return self
# overrides to be more string-like
def __unicode__(self):
return self.data
def __str__(self):
if six.PY3:
return self.__unicode__()
return self.data.encode('utf-8')
def __getstate__(self):
to_copy = ['_msg', '_right_extra_msg', '_left_extra_msg',
'domain', 'params', '_locale']
new_dict = self.__dict__.fromkeys(to_copy)
for attr in to_copy:
new_dict[attr] = copy.deepcopy(self.__dict__[attr])
return new_dict
def __setstate__(self, state):
for (k, v) in state.items():
setattr(self, k, v)
# operator overloads
def __add__(self, other):
copied = copy.deepcopy(self)
copied._right_extra_msg += other.__str__()
return copied
def __radd__(self, other):
copied = copy.deepcopy(self)
copied._left_extra_msg += other.__str__()
return copied
def __mod__(self, other):
# do a format string to catch and raise
# any possible KeyErrors from missing parameters
self.data % other
copied = copy.deepcopy(self)
return copied._save_parameters(other)
def __mul__(self, other):
return self.data * other
def __rmul__(self, other):
return other * self.data
def __getitem__(self, key):
return self.data[key]
def __getslice__(self, start, end):
return self.data.__getslice__(start, end)
def __getattribute__(self, name):
# NOTE(mrodden): handle lossy operations that we can't deal with yet
# These override the UserString implementation, since UserString
# uses our __class__ attribute to try and build a new message
# after running the inner data string through the operation.
# At that point, we have lost the gettext message id and can just
# safely resolve to a string instead.
ops = ['capitalize', 'center', 'decode', 'encode',
'expandtabs', 'ljust', 'lstrip', 'replace', 'rjust', 'rstrip',
'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
if name in ops:
return getattr(self.data, name)
else:
return _userString.UserString.__getattribute__(self, name)
def get_available_languages(domain):
"""Lists the available languages for the given translation domain.
:param domain: the domain to get languages for
"""
if domain in _AVAILABLE_LANGUAGES:
return copy.copy(_AVAILABLE_LANGUAGES[domain])
localedir = '%s_LOCALEDIR' % domain.upper()
find = lambda x: gettext.find(domain,
localedir=os.environ.get(localedir),
languages=[x])
# NOTE(mrodden): en_US should always be available (and first in case
# order matters) since our in-line message strings are en_US
language_list = ['en_US']
# NOTE(luisg): Babel <1.0 used a function called list(), which was
# renamed to locale_identifiers() in >=1.0, the requirements master list
# requires >=0.9.6, uncapped, so defensively work with both. We can remove
# this check when the master list updates to >=1.0, and update all projects
list_identifiers = (getattr(localedata, 'list', None) or
getattr(localedata, 'locale_identifiers'))
locale_identifiers = list_identifiers()
for i in locale_identifiers:
if find(i) is not None:
language_list.append(i)
_AVAILABLE_LANGUAGES[domain] = language_list
return copy.copy(language_list)
def get_localized_message(message, user_locale):
"""Gets a localized version of the given message in the given locale.
If the message is not a Message object the message is returned as-is.
If the locale is None the message is translated to the default locale.
:returns: the translated message in unicode, or the original message if
it could not be translated
"""
translated = message
if isinstance(message, Message):
original_locale = message.locale
message.locale = user_locale
translated = six.text_type(message)
message.locale = original_locale
return translated
class LocaleHandler(logging.Handler):
"""Handler that can have a locale associated to translate Messages.
A quick example of how to utilize the Message class above.
LocaleHandler takes a locale and a target logging.Handler object
to forward LogRecord objects to after translating the internal Message.
"""
def __init__(self, locale, target):
"""Initialize a LocaleHandler
:param locale: locale to use for translating messages
:param target: logging.Handler object to forward
LogRecord objects to after translation
"""
logging.Handler.__init__(self)
self.locale = locale
self.target = target
def emit(self, record):
if isinstance(record.msg, Message):
# set the locale and resolve to a string
record.msg.locale = self.locale
self.target.emit(record)

68
billingstack/openstack/common/importutils.py
View File

@@ -1,68 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Import related utilities and helper functions.
"""
import sys
import traceback
def import_class(import_str):
"""Returns a class from a string including module and class."""
mod_str, _sep, class_str = import_str.rpartition('.')
try:
__import__(mod_str)
return getattr(sys.modules[mod_str], class_str)
except (ValueError, AttributeError):
raise ImportError('Class %s cannot be found (%s)' %
(class_str,
traceback.format_exception(*sys.exc_info())))
def import_object(import_str, *args, **kwargs):
"""Import a class and return an instance of it."""
return import_class(import_str)(*args, **kwargs)
def import_object_ns(name_space, import_str, *args, **kwargs):
"""Tries to import object from default namespace.
Imports a class and return an instance of it, first by trying
to find the class in a default namespace, then failing back to
a full path if not found in the default namespace.
"""
import_value = "%s.%s" % (name_space, import_str)
try:
return import_class(import_value)(*args, **kwargs)
except ImportError:
return import_class(import_str)(*args, **kwargs)
def import_module(import_str):
"""Import a module."""
__import__(import_str)
return sys.modules[import_str]
def try_import(import_str, default=None):
"""Try to import a module and if it fails return default."""
try:
return import_module(import_str)
except ImportError:
return default

130
billingstack/openstack/common/iniparser.py
View File

@@ -1,130 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2012 OpenStack LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
class ParseError(Exception):
def __init__(self, message, lineno, line):
self.msg = message
self.line = line
self.lineno = lineno
def __str__(self):
return 'at line %d, %s: %r' % (self.lineno, self.msg, self.line)
class BaseParser(object):
lineno = 0
parse_exc = ParseError
def _assignment(self, key, value):
self.assignment(key, value)
return None, []
def _get_section(self, line):
if line[-1] != ']':
return self.error_no_section_end_bracket(line)
if len(line) <= 2:
return self.error_no_section_name(line)
return line[1:-1]
def _split_key_value(self, line):
colon = line.find(':')
equal = line.find('=')
if colon < 0 and equal < 0:
return self.error_invalid_assignment(line)
if colon < 0 or (equal >= 0 and equal < colon):
key, value = line[:equal], line[equal + 1:]
else:
key, value = line[:colon], line[colon + 1:]
value = value.strip()
if ((value and value[0] == value[-1]) and
(value[0] == "\"" or value[0] == "'")):
value = value[1:-1]
return key.strip(), [value]
def parse(self, lineiter):
key = None
value = []
for line in lineiter:
self.lineno += 1
line = line.rstrip()
if not line:
# Blank line, ends multi-line values
if key:
key, value = self._assignment(key, value)
continue
elif line[0] in (' ', '\t'):
# Continuation of previous assignment
if key is None:
self.error_unexpected_continuation(line)
else:
value.append(line.lstrip())
continue
if key:
# Flush previous assignment, if any
key, value = self._assignment(key, value)
if line[0] == '[':
# Section start
section = self._get_section(line)
if section:
self.new_section(section)
elif line[0] in '#;':
self.comment(line[1:].lstrip())
else:
key, value = self._split_key_value(line)
if not key:
return self.error_empty_key(line)
if key:
# Flush previous assignment, if any
self._assignment(key, value)
def assignment(self, key, value):
"""Called when a full assignment is parsed"""
raise NotImplementedError()
def new_section(self, section):
"""Called when a new section is started"""
raise NotImplementedError()
def comment(self, comment):
"""Called when a comment is parsed"""
pass
def error_invalid_assignment(self, line):
raise self.parse_exc("No ':' or '=' found in assignment",
self.lineno, line)
def error_empty_key(self, line):
raise self.parse_exc('Key cannot be empty', self.lineno, line)
def error_unexpected_continuation(self, line):
raise self.parse_exc('Unexpected continuation line',
self.lineno, line)
def error_no_section_end_bracket(self, line):
raise self.parse_exc('Invalid section (must end with ])',
self.lineno, line)
def error_no_section_name(self, line):
raise self.parse_exc('Empty section name', self.lineno, line)

180
billingstack/openstack/common/jsonutils.py
View File

@@ -1,180 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# Copyright 2011 Justin Santa Barbara
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
'''
JSON related utilities.
This module provides a few things:
1) A handy function for getting an object down to something that can be
JSON serialized. See to_primitive().
2) Wrappers around loads() and dumps(). The dumps() wrapper will
automatically use to_primitive() for you if needed.
3) This sets up anyjson to use the loads() and dumps() wrappers if anyjson
is available.
'''
import datetime
import functools
import inspect
import itertools
import json
try:
import xmlrpclib
except ImportError:
# NOTE(jd): xmlrpclib is not shipped with Python 3
xmlrpclib = None
import six
from billingstack.openstack.common import gettextutils
from billingstack.openstack.common import importutils
from billingstack.openstack.common import timeutils
netaddr = importutils.try_import("netaddr")
_nasty_type_tests = [inspect.ismodule, inspect.isclass, inspect.ismethod,
inspect.isfunction, inspect.isgeneratorfunction,
inspect.isgenerator, inspect.istraceback, inspect.isframe,
inspect.iscode, inspect.isbuiltin, inspect.isroutine,
inspect.isabstract]
_simple_types = (six.string_types + six.integer_types
+ (type(None), bool, float))
def to_primitive(value, convert_instances=False, convert_datetime=True,
level=0, max_depth=3):
"""Convert a complex object into primitives.
Handy for JSON serialization. We can optionally handle instances,
but since this is a recursive function, we could have cyclical
data structures.
To handle cyclical data structures we could track the actual objects
visited in a set, but not all objects are hashable. Instead we just
track the depth of the object inspections and don't go too deep.
Therefore, convert_instances=True is lossy ... be aware.
"""
# handle obvious types first - order of basic types determined by running
# full tests on nova project, resulting in the following counts:
# 572754 <type 'NoneType'>
# 460353 <type 'int'>
# 379632 <type 'unicode'>
# 274610 <type 'str'>
# 199918 <type 'dict'>
# 114200 <type 'datetime.datetime'>
# 51817 <type 'bool'>
# 26164 <type 'list'>
# 6491 <type 'float'>
# 283 <type 'tuple'>
# 19 <type 'long'>
if isinstance(value, _simple_types):
return value
if isinstance(value, datetime.datetime):
if convert_datetime:
return timeutils.strtime(value)
else:
return value
# value of itertools.count doesn't get caught by nasty_type_tests
# and results in infinite loop when list(value) is called.
if type(value) == itertools.count:
return six.text_type(value)
# FIXME(vish): Workaround for LP bug 852095. Without this workaround,
# tests that raise an exception in a mocked method that
# has a @wrap_exception with a notifier will fail. If
# we up the dependency to 0.5.4 (when it is released) we
# can remove this workaround.
if getattr(value, '__module__', None) == 'mox':
return 'mock'
if level > max_depth:
return '?'
# The try block may not be necessary after the class check above,
# but just in case ...
try:
recursive = functools.partial(to_primitive,
convert_instances=convert_instances,
convert_datetime=convert_datetime,
level=level,
max_depth=max_depth)
if isinstance(value, dict):
return dict((k, recursive(v)) for k, v in value.iteritems())
elif isinstance(value, (list, tuple)):
return [recursive(lv) for lv in value]
# It's not clear why xmlrpclib created their own DateTime type, but
# for our purposes, make it a datetime type which is explicitly
# handled
if xmlrpclib and isinstance(value, xmlrpclib.DateTime):
value = datetime.datetime(*tuple(value.timetuple())[:6])
if convert_datetime and isinstance(value, datetime.datetime):
return timeutils.strtime(value)
elif isinstance(value, gettextutils.Message):
return value.data
elif hasattr(value, 'iteritems'):
return recursive(dict(value.iteritems()), level=level + 1)
elif hasattr(value, '__iter__'):
return recursive(list(value))
elif convert_instances and hasattr(value, '__dict__'):
# Likely an instance of something. Watch for cycles.
# Ignore class member vars.
return recursive(value.__dict__, level=level + 1)
elif netaddr and isinstance(value, netaddr.IPAddress):
return six.text_type(value)
else:
if any(test(value) for test in _nasty_type_tests):
return six.text_type(value)
return value
except TypeError:
# Class objects are tricky since they may define something like
# __iter__ defined but it isn't callable as list().
return six.text_type(value)
def dumps(value, default=to_primitive, **kwargs):
return json.dumps(value, default=default, **kwargs)
def loads(s):
return json.loads(s)
def load(s):
return json.load(s)
try:
import anyjson
except ImportError:
pass
else:
anyjson._modules.append((__name__, 'dumps', TypeError,
'loads', ValueError, 'load'))
anyjson.force_implementation(__name__)

47
billingstack/openstack/common/local.py
View File

@@ -1,47 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Local storage of variables using weak references"""
import threading
import weakref
class WeakLocal(threading.local):
def __getattribute__(self, attr):
rval = super(WeakLocal, self).__getattribute__(attr)
if rval:
# NOTE(mikal): this bit is confusing. What is stored is a weak
# reference, not the value itself. We therefore need to lookup
# the weak reference and return the inner value here.
rval = rval()
return rval
def __setattr__(self, attr, value):
value = weakref.ref(value)
return super(WeakLocal, self).__setattr__(attr, value)
# NOTE(mikal): the name "store" should be deprecated in the future
store = WeakLocal()
# A "weak" store uses weak references and allows an object to fall out of scope
# when it falls out of scope in the code that uses the thread local storage. A
# "strong" store will hold a reference to the object so that it never falls out
# of scope.
weak_store = WeakLocal()
strong_store = threading.local()

305
billingstack/openstack/common/lockutils.py
View File

@@ -1,305 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import contextlib
import errno
import functools
import os
import shutil
import subprocess
import sys
import tempfile
import threading
import time
import weakref
from oslo.config import cfg
from billingstack.openstack.common import fileutils
from billingstack.openstack.common.gettextutils import _ # noqa
from billingstack.openstack.common import local
from billingstack.openstack.common import log as logging
LOG = logging.getLogger(__name__)
util_opts = [
cfg.BoolOpt('disable_process_locking', default=False,
help='Whether to disable inter-process locks'),
cfg.StrOpt('lock_path',
default=os.environ.get("BILLINGSTACK_LOCK_PATH"),
help=('Directory to use for lock files.'))
]
CONF = cfg.CONF
CONF.register_opts(util_opts)
def set_defaults(lock_path):
cfg.set_defaults(util_opts, lock_path=lock_path)
class _InterProcessLock(object):
"""Lock implementation which allows multiple locks, working around
issues like bugs.debian.org/cgi-bin/bugreport.cgi?bug=632857 and does
not require any cleanup. Since the lock is always held on a file
descriptor rather than outside of the process, the lock gets dropped
automatically if the process crashes, even if __exit__ is not executed.
There are no guarantees regarding usage by multiple green threads in a
single process here. This lock works only between processes. Exclusive
access between local threads should be achieved using the semaphores
in the @synchronized decorator.
Note these locks are released when the descriptor is closed, so it's not
safe to close the file descriptor while another green thread holds the
lock. Just opening and closing the lock file can break synchronisation,
so lock files must be accessed only using this abstraction.
"""
def __init__(self, name):
self.lockfile = None
self.fname = name
def __enter__(self):
self.lockfile = open(self.fname, 'w')
while True:
try:
# Using non-blocking locks since green threads are not
# patched to deal with blocking locking calls.
# Also upon reading the MSDN docs for locking(), it seems
# to have a laughable 10 attempts "blocking" mechanism.
self.trylock()
return self
except IOError as e:
if e.errno in (errno.EACCES, errno.EAGAIN):
# external locks synchronise things like iptables
# updates - give it some time to prevent busy spinning
time.sleep(0.01)
else:
raise
def __exit__(self, exc_type, exc_val, exc_tb):
try:
self.unlock()
self.lockfile.close()
except IOError:
LOG.exception(_("Could not release the acquired lock `%s`"),
self.fname)
def trylock(self):
raise NotImplementedError()
def unlock(self):
raise NotImplementedError()
class _WindowsLock(_InterProcessLock):
def trylock(self):
msvcrt.locking(self.lockfile.fileno(), msvcrt.LK_NBLCK, 1)
def unlock(self):
msvcrt.locking(self.lockfile.fileno(), msvcrt.LK_UNLCK, 1)
class _PosixLock(_InterProcessLock):
def trylock(self):
fcntl.lockf(self.lockfile, fcntl.LOCK_EX | fcntl.LOCK_NB)
def unlock(self):
fcntl.lockf(self.lockfile, fcntl.LOCK_UN)
if os.name == 'nt':
import msvcrt
InterProcessLock = _WindowsLock
else:
import fcntl
InterProcessLock = _PosixLock
_semaphores = weakref.WeakValueDictionary()
_semaphores_lock = threading.Lock()
@contextlib.contextmanager
def lock(name, lock_file_prefix=None, external=False, lock_path=None):
"""Context based lock
This function yields a `threading.Semaphore` instance (if we don't use
eventlet.monkey_patch(), else `semaphore.Semaphore`) unless external is
True, in which case, it'll yield an InterProcessLock instance.
:param lock_file_prefix: The lock_file_prefix argument is used to provide
lock files on disk with a meaningful prefix.
:param external: The external keyword argument denotes whether this lock
should work across multiple processes. This means that if two different
workers both run a a method decorated with @synchronized('mylock',
external=True), only one of them will execute at a time.
:param lock_path: The lock_path keyword argument is used to specify a
special location for external lock files to live. If nothing is set, then
CONF.lock_path is used as a default.
"""
with _semaphores_lock:
try:
sem = _semaphores[name]
except KeyError:
sem = threading.Semaphore()
_semaphores[name] = sem
with sem:
LOG.debug(_('Got semaphore "%(lock)s"'), {'lock': name})
# NOTE(mikal): I know this looks odd
if not hasattr(local.strong_store, 'locks_held'):
local.strong_store.locks_held = []
local.strong_store.locks_held.append(name)
try:
if external and not CONF.disable_process_locking:
LOG.debug(_('Attempting to grab file lock "%(lock)s"'),
{'lock': name})
# We need a copy of lock_path because it is non-local
local_lock_path = lock_path or CONF.lock_path
if not local_lock_path:
raise cfg.RequiredOptError('lock_path')
if not os.path.exists(local_lock_path):
fileutils.ensure_tree(local_lock_path)
LOG.info(_('Created lock path: %s'), local_lock_path)
def add_prefix(name, prefix):
if not prefix:
return name
sep = '' if prefix.endswith('-') else '-'
return '%s%s%s' % (prefix, sep, name)
# NOTE(mikal): the lock name cannot contain directory
# separators
lock_file_name = add_prefix(name.replace(os.sep, '_'),
lock_file_prefix)
lock_file_path = os.path.join(local_lock_path, lock_file_name)
try:
lock = InterProcessLock(lock_file_path)
with lock as lock:
LOG.debug(_('Got file lock "%(lock)s" at %(path)s'),
{'lock': name, 'path': lock_file_path})
yield lock
finally:
LOG.debug(_('Released file lock "%(lock)s" at %(path)s'),
{'lock': name, 'path': lock_file_path})
else:
yield sem
finally:
local.strong_store.locks_held.remove(name)
def synchronized(name, lock_file_prefix=None, external=False, lock_path=None):
"""Synchronization decorator.
Decorating a method like so::
@synchronized('mylock')
def foo(self, *args):
...
ensures that only one thread will execute the foo method at a time.
Different methods can share the same lock::
@synchronized('mylock')
def foo(self, *args):
...
@synchronized('mylock')
def bar(self, *args):
...
This way only one of either foo or bar can be executing at a time.
"""
def wrap(f):
@functools.wraps(f)
def inner(*args, **kwargs):
try:
with lock(name, lock_file_prefix, external, lock_path):
LOG.debug(_('Got semaphore / lock "%(function)s"'),
{'function': f.__name__})
return f(*args, **kwargs)
finally:
LOG.debug(_('Semaphore / lock released "%(function)s"'),
{'function': f.__name__})
return inner
return wrap
def synchronized_with_prefix(lock_file_prefix):
"""Partial object generator for the synchronization decorator.
Redefine @synchronized in each project like so::
(in nova/utils.py)
from nova.openstack.common import lockutils
synchronized = lockutils.synchronized_with_prefix('nova-')
(in nova/foo.py)
from nova import utils
@utils.synchronized('mylock')
def bar(self, *args):
...
The lock_file_prefix argument is used to provide lock files on disk with a
meaningful prefix.
"""
return functools.partial(synchronized, lock_file_prefix=lock_file_prefix)
def main(argv):
"""Create a dir for locks and pass it to command from arguments
If you run this:
python -m openstack.common.lockutils python setup.py testr <etc>
a temporary directory will be created for all your locks and passed to all
your tests in an environment variable. The temporary dir will be deleted
afterwards and the return value will be preserved.
"""
lock_dir = tempfile.mkdtemp()
os.environ["BILLINGSTACK_LOCK_PATH"] = lock_dir
try:
ret_val = subprocess.call(argv[1:])
finally:
shutil.rmtree(lock_dir, ignore_errors=True)
return ret_val
if __name__ == '__main__':
sys.exit(main(sys.argv))

626
billingstack/openstack/common/log.py
View File

@@ -1,626 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 OpenStack Foundation.
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Openstack logging handler.
This module adds to logging functionality by adding the option to specify
a context object when calling the various log methods. If the context object
is not specified, default formatting is used. Additionally, an instance uuid
may be passed as part of the log message, which is intended to make it easier
for admins to find messages related to a specific instance.
It also allows setting of formatting information through conf.
"""
import inspect
import itertools
import logging
import logging.config
import logging.handlers
import os
import re
import sys
import traceback
from oslo.config import cfg
import six
from six import moves
from billingstack.openstack.common.gettextutils import _ # noqa
from billingstack.openstack.common import importutils
from billingstack.openstack.common import jsonutils
from billingstack.openstack.common import local
_DEFAULT_LOG_DATE_FORMAT = "%Y-%m-%d %H:%M:%S"
_SANITIZE_KEYS = ['adminPass', 'admin_pass', 'password']
# NOTE(ldbragst): Let's build a list of regex objects using the list of
# _SANITIZE_KEYS we already have. This way, we only have to add the new key
# to the list of _SANITIZE_KEYS and we can generate regular expressions
# for XML and JSON automatically.
_SANITIZE_PATTERNS = []
_FORMAT_PATTERNS = [r'(%(key)s\s*[=]\s*[\"\']).*?([\"\'])',
r'(<%(key)s>).*?(</%(key)s>)',
r'([\"\']%(key)s[\"\']\s*:\s*[\"\']).*?([\"\'])',
r'([\'"].*?%(key)s[\'"]\s*:\s*u?[\'"]).*?([\'"])']
for key in _SANITIZE_KEYS:
for pattern in _FORMAT_PATTERNS:
reg_ex = re.compile(pattern % {'key': key}, re.DOTALL)
_SANITIZE_PATTERNS.append(reg_ex)
common_cli_opts = [
cfg.BoolOpt('debug',
short='d',
default=False,
help='Print debugging output (set logging level to '
'DEBUG instead of default WARNING level).'),
cfg.BoolOpt('verbose',
short='v',
default=False,
help='Print more verbose output (set logging level to '
'INFO instead of default WARNING level).'),
]
logging_cli_opts = [
cfg.StrOpt('log-config-append',
metavar='PATH',
deprecated_name='log-config',
help='The name of logging configuration file. It does not '
'disable existing loggers, but just appends specified '
'logging configuration to any other existing logging '
'options. Please see the Python logging module '
'documentation for details on logging configuration '
'files.'),
cfg.StrOpt('log-format',
default=None,
metavar='FORMAT',
help='DEPRECATED. '
'A logging.Formatter log message format string which may '
'use any of the available logging.LogRecord attributes. '
'This option is deprecated. Please use '
'logging_context_format_string and '
'logging_default_format_string instead.'),
cfg.StrOpt('log-date-format',
default=_DEFAULT_LOG_DATE_FORMAT,
metavar='DATE_FORMAT',
help='Format string for %%(asctime)s in log records. '
'Default: %(default)s'),
cfg.StrOpt('log-file',
metavar='PATH',
deprecated_name='logfile',
help='(Optional) Name of log file to output to. '
'If no default is set, logging will go to stdout.'),
cfg.StrOpt('log-dir',
deprecated_name='logdir',
help='(Optional) The base directory used for relative '
'--log-file paths'),
cfg.BoolOpt('use-syslog',
default=False,
help='Use syslog for logging.'),
cfg.StrOpt('syslog-log-facility',
default='LOG_USER',
help='syslog facility to receive log lines')
]
generic_log_opts = [
cfg.BoolOpt('use_stderr',
default=True,
help='Log output to standard error')
]
log_opts = [
cfg.StrOpt('logging_context_format_string',
default='%(asctime)s.%(msecs)03d %(process)d %(levelname)s '
'%(name)s [%(request_id)s %(user)s %(tenant)s] '
'%(instance)s%(message)s',
help='format string to use for log messages with context'),
cfg.StrOpt('logging_default_format_string',
default='%(asctime)s.%(msecs)03d %(process)d %(levelname)s '
'%(name)s [-] %(instance)s%(message)s',
help='format string to use for log messages without context'),
cfg.StrOpt('logging_debug_format_suffix',
default='%(funcName)s %(pathname)s:%(lineno)d',
help='data to append to log format when level is DEBUG'),
cfg.StrOpt('logging_exception_prefix',
default='%(asctime)s.%(msecs)03d %(process)d TRACE %(name)s '
'%(instance)s',
help='prefix each line of exception output with this format'),
cfg.ListOpt('default_log_levels',
default=[
'amqp=WARN',
'amqplib=WARN',
'boto=WARN',
'keystone=INFO',
'qpid=WARN',
'sqlalchemy=WARN',
'suds=INFO',
'iso8601=WARN',
],
help='list of logger=LEVEL pairs'),
cfg.BoolOpt('publish_errors',
default=False,
help='publish error events'),
cfg.BoolOpt('fatal_deprecations',
default=False,
help='make deprecations fatal'),
# NOTE(mikal): there are two options here because sometimes we are handed
# a full instance (and could include more information), and other times we
# are just handed a UUID for the instance.
cfg.StrOpt('instance_format',
default='[instance: %(uuid)s] ',
help='If an instance is passed with the log message, format '
'it like this'),
cfg.StrOpt('instance_uuid_format',
default='[instance: %(uuid)s] ',
help='If an instance UUID is passed with the log message, '
'format it like this'),
]
CONF = cfg.CONF
CONF.register_cli_opts(common_cli_opts)
CONF.register_cli_opts(logging_cli_opts)
CONF.register_opts(generic_log_opts)
CONF.register_opts(log_opts)
# our new audit level
# NOTE(jkoelker) Since we synthesized an audit level, make the logging
# module aware of it so it acts like other levels.
logging.AUDIT = logging.INFO + 1
logging.addLevelName(logging.AUDIT, 'AUDIT')
try:
NullHandler = logging.NullHandler
except AttributeError: # NOTE(jkoelker) NullHandler added in Python 2.7
class NullHandler(logging.Handler):
def handle(self, record):
pass
def emit(self, record):
pass
def createLock(self):
self.lock = None
def _dictify_context(context):
if context is None:
return None
if not isinstance(context, dict) and getattr(context, 'to_dict', None):
context = context.to_dict()
return context
def _get_binary_name():
return os.path.basename(inspect.stack()[-1][1])
def _get_log_file_path(binary=None):
logfile = CONF.log_file
logdir = CONF.log_dir
if logfile and not logdir:
return logfile
if logfile and logdir:
return os.path.join(logdir, logfile)
if logdir:
binary = binary or _get_binary_name()
return '%s.log' % (os.path.join(logdir, binary),)
return None
def mask_password(message, secret="***"):
"""Replace password with 'secret' in message.
:param message: The string which includes security information.
:param secret: value with which to replace passwords, defaults to "***".
:returns: The unicode value of message with the password fields masked.
For example:
>>> mask_password("'adminPass' : 'aaaaa'")
"'adminPass' : '***'"
>>> mask_password("'admin_pass' : 'aaaaa'")
"'admin_pass' : '***'"
>>> mask_password('"password" : "aaaaa"')
'"password" : "***"'
>>> mask_password("'original_password' : 'aaaaa'")
"'original_password' : '***'"
>>> mask_password("u'original_password' : u'aaaaa'")
"u'original_password' : u'***'"
"""
message = six.text_type(message)
# NOTE(ldbragst): Check to see if anything in message contains any key
# specified in _SANITIZE_KEYS, if not then just return the message since
# we don't have to mask any passwords.
if not any(key in message for key in _SANITIZE_KEYS):
return message
secret = r'\g<1>' + secret + r'\g<2>'
for pattern in _SANITIZE_PATTERNS:
message = re.sub(pattern, secret, message)
return message
class BaseLoggerAdapter(logging.LoggerAdapter):
def audit(self, msg, *args, **kwargs):
self.log(logging.AUDIT, msg, *args, **kwargs)
class LazyAdapter(BaseLoggerAdapter):
def __init__(self, name='unknown', version='unknown'):
self._logger = None
self.extra = {}
self.name = name
self.version = version
@property
def logger(self):
if not self._logger:
self._logger = getLogger(self.name, self.version)
return self._logger
class ContextAdapter(BaseLoggerAdapter):
warn = logging.LoggerAdapter.warning
def __init__(self, logger, project_name, version_string):
self.logger = logger
self.project = project_name
self.version = version_string
@property
def handlers(self):
return self.logger.handlers
def deprecated(self, msg, *args, **kwargs):
stdmsg = _("Deprecated: %s") % msg
if CONF.fatal_deprecations:
self.critical(stdmsg, *args, **kwargs)
raise DeprecatedConfig(msg=stdmsg)
else:
self.warn(stdmsg, *args, **kwargs)
def process(self, msg, kwargs):
# NOTE(mrodden): catch any Message/other object and
# coerce to unicode before they can get
# to the python logging and possibly
# cause string encoding trouble
if not isinstance(msg, six.string_types):
msg = six.text_type(msg)
if 'extra' not in kwargs:
kwargs['extra'] = {}
extra = kwargs['extra']
context = kwargs.pop('context', None)
if not context:
context = getattr(local.store, 'context', None)
if context:
extra.update(_dictify_context(context))
instance = kwargs.pop('instance', None)
instance_uuid = (extra.get('instance_uuid', None) or
kwargs.pop('instance_uuid', None))
instance_extra = ''
if instance:
instance_extra = CONF.instance_format % instance
elif instance_uuid:
instance_extra = (CONF.instance_uuid_format
% {'uuid': instance_uuid})
extra.update({'instance': instance_extra})
extra.update({"project": self.project})
extra.update({"version": self.version})
extra['extra'] = extra.copy()
return msg, kwargs
class JSONFormatter(logging.Formatter):
def __init__(self, fmt=None, datefmt=None):
# NOTE(jkoelker) we ignore the fmt argument, but its still there
# since logging.config.fileConfig passes it.
self.datefmt = datefmt
def formatException(self, ei, strip_newlines=True):
lines = traceback.format_exception(*ei)
if strip_newlines:
lines = [itertools.ifilter(
lambda x: x,
line.rstrip().splitlines()) for line in lines]
lines = list(itertools.chain(*lines))
return lines
def format(self, record):
message = {'message': record.getMessage(),
'asctime': self.formatTime(record, self.datefmt),
'name': record.name,
'msg': record.msg,
'args': record.args,
'levelname': record.levelname,
'levelno': record.levelno,
'pathname': record.pathname,
'filename': record.filename,
'module': record.module,
'lineno': record.lineno,
'funcname': record.funcName,
'created': record.created,
'msecs': record.msecs,
'relative_created': record.relativeCreated,
'thread': record.thread,
'thread_name': record.threadName,
'process_name': record.processName,
'process': record.process,
'traceback': None}
if hasattr(record, 'extra'):
message['extra'] = record.extra
if record.exc_info:
message['traceback'] = self.formatException(record.exc_info)
return jsonutils.dumps(message)
def _create_logging_excepthook(product_name):
def logging_excepthook(type, value, tb):
extra = {}
if CONF.verbose:
extra['exc_info'] = (type, value, tb)
getLogger(product_name).critical(str(value), **extra)
return logging_excepthook
class LogConfigError(Exception):
message = _('Error loading logging config %(log_config)s: %(err_msg)s')
def __init__(self, log_config, err_msg):
self.log_config = log_config
self.err_msg = err_msg
def __str__(self):
return self.message % dict(log_config=self.log_config,
err_msg=self.err_msg)
def _load_log_config(log_config_append):
try:
logging.config.fileConfig(log_config_append,
disable_existing_loggers=False)
except moves.configparser.Error as exc:
raise LogConfigError(log_config_append, str(exc))
def setup(product_name):
"""Setup logging."""
if CONF.log_config_append:
_load_log_config(CONF.log_config_append)
else:
_setup_logging_from_conf()
sys.excepthook = _create_logging_excepthook(product_name)
def set_defaults(logging_context_format_string):
cfg.set_defaults(log_opts,
logging_context_format_string=
logging_context_format_string)
def _find_facility_from_conf():
facility_names = logging.handlers.SysLogHandler.facility_names
facility = getattr(logging.handlers.SysLogHandler,
CONF.syslog_log_facility,
None)
if facility is None and CONF.syslog_log_facility in facility_names:
facility = facility_names.get(CONF.syslog_log_facility)
if facility is None:
valid_facilities = facility_names.keys()
consts = ['LOG_AUTH', 'LOG_AUTHPRIV', 'LOG_CRON', 'LOG_DAEMON',
'LOG_FTP', 'LOG_KERN', 'LOG_LPR', 'LOG_MAIL', 'LOG_NEWS',
'LOG_AUTH', 'LOG_SYSLOG', 'LOG_USER', 'LOG_UUCP',
'LOG_LOCAL0', 'LOG_LOCAL1', 'LOG_LOCAL2', 'LOG_LOCAL3',
'LOG_LOCAL4', 'LOG_LOCAL5', 'LOG_LOCAL6', 'LOG_LOCAL7']
valid_facilities.extend(consts)
raise TypeError(_('syslog facility must be one of: %s') %
', '.join("'%s'" % fac
for fac in valid_facilities))
return facility
def _setup_logging_from_conf():
log_root = getLogger(None).logger
for handler in log_root.handlers:
log_root.removeHandler(handler)
if CONF.use_syslog:
facility = _find_facility_from_conf()
syslog = logging.handlers.SysLogHandler(address='/dev/log',
facility=facility)
log_root.addHandler(syslog)
logpath = _get_log_file_path()
if logpath:
filelog = logging.handlers.WatchedFileHandler(logpath)
log_root.addHandler(filelog)
if CONF.use_stderr:
streamlog = ColorHandler()
log_root.addHandler(streamlog)
elif not CONF.log_file:
# pass sys.stdout as a positional argument
# python2.6 calls the argument strm, in 2.7 it's stream
streamlog = logging.StreamHandler(sys.stdout)
log_root.addHandler(streamlog)
if CONF.publish_errors:
handler = importutils.import_object(
"billingstack.openstack.common.log_handler.PublishErrorsHandler",
logging.ERROR)
log_root.addHandler(handler)
datefmt = CONF.log_date_format
for handler in log_root.handlers:
# NOTE(alaski): CONF.log_format overrides everything currently. This
# should be deprecated in favor of context aware formatting.
if CONF.log_format:
handler.setFormatter(logging.Formatter(fmt=CONF.log_format,
datefmt=datefmt))
log_root.info('Deprecated: log_format is now deprecated and will '
'be removed in the next release')
else:
handler.setFormatter(ContextFormatter(datefmt=datefmt))
if CONF.debug:
log_root.setLevel(logging.DEBUG)
elif CONF.verbose:
log_root.setLevel(logging.INFO)
else:
log_root.setLevel(logging.WARNING)
for pair in CONF.default_log_levels:
mod, _sep, level_name = pair.partition('=')
level = logging.getLevelName(level_name)
logger = logging.getLogger(mod)
logger.setLevel(level)
_loggers = {}
def getLogger(name='unknown', version='unknown'):
if name not in _loggers:
_loggers[name] = ContextAdapter(logging.getLogger(name),
name,
version)
return _loggers[name]
def getLazyLogger(name='unknown', version='unknown'):
"""Returns lazy logger.
Creates a pass-through logger that does not create the real logger
until it is really needed and delegates all calls to the real logger
once it is created.
"""
return LazyAdapter(name, version)
class WritableLogger(object):
"""A thin wrapper that responds to `write` and logs."""
def __init__(self, logger, level=logging.INFO):
self.logger = logger
self.level = level
def write(self, msg):
self.logger.log(self.level, msg)
class ContextFormatter(logging.Formatter):
"""A context.RequestContext aware formatter configured through flags.
The flags used to set format strings are: logging_context_format_string
and logging_default_format_string. You can also specify
logging_debug_format_suffix to append extra formatting if the log level is
debug.
For information about what variables are available for the formatter see:
http://docs.python.org/library/logging.html#formatter
"""
def format(self, record):
"""Uses contextstring if request_id is set, otherwise default."""
# NOTE(sdague): default the fancier formating params
# to an empty string so we don't throw an exception if
# they get used
for key in ('instance', 'color'):
if key not in record.__dict__:
record.__dict__[key] = ''
if record.__dict__.get('request_id', None):
self._fmt = CONF.logging_context_format_string
else:
self._fmt = CONF.logging_default_format_string
if (record.levelno == logging.DEBUG and
CONF.logging_debug_format_suffix):
self._fmt += " " + CONF.logging_debug_format_suffix
# Cache this on the record, Logger will respect our formated copy
if record.exc_info:
record.exc_text = self.formatException(record.exc_info, record)
return logging.Formatter.format(self, record)
def formatException(self, exc_info, record=None):
"""Format exception output with CONF.logging_exception_prefix."""
if not record:
return logging.Formatter.formatException(self, exc_info)
stringbuffer = moves.StringIO()
traceback.print_exception(exc_info[0], exc_info[1], exc_info[2],
None, stringbuffer)
lines = stringbuffer.getvalue().split('\n')
stringbuffer.close()
if CONF.logging_exception_prefix.find('%(asctime)') != -1:
record.asctime = self.formatTime(record, self.datefmt)
formatted_lines = []
for line in lines:
pl = CONF.logging_exception_prefix % record.__dict__
fl = '%s%s' % (pl, line)
formatted_lines.append(fl)
return '\n'.join(formatted_lines)
class ColorHandler(logging.StreamHandler):
LEVEL_COLORS = {
logging.DEBUG: '\033[00;32m', # GREEN
logging.INFO: '\033[00;36m', # CYAN
logging.AUDIT: '\033[01;36m', # BOLD CYAN
logging.WARN: '\033[01;33m', # BOLD YELLOW
logging.ERROR: '\033[01;31m', # BOLD RED
logging.CRITICAL: '\033[01;31m', # BOLD RED
}
def format(self, record):
record.color = self.LEVEL_COLORS[record.levelno]
return logging.StreamHandler.format(self, record)
class DeprecatedConfig(Exception):
message = _("Fatal call to deprecated config: %(msg)s")
def __init__(self, msg):
super(Exception, self).__init__(self.message % dict(msg=msg))

147
billingstack/openstack/common/loopingcall.py
View File

@@ -1,147 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# Copyright 2011 Justin Santa Barbara
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import sys
from eventlet import event
from eventlet import greenthread
from billingstack.openstack.common.gettextutils import _ # noqa
from billingstack.openstack.common import log as logging
from billingstack.openstack.common import timeutils
LOG = logging.getLogger(__name__)
class LoopingCallDone(Exception):
"""Exception to break out and stop a LoopingCall.
The poll-function passed to LoopingCall can raise this exception to
break out of the loop normally. This is somewhat analogous to
StopIteration.
An optional return-value can be included as the argument to the exception;
this return-value will be returned by LoopingCall.wait()
"""
def __init__(self, retvalue=True):
""":param retvalue: Value that LoopingCall.wait() should return."""
self.retvalue = retvalue
class LoopingCallBase(object):
def __init__(self, f=None, *args, **kw):
self.args = args
self.kw = kw
self.f = f
self._running = False
self.done = None
def stop(self):
self._running = False
def wait(self):
return self.done.wait()
class FixedIntervalLoopingCall(LoopingCallBase):
"""A fixed interval looping call."""
def start(self, interval, initial_delay=None):
self._running = True
done = event.Event()
def _inner():
if initial_delay:
greenthread.sleep(initial_delay)
try:
while self._running:
start = timeutils.utcnow()
self.f(*self.args, **self.kw)
end = timeutils.utcnow()
if not self._running:
break
delay = interval - timeutils.delta_seconds(start, end)
if delay <= 0:
LOG.warn(_('task run outlasted interval by %s sec') %
-delay)
greenthread.sleep(delay if delay > 0 else 0)
except LoopingCallDone as e:
self.stop()
done.send(e.retvalue)
except Exception:
LOG.exception(_('in fixed duration looping call'))
done.send_exception(*sys.exc_info())
return
else:
done.send(True)
self.done = done
greenthread.spawn_n(_inner)
return self.done
# TODO(mikal): this class name is deprecated in Havana and should be removed
# in the I release
LoopingCall = FixedIntervalLoopingCall
class DynamicLoopingCall(LoopingCallBase):
"""A looping call which sleeps until the next known event.
The function called should return how long to sleep for before being
called again.
"""
def start(self, initial_delay=None, periodic_interval_max=None):
self._running = True
done = event.Event()
def _inner():
if initial_delay:
greenthread.sleep(initial_delay)
try:
while self._running:
idle = self.f(*self.args, **self.kw)
if not self._running:
break
if periodic_interval_max is not None:
idle = min(idle, periodic_interval_max)
LOG.debug(_('Dynamic looping call sleeping for %.02f '
'seconds'), idle)
greenthread.sleep(idle)
except LoopingCallDone as e:
self.stop()
done.send(e.retvalue)
except Exception:
LOG.exception(_('in dynamic looping call'))
done.send_exception(*sys.exc_info())
return
else:
done.send(True)
self.done = done
greenthread.spawn(_inner)
return self.done

81
billingstack/openstack/common/network_utils.py
View File

@@ -1,81 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2012 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Network-related utilities and helper functions.
"""
import urlparse
def parse_host_port(address, default_port=None):
"""Interpret a string as a host:port pair.
An IPv6 address MUST be escaped if accompanied by a port,
because otherwise ambiguity ensues: 2001:db8:85a3::8a2e:370:7334
means both [2001:db8:85a3::8a2e:370:7334] and
[2001:db8:85a3::8a2e:370]:7334.
>>> parse_host_port('server01:80')
('server01', 80)
>>> parse_host_port('server01')
('server01', None)
>>> parse_host_port('server01', default_port=1234)
('server01', 1234)
>>> parse_host_port('[::1]:80')
('::1', 80)
>>> parse_host_port('[::1]')
('::1', None)
>>> parse_host_port('[::1]', default_port=1234)
('::1', 1234)
>>> parse_host_port('2001:db8:85a3::8a2e:370:7334', default_port=1234)
('2001:db8:85a3::8a2e:370:7334', 1234)
"""
if address[0] == '[':
# Escaped ipv6
_host, _port = address[1:].split(']')
host = _host
if ':' in _port:
port = _port.split(':')[1]
else:
port = default_port
else:
if address.count(':') == 1:
host, port = address.split(':')
else:
# 0 means ipv4, >1 means ipv6.
# We prohibit unescaped ipv6 addresses with port.
host = address
port = default_port
return (host, None if port is None else int(port))
def urlsplit(url, scheme='', allow_fragments=True):
"""Parse a URL using urlparse.urlsplit(), splitting query and fragments.
This function papers over Python issue9374 when needed.
The parameters are the same as urlparse.urlsplit.
"""
scheme, netloc, path, query, fragment = urlparse.urlsplit(
url, scheme, allow_fragments)
if allow_fragments and '#' in path:
path, fragment = path.split('#', 1)
if '?' in path:
path, query = path.split('?', 1)
return urlparse.SplitResult(scheme, netloc, path, query, fragment)

14
billingstack/openstack/common/notifier/__init__.py
View File

@@ -1,14 +0,0 @@
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

173
billingstack/openstack/common/notifier/api.py
View File

@@ -1,173 +0,0 @@
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import socket
import uuid
from oslo.config import cfg
from billingstack.openstack.common import context
from billingstack.openstack.common.gettextutils import _ # noqa
from billingstack.openstack.common import importutils
from billingstack.openstack.common import jsonutils
from billingstack.openstack.common import log as logging
from billingstack.openstack.common import timeutils
LOG = logging.getLogger(__name__)
notifier_opts = [
cfg.MultiStrOpt('notification_driver',
default=[],
help='Driver or drivers to handle sending notifications'),
cfg.StrOpt('default_notification_level',
default='INFO',
help='Default notification level for outgoing notifications'),
cfg.StrOpt('default_publisher_id',
default=None,
help='Default publisher_id for outgoing notifications'),
]
CONF = cfg.CONF
CONF.register_opts(notifier_opts)
WARN = 'WARN'
INFO = 'INFO'
ERROR = 'ERROR'
CRITICAL = 'CRITICAL'
DEBUG = 'DEBUG'
log_levels = (DEBUG, WARN, INFO, ERROR, CRITICAL)
class BadPriorityException(Exception):
pass
def notify_decorator(name, fn):
"""Decorator for notify which is used from utils.monkey_patch().
:param name: name of the function
:param function: - object of the function
:returns: function -- decorated function
"""
def wrapped_func(*args, **kwarg):
body = {}
body['args'] = []
body['kwarg'] = {}
for arg in args:
body['args'].append(arg)
for key in kwarg:
body['kwarg'][key] = kwarg[key]
ctxt = context.get_context_from_function_and_args(fn, args, kwarg)
notify(ctxt,
CONF.default_publisher_id or socket.gethostname(),
name,
CONF.default_notification_level,
body)
return fn(*args, **kwarg)
return wrapped_func
def publisher_id(service, host=None):
if not host:
try:
host = CONF.host
except AttributeError:
host = CONF.default_publisher_id or socket.gethostname()
return "%s.%s" % (service, host)
def notify(context, publisher_id, event_type, priority, payload):
"""Sends a notification using the specified driver
:param publisher_id: the source worker_type.host of the message
:param event_type: the literal type of event (ex. Instance Creation)
:param priority: patterned after the enumeration of Python logging
levels in the set (DEBUG, WARN, INFO, ERROR, CRITICAL)
:param payload: A python dictionary of attributes
Outgoing message format includes the above parameters, and appends the
following:
message_id
a UUID representing the id for this notification
timestamp
the GMT timestamp the notification was sent at
The composite message will be constructed as a dictionary of the above
attributes, which will then be sent via the transport mechanism defined
by the driver.
Message example::
{'message_id': str(uuid.uuid4()),
'publisher_id': 'compute.host1',
'timestamp': timeutils.utcnow(),
'priority': 'WARN',
'event_type': 'compute.create_instance',
'payload': {'instance_id': 12, ... }}
"""
if priority not in log_levels:
raise BadPriorityException(
_('%s not in valid priorities') % priority)
# Ensure everything is JSON serializable.
payload = jsonutils.to_primitive(payload, convert_instances=True)
msg = dict(message_id=str(uuid.uuid4()),
publisher_id=publisher_id,
event_type=event_type,
priority=priority,
payload=payload,
timestamp=str(timeutils.utcnow()))
for driver in _get_drivers():
try:
driver.notify(context, msg)
except Exception as e:
LOG.exception(_("Problem '%(e)s' attempting to "
"send to notification system. "
"Payload=%(payload)s")
% dict(e=e, payload=payload))
_drivers = None
def _get_drivers():
"""Instantiate, cache, and return drivers based on the CONF."""
global _drivers
if _drivers is None:
_drivers = {}
for notification_driver in CONF.notification_driver:
try:
driver = importutils.import_module(notification_driver)
_drivers[notification_driver] = driver
except ImportError:
LOG.exception(_("Failed to load notifier %s. "
"These notifications will not be sent.") %
notification_driver)
return _drivers.values()
def _reset_drivers():
"""Used by unit tests to reset the drivers."""
global _drivers
_drivers = None

37
billingstack/openstack/common/notifier/log_notifier.py
View File

@@ -1,37 +0,0 @@
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
from billingstack.openstack.common import jsonutils
from billingstack.openstack.common import log as logging
CONF = cfg.CONF
def notify(_context, message):
"""Notifies the recipient of the desired event given the model.
Log notifications using OpenStack's default logging system.
"""
priority = message.get('priority',
CONF.default_notification_level)
priority = priority.lower()
logger = logging.getLogger(
'billingstack.openstack.common.notification.%s' %
message['event_type'])
getattr(logger, priority)(jsonutils.dumps(message))

19
billingstack/openstack/common/notifier/no_op_notifier.py
View File

@@ -1,19 +0,0 @@
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
def notify(_context, message):
"""Notifies the recipient of the desired event given the model."""
pass

46
billingstack/openstack/common/notifier/rabbit_notifier.py
View File

@@ -1,46 +0,0 @@
# Copyright 2011 OpenStack LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from billingstack.openstack.common import cfg
from billingstack.openstack.common import context as req_context
from billingstack.openstack.common.gettextutils import _
from billingstack.openstack.common import log as logging
from billingstack.openstack.common import rpc
LOG = logging.getLogger(__name__)
notification_topic_opt = cfg.ListOpt(
'notification_topics', default=['notifications', ],
help='AMQP topic used for openstack notifications')
CONF = cfg.CONF
CONF.register_opt(notification_topic_opt)
def notify(context, message):
"""Sends a notification to the RabbitMQ"""
if not context:
context = req_context.get_admin_context()
priority = message.get('priority',
CONF.default_notification_level)
priority = priority.lower()
for topic in CONF.notification_topics:
topic = '%s.%s' % (topic, priority)
try:
rpc.notify(context, topic, message)
except Exception, e:
LOG.exception(_("Could not send notification to %(topic)s. "
"Payload=%(message)s"), locals())

47
billingstack/openstack/common/notifier/rpc_notifier.py
View File

@@ -1,47 +0,0 @@
# Copyright 2011 OpenStack Foundation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo.config import cfg
from billingstack.openstack.common import context as req_context
from billingstack.openstack.common.gettextutils import _ # noqa
from billingstack.openstack.common import log as logging
from billingstack.openstack.common import rpc
LOG = logging.getLogger(__name__)
notification_topic_opt = cfg.ListOpt(
'notification_topics', default=['notifications', ],
help='AMQP topic used for OpenStack notifications')
CONF = cfg.CONF
CONF.register_opt(notification_topic_opt)
def notify(context, message):
"""Sends a notification via RPC."""
if not context:
context = req_context.get_admin_context()
priority = message.get('priority',
CONF.default_notification_level)
priority = priority.lower()
for topic in CONF.notification_topics:
topic = '%s.%s' % (topic, priority)
try:
rpc.notify(context, topic, message)
except Exception:
LOG.exception(_("Could not send notification to %(topic)s. "
"Payload=%(message)s"),
{"topic": topic, "message": message})

Some files were not shown because too many files have changed in this diff Show More