Merged pull 37. Removes bottle, adds configuration, and adds daemonization

This commit is contained in:
Ziad Sawalha 2011-05-14 23:14:51 -05:00
parent a3d6a8d8b9
commit d0447d4aba
12 changed files with 148 additions and 166 deletions

29
HACKING
View File

@ -1,5 +1,5 @@
Nova Style Commandments
=======================
Keystone Style Commandments (pilfered from Nova and added to)
=============================================================
Step 1: Read http://www.python.org/dev/peps/pep-0008/
Step 2: Read http://www.python.org/dev/peps/pep-0008/ again
@ -16,7 +16,7 @@ Imports
# vim: tabstop=4 shiftwidth=4 softtabstop=4
{{stdlib imports in human alphabetical order}}
\n
{{nova imports in human alphabetical order}}
{{OpenStack/Keystone imports in human alphabetical order}}
\n
\n
{{begin your code}}
@ -27,8 +27,9 @@ General
- thou shalt put two newlines twixt toplevel code (funcs, classes, etc)
- thou shalt put one newline twixt methods in classes and anywhere else
- thou shalt not write "except:", use "except Exception:" at the very least
- thou shalt include your name with TODOs as in "TODO(termie)"
- thou shalt include your name with TODOs as in "TODO(waldo)"
- thou shalt not name anything the same name as a builtin or reserved word
- thou shouldeth comment profusely
- thou shalt not violate causality in our time cone, or else
@ -42,14 +43,12 @@ Human Alphabetical Order Examples
import time
import unittest
from nova import flags
from nova import test
from nova.auth import users
from nova.endpoint import api
from nova.endpoint import cloud
import keystone.logic.types.fault as fault
import keystone.db.sqlalchemy.api as db_api
Docstrings
----------
Add them to modules, classes, and functions:
"""Summary of the function, class or method, less than 80 characters.
New paragraph after newline that explains in more detail any general
@ -66,3 +65,15 @@ Docstrings
:returns: description of the return value
"""
Done/Done Criteria
------------------
How we define our code is done and ready for release:
1. PEP-8 compliance
2. pylint (same rules as Nova)
3. McCabe 10 or less
4. 65.258% test coverage
5. All functional and unit tests pass
6. Q/A Approval (if applicable - it is for Rackspace Integration dev teams)
7. No sev A bugs (this shoud have been #1)

View File

@ -1,4 +1,3 @@
Keystone: Identity Service
==========================
@ -26,102 +25,100 @@ Also included:
ENVIRONMENT & DEPENDENCIES:
---------------------------
see pip-requires for dependency list
See pip-requires for dependency list
Setup:
Install http://pypi.python.org/pypi/setuptools
sudo easy_install pip
sudo pip install -r pip-requires
Configuration:
Keystone gets its configuration from command-line parameters or a .conf file. The file can be provided explicitely
on the command line otherwise the following logic applies (the conf file in use will be output to help
in troubleshooting:
1. config.py takes the config file from <topdir>/etc/keystone.conf
2. If the keystone package is also intalled on the system,
/etc/keystone.conf or /etc/keystone/keystone.conf have higher priority than <top_dir>/etc/keystone.conf.
If you are also doing development on a system that has keystone.conf installed in /etc you may need to disambiguate it by providing the conf file in the command-line
$ bin/keystone-control --confg-file etc/keystone.conf --pid-file <pidfile> auth <start|stop|restart>
Path:
keystone-control calls keystone-auth and it needs to be in the PATH
$ export PATH=<top_dir>/bin:$PATH
RUNNING KEYSTONE:
-----------------
During development, you can simply run
$ cd bin
$ ./keystone-auth
$ bin/keystone-auth
RUNNING KEYSTONE FOR DEVELOPMENT (HACKING):
------------------------------
During development, you can simply run as user (root not needed)
From the top Keystone directory (<topdir>)
$ bin/keystone=auth
It dumps stdout and stderr onto the terminal.
If you want to specify additional parameters (optional):
RUNNING KEYSOTNE IN AS ROOT IN PRODUCTION
---------------------------------------------
In production, stdout and stderr need to be closed and all theoutput needs tobe redirected to a log file.
Once the package is installed through setup tools, RPM, deb, or ebuild keystone-control is installed
as /usr/sbin/keystone-control. Typically, it will be started a script in /etc/init.d/keystoned
$ bin/keystone-control --pid-file <pidfile> --config-file etc/keystone.conf auth <start|stop|restart>
RUNNING KEYSTONE AS ROOT IN PRODUCTION
--------------------------------------
In production, stdout and stderr need to be closed and all the output needs to be redirected to a log file.
Once the package is installed through setup tools, RPM, deb, or ebuild keystone-control is installed as /usr/sbin/keystone-control. Typically, it will be started a script in /etc/init.d/keystoned
keystone-control can invoke keystone-auth and start the keystone daemon with
$ /usr/sbin/keystone-control auth start
It writes the process id of the daemon into /var/run/keystone/keystine-auth.pid. he daemon can be stopped with
It writes the process id of the daemon into /var/run/keystone/keystine-auth.pid.
The daemon can be stopped with
$ /usr/sbin/keystone-control auth stop
keystone-control has the infrastructure to start and stop multiple servers keystone-xxx
DEVELOPMENT OF keystone-control
-------------------------------
During the development of keystone-control can be started as a user instead of root
From the topdir
$ bin/keystone-control --pid-file pidfile auth <start|stop|restart>
config.py takes the config file from topdir/etc/keystone.conf
If the keystone package is also intalled on the system
/etc/keystone.conf or /etc/keystone/keystone.conf has higher priority
than <top_dir>/etc/keystone.conf. If you are also doing development on a
system that has keystone.conf installed in /etc/you need to disambiguate it by
$ bin/keystone-control --confg-file etc/keystone.conf --pid-file pidfile auth <start|stop|restart>
Also, keystone-control calls keystone-auth and it need to be in the PATH
$ export PATH=<top_dir>/bin:$PATH
RUNNING TEST SERVICE:
---------------------
Standalone stack (with Auth_Token)
$ cd echo/echo
$ python echo.py
$ cd echo/bin
$ ./echod
Distributed stack (with RemoteAuth local and Auth_Token remote)
$ cd echo/echo
$ python echo.py --remote
$ cd echo/bin
$ ./echod --remote
in separate session
$ cd keystone/auth_protocols
$ python auth_token.py --remote
DEMO CLIENT:
---------------------
$ cd echo/echo
$ python echo_client.py
INSTALLING KEYSTONE:
--------------------
$ python setup.py build
$ sudo python setup.py install
INSTALLING TEST SERVICE:
------------------------
$ cd echo
$ python setup.py build
$ sudo python setup.py install
Note: this requires tests data. See section TESTING for initializing data
TESTING
-------
After starting identity.py a keystone.db sql-lite database should be created.
After starting keystone a keystone.db sqlite database should be created in the keystone folder.
To test setup the test database:
Add test data to the database:
$ sqlite3 keystone/keystone.db < test/test_setup.sql
@ -131,49 +128,34 @@ To clean the test database
To run client demo (with all auth middleware running locally on sample service):
$ python echo/echo/echo.py
$ ./echo/bin/echod
$ python echo/echo/echo_client.py
To run unit tests:
* go to unit test/unit directory
* run tests: python test_keystone
There are 8 groups of tests. They can be run individually or as an entire colection. To run the entire test suite run
$ python test_keystone.py
A test can also be run individually e.g.
$ python test_token.py
For more on unit testing please refer
$ python test_keystone.py --help
To perform contract validation and load testing, use SoapUI (for now).
Using SOAPUI:
Download [SOAPUI](http://sourceforge.net/projects/soapui/files/):
To Test Identity Service:
To Test Keystone Service:
* File->Import Project
* Select tests/IdentitySOAPUI.xml
* Double click on "Keystone Tests" and press the green play (>) button
Unit Test on Identity Services
------------------------------
In order to run the unit test on identity services start the auth sever
$ cd test/unit
$ ../../bin/keystone-auth
There are 8 groups of tests. They can be run individually or as an entire colection. To run the entire test suite run
$ python test_keystone
A test can also be run individually e.g.
$ python test_token
DATABASE SCHEMA
---------------
CREATE TABLE groups(group_id varchar(255),group_desc varchar(255),tenant_id varchar(255),FOREIGN KEY(tenant_id) REFERENCES tenant(tenant_id));
CREATE TABLE tenants(tenant_id varchar(255), tenant_desc varchar(255), tenant_enabled INTEGER, PRIMARY KEY(tenant_id ASC));
CREATE TABLE token(token_id varchar(255),user_id varchar(255),expires datetime,tenant_id varchar(255));
CREATE TABLE user_group(user_id varchar(255),group_id varchar(255), FOREIGN KEY(user_id) REFERENCES user(id), FOREIGN KEY(group_id) REFERENCES groups(group_id));
CREATE TABLE user_tenant(tenant_id varchar(255),user_id varchar(255),FOREIGN KEY(tenant_id) REFERENCES tenant(tenant_id),FOREIGN KEY(user_id) REFERENCES user(id));
CREATE TABLE users(id varchar(255),password varchar(255),email varchar(255),enabled integer);

View File

@ -1,26 +0,0 @@
#!/bin/sh
# Copyright (C) 2011 OpenStack LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# If ../keystone/__init__.py exists, add ../ to the Python search path so
# that it will override whatever may be installed in the default Python
# search path.
script_dir=`dirname $0`
if [ -f "$script_dir/../keystone/__init__.py" ]
then
PYTHONPATH="$script_dir/..:$PYTHONPATH"
export PYTHONPATH
fi
/usr/bin/env python -m keystone.server $*

View File

@ -72,7 +72,7 @@
<xs:attribute name="rel" use="required" type="atom:relation">
<xs:annotation>
<xs:documentation>
<html:p>TODO</html:p>
<html:p>TODO(Jorge)</html:p>
</xs:documentation>
</xs:annotation>
</xs:attribute>
@ -80,7 +80,7 @@
<xs:attribute name="type" use="optional" type="xs:string">
<xs:annotation>
<xs:documentation>
<html:p>TODO</html:p>
<html:p>TODO(Jorge)</html:p>
</xs:documentation>
</xs:annotation>
</xs:attribute>
@ -88,7 +88,7 @@
<xs:attribute name="href" use="required" type="xs:anyURI">
<xs:annotation>
<xs:documentation>
<html:p>TODO</html:p>
<html:p>TODO(Jorge)</html:p>
</xs:documentation>
</xs:annotation>
</xs:attribute>
@ -96,7 +96,7 @@
<xs:attribute name="hreflang" use="optional" type="xs:NMTOKEN">
<xs:annotation>
<xs:documentation>
<html:p>TODO</html:p>
<html:p>TODO(Jorge)</html:p>
</xs:documentation>
</xs:annotation>
</xs:attribute>
@ -104,7 +104,7 @@
<xs:attribute name="title" use="optional" type="xs:string">
<xs:annotation>
<xs:documentation>
<html:p>TODO</html:p>
<html:p>TODO(Jorge)</html:p>
</xs:documentation>
</xs:annotation>
</xs:attribute>

View File

@ -17,7 +17,7 @@
# under the License.
"""
Routines for configuring Glance
Routines for configuring OpenStack Service
"""
import ConfigParser
@ -26,11 +26,10 @@ import logging.config
import logging.handlers
import optparse
import os
from paste import deploy
import re
import sys
from paste import deploy
import keystone.common.exception as exception
DEFAULT_LOG_FORMAT = "%(asctime)s %(levelname)8s [%(name)s] %(message)s"

View File

@ -17,8 +17,8 @@
# under the License.
"""
Nova base exception handling, including decorator for re-raising
Nova-type exceptions. SHOULD include dedicated exception logging.
OpenStack base exception handling, including decorator for re-raising
OpenSTack-type exceptions. SHOULD include dedicated exception logging.
"""
import logging

View File

@ -1,5 +1,26 @@
#
# Copyright (c) 2011, Marcel Hellkamp.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
#
# original code copied from bottle.py
##""" code cut and paste from bottle.py """
import cgi
import re
@ -7,6 +28,7 @@ import os
import functools
import time
from webob import Response
import keystone.logic.types.fault as fault
TEMPLATES = {}
@ -15,11 +37,12 @@ TEMPLATE_PATH = ['./', './views/']
class BaseTemplate(object):
""" Base class and minimal API for template adapters """
extentions = ['tpl','html','thtml','stpl']
settings = {} #used in prepare()
defaults = {} #used in render()
extentions = ['tpl', 'html', 'thtml', 'stpl']
settings = {} #used in prepare()
defaults = {} #used in render()
def __init__(self, source=None, name=None, lookup=[], encoding='utf8', **settings):
def __init__(self, source=None, name=None, lookup=[], encoding='utf8',
**settings):
""" Create a new template.
If the source parameter (str or buffer) is missing, the name argument
is used to guess a template filename. Subclasses can assume that
@ -49,7 +72,8 @@ class BaseTemplate(object):
def search(cls, name, lookup=[]):
""" Search name in all directories specified in lookup.
First without, then with common extensions. Return first hit. """
if os.path.isfile(name): return name
if os.path.isfile(name):
return name
for spath in lookup:
fname = os.path.join(spath, name)
if os.path.isfile(fname):
@ -133,9 +157,12 @@ class SimpleTemplate(BaseTemplate):
cline = ''
for line in ptrbuffer:
for token, value in line:
if token == 'TXT': cline += repr(value)
elif token == 'RAW': cline += '_str(%s)' % value
elif token == 'CMD': cline += '_escape(%s)' % value
if token == 'TXT':
cline += repr(value)
elif token == 'RAW':
cline += '_str(%s)' % value
elif token == 'CMD':
cline += '_escape(%s)' % value
cline += ', '
cline = cline[:-2] + '\\\n'
cline = cline[:-2]

View File

@ -15,18 +15,16 @@
from datetime import datetime
from datetime import timedelta
import uuid
import keystone.logic.types.auth as auth
import keystone.logic.types.tenant as tenants
import keystone.logic.types.atom as atom
import keystone.logic.types.fault as fault
import keystone.logic.types.user as users
import keystone.db.sqlalchemy.api as db_api
import keystone.db.sqlalchemy.models as db_models
import uuid
class IDMService(object):
"This is the logical implemenation of the IDM service"
@ -54,7 +52,8 @@ class IDMService(object):
if not credentials.tenant_id:
dtoken = db_api.token_for_user(duser.id)
else:
dtoken = db_api.token_for_user_tenant(duser.id, credentials.tenant_id)
dtoken = db_api.token_for_user_tenant(duser.id,
credentials.tenant_id)
if not dtoken or dtoken.expires < datetime.now():
dtoken = db_models.Token()
dtoken.token_id = str(uuid.uuid4())
@ -63,7 +62,8 @@ class IDMService(object):
if not duser.tenants:
raise fault.IDMFault("Strange: user %s is not associated "
"with a tenant!" % duser.id)
if not credentials.tenant_id and db_api.user_get_by_tenant(duser.id, credentials.tenant_id):
user = db_api.user_get_by_tenant(duser.id, credentials.tenant_id)
if not credentials.tenant_id and user:
raise fault.IDMFault("Error: user %s is not associated "
"with a tenant! %s" % (duser.id,
credentials.tenant_id))
@ -90,7 +90,6 @@ class IDMService(object):
% user.id)
return self.__get_auth_data(token, user)
def revoke_token(self, admin_token, token_id):
self.__validate_token(admin_token)
@ -123,14 +122,11 @@ class IDMService(object):
dtenant.enabled = tenant.enabled
db_api.tenant_create(dtenant)
return tenant
##
## GET Tenants with Pagination
##
def get_tenants(self, admin_token, marker, limit, url):
self.__validate_token(admin_token)
@ -202,7 +198,6 @@ class IDMService(object):
if dtenant == None:
raise fault.ItemNotFoundFault("The tenant not found")
if group.group_id == None:
raise fault.BadRequestFault("Expecting a Group Id")
@ -389,11 +384,9 @@ class IDMService(object):
db_api.user_tenant_group_delete(user, group)
return None
#
# Private Operations
#
def __get_dauth_data(self, token_id):
"""return token and user object for a token_id"""
@ -432,7 +425,6 @@ class IDMService(object):
raise fault.EmailConflictFault(
"Email already exists")
duser_tenant = db_models.UserTenantAssociation()
duser_tenant.user_id = user.user_id
duser_tenant.tenant_id = tenant_id
@ -521,7 +513,6 @@ class IDMService(object):
if not duser.enabled:
raise fault.UserDisabledFault("User has been disabled")
if not isinstance(user, users.User):
raise fault.BadRequestFault("Expecting a User")
@ -608,7 +599,8 @@ class IDMService(object):
db_api.user_delete_tenant(user_id, tenant_id)
return None
def get_user_groups(self, admin_token, tenant_id, user_id, marker, limit, url):
def get_user_groups(self, admin_token, tenant_id, user_id, marker, limit,
url):
self.__validate_token(admin_token)
if tenant_id == None:
@ -625,8 +617,6 @@ class IDMService(object):
limit)
for dusergroup, dusergroupAsso in dusergroups:
ts.append(tenants.Group(dusergroup.id, dusergroup.desc,
dusergroup.tenant_id))
links = []
@ -641,13 +631,11 @@ class IDMService(object):
(url, next, limit)))
return tenants.Groups(ts, links)
#
# Global Group Operations
# TODO:(India Team) Rename functions
# and to maintain consistency
# with server.py
def __check_create_global_tenant(self):
dtenant = db_api.tenant_get('GlobalTenant')

View File

@ -17,9 +17,10 @@ from datetime import datetime
from abc import ABCMeta
import json
import keystone.logic.types.fault as fault
from lxml import etree
import keystone.logic.types.fault as fault
class PasswordCredentials(object):
"Credentials based on username, password, and (optional) tenant_id."

View File

@ -14,10 +14,11 @@
# limitations under the License.
import json
import keystone.logic.types.fault as fault
from lxml import etree
import string
import keystone.logic.types.fault as fault
class Tenant(object):
"Describes a tenant in the auth system"

View File

@ -36,13 +36,12 @@ HTTP_X_AUTHORIZATION: the client identity being passed in
"""
import functools
import logging
import os
import sys
import httplib
import json
import logging
import os
import routes
import sys
from webob import Response
from webob import Request
from webob import descriptors

View File

@ -1,4 +1,4 @@
bottle
bottle #still used in queryext
eventlet
lxml
paste