openstack
/
keystone
Code Issues Proposed changes

Remove eventlet support 

Eventlet has been deprecated since the Kilo release and is
being removed in Newton.

A follow on patch will be proposed to remove the [ssl] section
since it is now redundant.

Co-Authored-By: Grzegorz Grasza <grzegorz.grasza@intel.com>
Partially implements: bp removed-as-of-newton

Change-Id: I963d94bbd188dbb6eba68623a42c5bc3f2289da4
changes/86/249486/34
Steve Martinelli 2015-11-24 19:08:14 -05:00
parent c382857e8f
commit ac039414ce
30 changed files with 82 additions and 1340 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
config-generator/keystone.conf
View File

@ -9,7 +9,6 @@ namespace = oslo.messaging
namespace = oslo.policy
namespace = oslo.db
namespace = oslo.middleware
namespace = oslo.service.service
namespace = osprofiler
# We don't use oslo.concurrency config options in
# keystone now, just in case it slips through unnoticed.

4
doc/source/conf.py
View File

@ -111,9 +111,7 @@ modindex_common_prefix = ['keystone.']
man_pages = [
('man/keystone-manage', 'keystone-manage', u'Keystone Management Utility',
[u'OpenStack'], 1),
('man/keystone-all', 'keystone-all', u'Keystone Startup Command',
[u'OpenStack'], 1),
[u'OpenStack'], 1)
]

66
doc/source/configuration.rst
View File

@ -22,7 +22,6 @@ Configuring Keystone
:maxdepth: 1
man/keystone-manage
man/keystone-all
Once Keystone is installed, it is configured via a primary configuration file
(``etc/keystone.conf``), a PasteDeploy configuration file
@ -45,48 +44,6 @@ To make the above change persistent,
``net.ipv4.ip_local_reserved_ports = 35357`` should be added to
``/etc/sysctl.conf`` or to ``/etc/sysctl.d/keystone.conf``.
Starting and Stopping Keystone under Eventlet
=============================================
.. WARNING::
Running keystone under eventlet has been deprecated as of the Kilo release.
Support for utilizing eventlet will be removed as of the M-release. The
recommended deployment is to run keystone in a WSGI server such as Apache
httpd with ``mod_wsgi``.
Keystone can be run using either its built-in eventlet server or it can be run
embedded in a web server. While the eventlet server is convenient and easy to
use, it's lacking in security features that have been developed into Internet-
based web servers over the years. As such, running the eventlet server as
described in this section is not recommended.
Start Keystone services using the command:
.. code-block:: bash
$ keystone-all
Invoking this command starts up two ``wsgi.Server`` instances, ``admin`` (the
administration API) and ``main`` (the primary/public API interface). Both
services are configured to run in a single process.
.. NOTE::
The separation into ``admin`` and ``main`` interfaces is a historical
anomaly. The new V3 API provides the same interface on both the admin and
main interfaces (this can be configured in ``keystone-paste.ini``, but the
default is to have both the same). The V2.0 API provides a limited public
API (getting and validating tokens) on ``main``, and an administrative API
(which can include creating users and such) on the ``admin`` interface.
Stop the process using ``Control-C``.
.. NOTE::
If you have not already configured Keystone, it may not start as expected.
Configuration Files
===================
@ -115,8 +72,6 @@ The primary configuration file is organized into the following sections:
* ``[credential]`` - Credential system driver configuration
* ``[endpoint_filter]`` - Endpoint filtering configuration
* ``[endpoint_policy]`` - Endpoint policy configuration
* ``[eventlet_server]`` - Eventlet server configuration
* ``[eventlet_server_ssl]`` - Eventlet server SSL configuration
* ``[federation]`` - Federation driver configuration
* ``[identity]`` - Identity system driver configuration
* ``[identity_mapping]`` - Identity mapping system driver configuration
@ -977,32 +932,19 @@ certificates are just provided as an example.
Configuration
^^^^^^^^^^^^^
To enable SSL modify the ``etc/keystone.conf`` file under the ``[ssl]`` and
``[eventlet_server_ssl]`` sections. The following is an SSL configuration
example using the included sample certificates:
To enable SSL a deployment should configure a web server (such as Apache) to
use SSL. Keystone is able to generate SSL certificates by modifying the
``[ssl]`` section in the ``etc/keystone.conf`` file. The following is an SSL
configuration example using the included sample certificates:
.. code-block:: ini
[eventlet_server_ssl]
enable = True
certfile = <path to keystone.pem>
keyfile = <path to keystonekey.pem>
ca_certs = <path to ca.pem>
cert_required = False
[ssl]
ca_key = <path to cakey.pem>
key_size = 1024
valid_days=3650
cert_subject=/C=US/ST=Unset/L=Unset/O=Unset/CN=localhost
* ``enable``: True enables SSL. Defaults to False.
* ``certfile``: Path to Keystone public certificate file.
* ``keyfile``: Path to Keystone private certificate file. If the private key is
included in the certfile, the keyfile may be omitted.
* ``ca_certs``: Path to CA trust chain.
* ``cert_required``: Requires client certificate. Defaults to False.
When generating SSL certificates the following values are read
* ``key_size``: Key size to create. Defaults to 1024.

5
doc/source/http-api.rst
View File

@ -222,6 +222,5 @@ HTTP/1.1 Chunked Encoding
Running Keystone under HTTPD in the recommended (and tested) configuration does not support
the use of ``Transfer-Encoding: chunked``. This is due to a limitation with the WSGI spec
and the implementation used by ``mod_wsgi``. Support for chunked encoding under ``eventlet``
may or may not continue. It is recommended that all clients assume Keystone will not support
``Transfer-Encoding: chunked``.
and the implementation used by ``mod_wsgi``. It is recommended that all
clients assume Keystone will not support ``Transfer-Encoding: chunked``.

1
doc/source/index.rst
View File

@ -78,7 +78,6 @@ Man Pages
.. toctree::
:maxdepth: 1
man/keystone-all
man/keystone-manage
Developers Documentation

112
doc/source/man/keystone-all.rst
View File

@ -1,112 +0,0 @@
============
keystone-all
============
------------------------
Keystone Startup Command
------------------------
:Author: openstack@lists.openstack.org
:Date: 2015-10-15
:Copyright: OpenStack Foundation
:Version: 8.0.0
:Manual section: 1
:Manual group: cloud computing
SYNOPSIS
========
::
keystone-all [-h] [--config-dir DIR] [--config-file PATH] [--debug]
[--log-config-append PATH] [--log-date-format DATE_FORMAT]
[--log-dir LOG_DIR] [--log-file PATH]
[--log-format FORMAT] [--nodebug] [--nostandard-threads]
[--nouse-syslog] [--nouse-syslog-rfc-format] [--noverbose]
[--pydev-debug-host PYDEV_DEBUG_HOST]
[--pydev-debug-port PYDEV_DEBUG_PORT] [--standard-threads]
[--syslog-log-facility SYSLOG_LOG_FACILITY] [--use-syslog]
[--use-syslog-rfc-format] [--verbose] [--version]
DESCRIPTION
===========
keystone-all starts both the service and administrative APIs in a single
process to provide catalog, authorization, and authentication services for
OpenStack.
OPTIONS
=======
-h, --help show this help message and exit
--config-dir DIR Path to a config directory to pull \*.conf files from.
This file set is sorted, so as to provide a
predictable parse order if individual options are
over-ridden. The set is parsed after the file(s)
specified via previous --config-file, arguments hence
over-ridden options in the directory take precedence.
--config-file PATH Path to a config file to use. Multiple config files
can be specified, with values in later files taking
precedence. The default files used are: None.
--debug, -d Print debugging output (set logging level to DEBUG
instead of default WARNING level).
--log-config-append PATH, --log_config PATH
The name of a logging configuration file. This file is
appended to any existing logging configuration files.
For details about logging configuration files, see the
Python logging module documentation.
--log-date-format DATE_FORMAT
Format string for %(asctime)s in log records. Default:
None .
--log-dir LOG_DIR, --logdir LOG_DIR
(Optional) The base directory used for relative --log-
file paths.
--log-file PATH, --logfile PATH
(Optional) Name of log file to output to. If no
default is set, logging will go to stdout.
--log-format FORMAT DEPRECATED. A logging.Formatter log message format
string which may use any of the available
logging.LogRecord attributes. This option is
deprecated. Please use logging_context_format_string
and logging_default_format_string instead.
--nodebug The inverse of --debug
--nostandard-threads The inverse of --standard-threads
--nouse-syslog The inverse of --use-syslog
--nouse-syslog-rfc-format
The inverse of --use-syslog-rfc-format
--noverbose The inverse of --verbose
--pydev-debug-host PYDEV_DEBUG_HOST
Host to connect to for remote debugger.
--pydev-debug-port PYDEV_DEBUG_PORT
Port to connect to for remote debugger.
--standard-threads Do not monkey-patch threading system modules.
--syslog-log-facility SYSLOG_LOG_FACILITY
Syslog facility to receive log lines.
--use-syslog Use syslog for logging. Existing syslog format is
DEPRECATED during I, and will change in J to honor
RFC5424.
--use-syslog-rfc-format
(Optional) Enables or disables syslog rfc5424 format
for logging. If enabled, prefixes the MSG part of the
syslog message with APP-NAME (RFC5424). The format
without the APP-NAME is deprecated in I, and will be
removed in J.
--verbose, -v Print more verbose output (set logging level to INFO
instead of default WARNING level).
--version show program's version number and exit
FILES
=====
None
SEE ALSO
========
* `OpenStack Keystone <http://keystone.openstack.org>`__
SOURCE
======
* Keystone source is managed in Gerrit git `Keystone <https://git.openstack.org/cgit/openstack/keystone>`__
* Keystone bugs are managed at Launchpad `Keystone <https://bugs.launchpad.net/keystone>`__

39
keystone/cmd/all.py
View File

@ -1,39 +0,0 @@
#!/usr/bin/env python
# Copyright 2013 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import os
import sys
# If ../../keystone/__init__.py exists, add ../../ to Python search path, so
# that it will override what happens to be installed in
# /usr/(local/)lib/python...
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(__file__),
os.pardir,
os.pardir,
os.pardir))
if os.path.exists(os.path.join(possible_topdir,
'keystone',
'__init__.py')):
sys.path.insert(0, possible_topdir)
from keystone.server import eventlet as eventlet_server
# entry point.
def main():
eventlet_server.run(possible_topdir)

3
keystone/cmd/manage.py
View File

@ -30,13 +30,10 @@ if os.path.exists(os.path.join(possible_topdir,
sys.path.insert(0, possible_topdir)
from keystone.cmd import cli
from keystone.common import environment
# entry point.
def main():
environment.use_stdlib()
dev_conf = os.path.join(possible_topdir,
'etc',
'keystone.conf')

76
keystone/common/config.py
View File

@ -49,6 +49,11 @@ _DEPRECATE_DII_MSG = ('The option to set domain_id_immutable to false '
'has been deprecated in the M release and will '
'be removed in the O release.')
_DEPRECATE_EVENTLET_MSG = ('Support for running keystone under eventlet has '
'been removed in the N release. These options '
'remain for backwards compatibility because they '
'are used for URL substitutions.')
FILE_OPTIONS = {
None: [
cfg.StrOpt('admin_token', secret=True, default=None,
@ -999,20 +1004,6 @@ FILE_OPTIONS = {
'assertions.'),
],
'eventlet_server': [
cfg.IntOpt('public_workers',
deprecated_name='public_workers',
deprecated_group='DEFAULT',
deprecated_for_removal=True,
help='The number of worker processes to serve the public '
'eventlet application. Defaults to number of CPUs '
'(minimum of 2).'),
cfg.IntOpt('admin_workers',
deprecated_name='admin_workers',
deprecated_group='DEFAULT',
deprecated_for_removal=True,
help='The number of worker processes to serve the admin '
'eventlet application. Defaults to number of CPUs '
'(minimum of 2).'),
cfg.StrOpt('public_bind_host',
default='0.0.0.0', # nosec : Bind to all interfaces by
# default for backwards compatibility.
@ -1021,12 +1012,14 @@ FILE_OPTIONS = {
cfg.DeprecatedOpt('public_bind_host',
group='DEFAULT'), ],
deprecated_for_removal=True,
deprecated_reason=_DEPRECATE_EVENTLET_MSG,
help='The IP address of the network interface for the '
'public service to listen on.'),
cfg.PortOpt('public_port', default=5000,
deprecated_name='public_port',
deprecated_group='DEFAULT',
deprecated_for_removal=True,
deprecated_reason=_DEPRECATE_EVENTLET_MSG,
help='The port number which the public service listens '
'on.'),
cfg.StrOpt('admin_bind_host',
@ -1037,6 +1030,7 @@ FILE_OPTIONS = {
cfg.DeprecatedOpt('admin_bind_host',
group='DEFAULT')],
deprecated_for_removal=True,
deprecated_reason=_DEPRECATE_EVENTLET_MSG,
help='The IP address of the network interface for the '
'admin service to listen on.'),
cfg.PortOpt('admin_port', default=35357,
@ -1045,60 +1039,6 @@ FILE_OPTIONS = {
deprecated_for_removal=True,
help='The port number which the admin service listens '
'on.'),
cfg.BoolOpt('wsgi_keep_alive', default=True,
help='If set to false, disables keepalives on the server; '
'all connections will be closed after serving one '
'request.'),
cfg.IntOpt('client_socket_timeout', default=900,
help='Timeout for socket operations on a client '
'connection. If an incoming connection is idle for '
'this number of seconds it will be closed. A value '
'of "0" means wait forever.'),
cfg.BoolOpt('tcp_keepalive', default=False,
deprecated_name='tcp_keepalive',
deprecated_group='DEFAULT',
deprecated_for_removal=True,
help='Set this to true if you want to enable '
'TCP_KEEPALIVE on server sockets, i.e. sockets used '
'by the Keystone wsgi server for client '
'connections.'),
cfg.IntOpt('tcp_keepidle',
default=600,
deprecated_name='tcp_keepidle',
deprecated_group='DEFAULT',
deprecated_for_removal=True,
help='Sets the value of TCP_KEEPIDLE in seconds for each '
'server socket. Only applies if tcp_keepalive is '
'true. Ignored if system does not support it.'),
],
'eventlet_server_ssl': [
cfg.BoolOpt('enable', default=False, deprecated_name='enable',
deprecated_group='ssl',
deprecated_for_removal=True,
help='Toggle for SSL support on the Keystone '
'eventlet servers.'),
cfg.StrOpt('certfile',
default='/etc/keystone/ssl/certs/keystone.pem',
deprecated_name='certfile', deprecated_group='ssl',
deprecated_for_removal=True,
help='Path of the certfile for SSL. For non-production '
'environments, you may be interested in using '
'`keystone-manage ssl_setup` to generate self-signed '
'certificates.'),
cfg.StrOpt('keyfile',
default='/etc/keystone/ssl/private/keystonekey.pem',
deprecated_name='keyfile', deprecated_group='ssl',
deprecated_for_removal=True,
help='Path of the keyfile for SSL.'),
cfg.StrOpt('ca_certs',
default='/etc/keystone/ssl/certs/ca.pem',
deprecated_name='ca_certs', deprecated_group='ssl',
deprecated_for_removal=True,
help='Path of the CA cert file for SSL.'),
cfg.BoolOpt('cert_required', default=False,
deprecated_name='cert_required', deprecated_group='ssl',
deprecated_for_removal=True,
help='Require client certificate.'),
],
}

102
keystone/common/environment/__init__.py
View File

@ -1,102 +0,0 @@
# Copyright 2013 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import functools
import os
from oslo_log import log
LOG = log.getLogger(__name__)
__all__ = ('Server', 'httplib', 'subprocess')
_configured = False
Server = None
httplib = None
subprocess = None
def configure_once(name):
"""Ensure that environment configuration is only run once.
If environment is reconfigured in the same way then it is ignored.
It is an error to attempt to reconfigure environment in a different way.
"""
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
global _configured
if _configured:
if _configured == name:
return
else:
raise SystemError("Environment has already been "
"configured as %s" % _configured)
LOG.debug("Environment configured as: %s", name)
_configured = name
return func(*args, **kwargs)
return wrapper
return decorator
@configure_once('eventlet')
def use_eventlet(monkeypatch_thread=None):
global httplib, subprocess, Server
# This must be set before the initial import of eventlet because if
# dnspython is present in your environment then eventlet monkeypatches
# socket.getaddrinfo() with an implementation which doesn't work for IPv6.
os.environ['EVENTLET_NO_GREENDNS'] = 'yes'
import eventlet
from eventlet.green import httplib as _httplib
from eventlet.green import subprocess as _subprocess
from keystone.common.environment import eventlet_server
if monkeypatch_thread is None:
monkeypatch_thread = not os.getenv('STANDARD_THREADS')
# Raise the default from 8192 to accommodate large tokens
eventlet.wsgi.MAX_HEADER_LINE = 16384
# NOTE(ldbragst): Explicitly declare what should be monkey patched and
# what shouldn't. Doing this allows for more readable code when
# understanding Eventlet in Keystone. The following is a complete list
# of what is monkey patched instead of passing all=False and then passing
# module=True to monkey patch a specific module.
eventlet.patcher.monkey_patch(os=False, select=True, socket=True,
thread=monkeypatch_thread, time=True,
psycopg=False, MySQLdb=False)
Server = eventlet_server.Server
httplib = _httplib
subprocess = _subprocess
@configure_once('stdlib')
def use_stdlib():
global httplib, subprocess
import six.moves.http_client as _httplib
import subprocess as _subprocess # nosec : This is used in .federation.idp
# and .common.openssl. See there.
httplib = _httplib
subprocess = _subprocess

212
keystone/common/environment/eventlet_server.py
View File

@ -1,212 +0,0 @@
# Copyright 2012 OpenStack Foundation
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# Copyright 2010 OpenStack Foundation
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import errno
import re
import socket
import ssl
import sys
import eventlet
import eventlet.wsgi
import greenlet
from oslo_config import cfg
from oslo_log import log
from oslo_service import service
from keystone.i18n import _LE, _LI
CONF = cfg.CONF
LOG = log.getLogger(__name__)
# The size of a pool that is used to spawn a single green thread in which
# a wsgi server is then started. The size of one is enough, because in case
# of several workers the parent process forks and each child gets a copy
# of a pool, which does not include any greenthread object as the spawn is
# done after the fork.
POOL_SIZE = 1
class EventletFilteringLogger(object):
# NOTE(morganfainberg): This logger is designed to filter out specific
# Tracebacks to limit the amount of data that eventlet can log. In the
# case of broken sockets (EPIPE and ECONNRESET), we are seeing a huge
# volume of data being written to the logs due to ~14 lines+ per traceback.
# The traceback in these cases are, at best, useful for limited debugging
# cases.
def __init__(self, logger, level=log.INFO):
self.logger = logger
self.level = level
self.regex = re.compile(r'errno (%d|%d)' %
(errno.EPIPE, errno.ECONNRESET), re.IGNORECASE)
def write(self, msg):
m = self.regex.search(msg)
if m:
self.logger.log(log.logging.DEBUG, 'Error(%s) writing to socket.',
m.group(1))
else:
self.logger.log(self.level, msg.rstrip())
class Server(service.ServiceBase):
"""Server class to manage multiple WSGI sockets and applications."""
def __init__(self, application, host=None, port=None, keepalive=False,
keepidle=None):
self.application = application
self.host = host or '0.0.0.0' # nosec : Bind to all interfaces by
# default for backwards compatibility.
self.port = port or 0
# Pool for a green thread in which wsgi server will be running
self.pool = eventlet.GreenPool(POOL_SIZE)
self.socket_info = {}
self.greenthread = None
self.do_ssl = False
self.cert_required = False
self.keepalive = keepalive
self.keepidle = keepidle
self.socket = None
def listen(self, key=None, backlog=128):
"""Create and start listening on socket.
Call before forking worker processes.
Raises Exception if this has already been called.
"""
# TODO(dims): eventlet's green dns/socket module does not actually
# support IPv6 in getaddrinfo(). We need to get around this in the
# future or monitor upstream for a fix.
# Please refer below link
# (https://bitbucket.org/eventlet/eventlet/
# src/e0f578180d7d82d2ed3d8a96d520103503c524ec/eventlet/support/
# greendns.py?at=0.12#cl-163)
info = socket.getaddrinfo(self.host,
self.port,
socket.AF_UNSPEC,
socket.SOCK_STREAM)[0]
try:
self.socket = eventlet.listen(info[-1], family=info[0],
backlog=backlog)
except EnvironmentError:
LOG.error(_LE("Could not bind to %(host)s:%(port)s"),
{'host': self.host, 'port': self.port})
raise
LOG.info(_LI('Starting %(arg0)s on %(host)s:%(port)s'),
{'arg0': sys.argv[0],
'host': self.host,
'port': self.port})
def start(self, key=None, backlog=128):
"""Run a WSGI server with the given application."""
if self.socket is None:
self.listen(key=key, backlog=backlog)
dup_socket = self.socket.dup()
if key:
self.socket_info[key] = self.socket.getsockname()
# SSL is enabled
if self.do_ssl:
if self.cert_required:
cert_reqs = ssl.CERT_REQUIRED
else:
cert_reqs = ssl.CERT_NONE
dup_socket = eventlet.wrap_ssl(dup_socket, certfile=self.certfile,
keyfile=self.keyfile,
server_side=True,
cert_reqs=cert_reqs,
ca_certs=self.ca_certs)
# Optionally enable keepalive on the wsgi socket.
if self.keepalive:
dup_socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
if self.keepidle is not None:
if hasattr(socket, 'TCP_KEEPIDLE'):
dup_socket.setsockopt(socket.IPPROTO_TCP,
socket.TCP_KEEPIDLE,
self.keepidle)
else:
LOG.warning("System does not support TCP_KEEPIDLE but "
"tcp_keepidle has been set. Ignoring.")
self.greenthread = self.pool.spawn(self._run,
self.application,
dup_socket)
def set_ssl(self, certfile, keyfile=None, ca_certs=None,
cert_required=True):
self.certfile = certfile
self.keyfile = keyfile
self.ca_certs = ca_certs
self.cert_required = cert_required
self.do_ssl = True
def stop(self):
if self.greenthread is not None:
self.greenthread.kill()
def wait(self):
"""Wait until all servers have completed running."""
try:
self.pool.waitall()
except KeyboardInterrupt: # nosec
# If CTRL-C, just break out of the loop.
pass
except greenlet.GreenletExit: # nosec
# If exiting, break out of the loop.
pass
def reset(self):
"""Required by the service interface.
The service interface is used by the launcher when receiving a
SIGHUP. The service interface is defined in
oslo_service.service.Service.
Keystone does not need to do anything here.
"""
pass
def _run(self, application, socket):
"""Start a WSGI server with a new green thread pool."""
logger = log.getLogger('eventlet.wsgi.server')
# NOTE(dolph): [eventlet_server] client_socket_timeout is required to
# be an integer in keystone.conf, but in order to make
# eventlet.wsgi.server() wait forever, we pass None instead of 0.
socket_timeout = CONF.eventlet_server.client_socket_timeout or None
try:
eventlet.wsgi.server(
socket, application, log=EventletFilteringLogger(logger),
debug=False, keepalive=CONF.eventlet_server.wsgi_keep_alive,
socket_timeout=socket_timeout)
except greenlet.GreenletExit: # nosec
# Wait until all servers have completed running
pass
except Exception:
LOG.exception(_LE('Server error'))
raise

32
keystone/common/openssl.py
View File

@ -14,11 +14,11 @@
#
import os
import subprocess # nosec : see comments in the code below
from oslo_config import cfg
from oslo_log import log
from keystone.common import environment
from keystone.common import utils
from keystone.i18n import _LI, _LE, _LW
@ -42,22 +42,22 @@ class BaseCertificateConfigure(object):
"""
def __init__(self, conf_obj, server_conf_obj, keystone_user,
def __init__(self, conf_obj, keystone_user,
keystone_group, rebuild, **kwargs):
self.conf_dir = os.path.dirname(server_conf_obj.ca_certs)
self.conf_dir = os.path.dirname(conf_obj.ca_certs)
self.use_keystone_user = keystone_user
self.use_keystone_group = keystone_group
self.rebuild = rebuild
self.ssl_config_file_name = os.path.join(self.conf_dir, "openssl.conf")
self.request_file_name = os.path.join(self.conf_dir, "req.pem")
self.ssl_dictionary = {'conf_dir': self.conf_dir,
'ca_cert': server_conf_obj.ca_certs,
'ca_cert': conf_obj.ca_certs,
'default_md': 'default',
'ssl_config': self.ssl_config_file_name,
'ca_private_key': conf_obj.ca_key,
'request_file': self.request_file_name,
'signing_key': server_conf_obj.keyfile,
'signing_cert': server_conf_obj.certfile,
'signing_key': conf_obj.keyfile,
'signing_cert': conf_obj.certfile,
'key_size': int(conf_obj.key_size),
'valid_days': int(conf_obj.valid_days),
'cert_subject': conf_obj.cert_subject}
@ -65,12 +65,12 @@ class BaseCertificateConfigure(object):
try:
# OpenSSL 1.0 and newer support default_md = default,
# older versions do not
openssl_ver = environment.subprocess.check_output( # the arguments
openssl_ver = subprocess.check_output( # nosec : the arguments
# are hardcoded and just check the openssl version
['openssl', 'version'])
if b'OpenSSL 0.' in openssl_ver:
self.ssl_dictionary['default_md'] = 'sha1'
except environment.subprocess.CalledProcessError:
except subprocess.CalledProcessError:
LOG.warning(_LW('Failed to invoke ``openssl version``, '
'assuming is v1.0 or newer'))
self.ssl_dictionary.update(kwargs)
@ -81,12 +81,12 @@ class BaseCertificateConfigure(object):
try:
# NOTE(shaleh): use check_output instead of the simpler
# `check_call()` in order to log any output from an error.
environment.subprocess.check_output( # the arguments being passed
subprocess.check_output( # nosec : the arguments being passed
# in are defined in this file and trusted to build CAs, keys
# and certs
to_exec,
stderr=environment.subprocess.STDOUT)
except environment.subprocess.CalledProcessError as e:
stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as e:
LOG.error(_LE('Command %(to_exec)s exited with %(retcode)s '
'- %(output)s'),
{'to_exec': to_exec,
@ -249,9 +249,8 @@ class ConfigurePKI(BaseCertificateConfigure):
"""
def __init__(self, keystone_user, keystone_group, rebuild=False):
super(ConfigurePKI, self).__init__(CONF.signing, CONF.signing,
keystone_user, keystone_group,
rebuild=rebuild)
super(ConfigurePKI, self).__init__(CONF.signing, keystone_user,
keystone_group, rebuild=rebuild)
class ConfigureSSL(BaseCertificateConfigure):
@ -262,9 +261,8 @@ class ConfigureSSL(BaseCertificateConfigure):
"""
def __init__(self, keystone_user, keystone_group, rebuild=False):
super(ConfigureSSL, self).__init__(CONF.ssl, CONF.eventlet_server_ssl,
keystone_user, keystone_group,
rebuild=rebuild)
super(ConfigureSSL, self).__init__(CONF.ssl, keystone_user,
keystone_group, rebuild=rebuild)
BaseCertificateConfigure.sslconfig = """

3
keystone/federation/idp.py
View File

@ -12,6 +12,7 @@
import datetime
import os
import subprocess # nosec : see comments in the code below
import uuid
from oslo_config import cfg
@ -31,7 +32,6 @@ xmldsig = importutils.try_import("saml2.xmldsig")
if not xmldsig:
xmldsig = importutils.try_import("xmldsig")
from keystone.common import environment
from keystone.common import utils
from keystone import exception
from keystone.i18n import _, _LE
@ -422,7 +422,6 @@ def _sign_assertion(assertion):
nspair={'saml': saml2.NAMESPACE,
'xmldsig': xmldsig.NAMESPACE}))
command_list.append(file_path)
subprocess = environment.subprocess
stdout = subprocess.check_output(command_list, # nosec : The contents
# of the command list are coming from
# a trusted source because the

158
keystone/server/eventlet.py
View File

@ -1,158 +0,0 @@
# Copyright 2013 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import logging
import os
import socket
from oslo_concurrency import processutils
from oslo_config import cfg
import oslo_i18n
from oslo_service import service
from oslo_service import systemd
import pbr.version
# NOTE(dstanek): i18n.enable_lazy() must be called before
# keystone.i18n._() is called to ensure it has the desired lazy lookup
# behavior. This includes cases, like keystone.exceptions, where
# keystone.i18n._() is called at import time.
oslo_i18n.enable_lazy()
from keystone.common import config
from keystone.common import environment
from keystone.common import profiler
from keystone.common import utils
from keystone.i18n import _
from keystone.server import common
from keystone.version import service as keystone_service
CONF = cfg.CONF
class ServerWrapper(object):
"""Wrap a Server with some launching info & capabilities."""
def __init__(self, server, workers):
self.server = server
self.workers = workers
def launch_with(self, launcher):
self.server.listen()
if self.workers > 1:
# Use multi-process launcher
launcher.launch_service(self.server, self.workers)
else:
# Use single process launcher
launcher.launch_service(self.server)
def create_server(conf, name, host, port, workers):
app = keystone_service.loadapp('config:%s' % conf, name)
server = environment.Server(app, host=host, port=port,
keepalive=CONF.eventlet_server.tcp_keepalive,
keepidle=CONF.eventlet_server.tcp_keepidle)
profiler.setup(name, host)
if CONF.eventlet_server_ssl.enable:
server.set_ssl(CONF.eventlet_server_ssl.certfile,
CONF.eventlet_server_ssl.keyfile,
CONF.eventlet_server_ssl.ca_certs,
CONF.eventlet_server_ssl.cert_required)
return name, ServerWrapper(server, workers)
def serve(*servers):
logging.warning(_('Running keystone via eventlet is deprecated as of Kilo '
'in favor of running in a WSGI server (e.g. mod_wsgi). '
'Support for keystone under eventlet will be removed in '
'the "M"-Release.'))
if max([server[1].workers for server in servers]) > 1:
launcher = service.ProcessLauncher(CONF)
else:
launcher = service.ServiceLauncher(CONF)
for name, server in servers:
try:
server.launch_with(launcher)
except socket.error:
logging.exception(_('Failed to start the %(name)s server') % {
'name': name})
raise
# notify calling process we are ready to serve
systemd.notify_once()
for name, server in servers:
launcher.wait()
def _get_workers(worker_type_config_opt):
# Get the value from config, if the config value is None (not set), return
# the number of cpus with a minimum of 2.
worker_count = CONF.eventlet_server.get(worker_type_config_opt)
if not worker_count:
worker_count = max(2, processutils.get_worker_count())
return worker_count
def configure_threading():
monkeypatch_thread = not CONF.standard_threads
pydev_debug_url = utils.setup_remote_pydev_debug()
if pydev_debug_url:
# in order to work around errors caused by monkey patching we have to
# set the thread to False. An explanation is here:
# http://lists.openstack.org/pipermail/openstack-dev/2012-August/
# 000794.html
monkeypatch_thread = False
environment.use_eventlet(monkeypatch_thread)
def run(possible_topdir):
dev_conf = os.path.join(possible_topdir,
'etc',
'keystone.conf')
config_files = None
if os.path.exists(dev_conf):
config_files = [dev_conf]
common.configure(
version=pbr.version.VersionInfo('keystone').version_string(),
config_files=config_files,
pre_setup_logging_fn=configure_threading)
paste_config = config.find_paste_config()
def create_servers():
admin_worker_count = _get_workers('admin_workers')
public_worker_count = _get_workers('public_workers')
servers = []
servers.append(create_server(paste_config,
'admin',
CONF.eventlet_server.admin_bind_host,
CONF.eventlet_server.admin_port,
admin_worker_count))
servers.append(create_server(paste_config,
'main',
CONF.eventlet_server.public_bind_host,
CONF.eventlet_server.public_port,
public_worker_count))
return servers
_unused, servers = common.setup_backends(
startup_application_fn=create_servers)
serve(*servers)

3
keystone/server/wsgi.py
View File

@ -28,7 +28,6 @@ oslo_i18n.enable_lazy()
from keystone.common import config
from keystone.common import environment
from keystone.server import common
from keystone.version import service as keystone_service
@ -43,8 +42,6 @@ def initialize_application(name, post_log_configured_function=lambda: None):
if CONF.debug:
CONF.log_opt_values(logging.getLogger(CONF.prog), logging.DEBUG)
environment.use_stdlib()
post_log_configured_function()
def loadapp():

8
keystone/tests/unit/core.py
View File

@ -42,12 +42,6 @@ from sqlalchemy import exc
import testtools
from testtools import testcase
# NOTE(ayoung)
# environment.use_eventlet must run before any of the code that will
# call the eventlet monkeypatching.
from keystone.common import environment # noqa
environment.use_eventlet()
from keystone import auth
from keystone.common import config
from keystone.common import dependency
@ -648,7 +642,7 @@ class TestCase(BaseTestCase):
# log module. This is not in a function or otherwise available to use
# without having a CONF object to setup logging. This should help to
# reduce the log size by limiting what we log (similar to how Keystone
# would run under mod_wsgi or eventlet).
# would run under mod_wsgi).
for pair in CONF.default_log_levels:
mod, _sep, level_name = pair.partition('=')
logger = logging.getLogger(mod)

79
keystone/tests/unit/ksfixtures/appserver.py
View File

@ -1,79 +0,0 @@
# Copyright 2013 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import absolute_import
import fixtures
from oslo_config import cfg
from paste import deploy
from keystone.common import environment
CONF = cfg.CONF
MAIN = 'main'
ADMIN = 'admin'
class AppServer(fixtures.Fixture):
"""A fixture for managing an application server instance."""
def __init__(self, config, name, cert=None, key=None, ca=None,
cert_required=False, host='127.0.0.1', port=0):
super(AppServer, self).__init__()
self.config = config
self.name = name
self.cert = cert
self.key = key
self.ca = ca
self.cert_required = cert_required
self.host = host
self.port = port
def setUp(self):
super(AppServer, self).setUp()
app = deploy.loadapp(self.config, name=self.name)
self.server = environment.Server(app, self.host, self.port)
self._setup_SSL_if_requested()
self.server.start(key='socket')
# some tests need to know the port we ran on.
self.port = self.server.socket_info['socket'][1]
self._update_config_opt()
self.addCleanup(self.server.stop)
def _setup_SSL_if_requested(self):
# TODO(dstanek): fix environment.Server to take a SSLOpts instance
# so that the params are either always set or not
if (self.cert is not None and
self.ca is not None and
self.key is not None):
self.server.set_ssl(certfile=self.cert,
keyfile=self.key,
ca_certs=self.ca,
cert_required=self.cert_required)
def _update_config_opt(self):
"""Update the config with the actual port used."""
opt_name = self._get_config_option_for_section_name()
CONF.set_override(opt_name, self.port, group='eventlet_server',
enforce_type=True)
def _get_config_option_for_section_name(self):
"""Map Paster config section names to port option names."""
return {'admin': 'admin_port', 'main': 'public_port'}[self.name]

68
keystone/tests/unit/test_cert_setup.py
View File

@ -15,12 +15,12 @@
import os
import shutil
import subprocess
import mock
from six.moves import http_client
from testtools import matchers
from keystone.common import environment
from keystone.common import openssl
from keystone import exception
from keystone.tests import unit
@ -60,14 +60,6 @@ class CertSetupTestCase(rest.RestfulTestCase):
ca_certs=ca_certs,
ca_key=ca_key,
keyfile=os.path.join(KEYDIR, 'signing_key.pem'))
self.config_fixture.config(
group='ssl',
ca_key=ca_key)
self.config_fixture.config(
group='eventlet_server_ssl',
ca_certs=ca_certs,
certfile=os.path.join(CERTDIR, 'keystone.pem'),
keyfile=os.path.join(KEYDIR, 'keystonekey.pem'))
self.config_fixture.config(group='token', provider='pkiz')
def test_can_handle_missing_certs(self):
@ -93,13 +85,6 @@ class CertSetupTestCase(rest.RestfulTestCase):
self.assertTrue(os.path.exists(CONF.signing.ca_certs))
self.assertTrue(os.path.exists(CONF.signing.keyfile))
def test_create_ssl_certs(self, rebuild=False):
ssl = openssl.ConfigureSSL(None, None, rebuild=rebuild)
ssl.run()
self.assertTrue(os.path.exists(CONF.eventlet_server_ssl.ca_certs))
self.assertTrue(os.path.exists(CONF.eventlet_server_ssl.certfile))
self.assertTrue(os.path.exists(CONF.eventlet_server_ssl.keyfile))
def test_fetch_signing_cert(self, rebuild=False):
pki = openssl.ConfigurePKI(None, None, rebuild=rebuild)
pki.run()
@ -156,17 +141,6 @@ class CertSetupTestCase(rest.RestfulTestCase):
self.assertNotEqual(cert_file1, cert_file2)
def test_ssl_certs_rebuild(self):
self.test_create_ssl_certs()
with open(CONF.eventlet_server_ssl.certfile) as f:
cert_file1 = f.read()
self.test_create_ssl_certs(rebuild=True)
with open(CONF.eventlet_server_ssl.certfile) as f:
cert_file2 = f.read()
self.assertNotEqual(cert_file1, cert_file2)
@mock.patch.object(os, 'remove')
def test_rebuild_pki_certs_remove_error(self, mock_remove):
self.test_create_pki_certs()
@ -180,19 +154,6 @@ class CertSetupTestCase(rest.RestfulTestCase):
self.assertEqual(cert_file1, cert_file2)
@mock.patch.object(os, 'remove')
def test_rebuild_ssl_certs_remove_error(self, mock_remove):
self.test_create_ssl_certs()
with open(CONF.eventlet_server_ssl.certfile) as f:
cert_file1 = f.read()
mock_remove.side_effect = OSError()
self.test_create_ssl_certs(rebuild=True)
with open(CONF.eventlet_server_ssl.certfile) as f:
cert_file2 = f.read()
self.assertEqual(cert_file1, cert_file2)
def test_create_pki_certs_twice_without_rebuild(self):
self.test_create_pki_certs()
with open(CONF.signing.certfile) as f:
@ -204,40 +165,29 @@ class CertSetupTestCase(rest.RestfulTestCase):
self.assertEqual(cert_file1, cert_file2)
def test_create_ssl_certs_twice_without_rebuild(self):
self.test_create_ssl_certs()
with open(CONF.eventlet_server_ssl.certfile) as f:
cert_file1 = f.read()
self.test_create_ssl_certs()
with open(CONF.eventlet_server_ssl.certfile) as f:
cert_file2 = f.read()
self.assertEqual(cert_file1, cert_file2)
class TestExecCommand(unit.TestCase):
@mock.patch.object(environment.subprocess.Popen, 'poll')
@mock.patch.object(subprocess.Popen, 'poll')
def test_running_a_successful_command(self, mock_poll):
mock_poll.return_value = 0
ssl = openssl.ConfigureSSL('keystone_user', 'keystone_group')
ssl = openssl.ConfigurePKI('keystone_user', 'keystone_group')
ssl.exec_command(['ls'])
@mock.patch.object(environment.subprocess, 'check_output')
@mock.patch.object(subprocess, 'check_output')
def test_running_an_invalid_command(self, mock_check_output):
cmd = ['ls']
output = 'this is the output string'
error = environment.subprocess.CalledProcessError(returncode=1,
cmd=cmd,
output=output)
error = subprocess.CalledProcessError(returncode=1,
cmd=cmd,
output=output)
mock_check_output.side_effect = error
ssl = openssl.ConfigureSSL('keystone_user', 'keystone_group')
e = self.assertRaises(environment.subprocess.CalledProcessError,
ssl = openssl.ConfigurePKI('keystone_user', 'keystone_group')
e = self.assertRaises(subprocess.CalledProcessError,
ssl.exec_command,
cmd)
self.assertThat(e.output, matchers.Equals(output))

51
keystone/tests/unit/test_ipv6.py
View File

@ -1,51 +0,0 @@
# Copyright 2012 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from oslo_config import cfg
from keystone.common import environment
from keystone.tests import unit
from keystone.tests.unit.ksfixtures import appserver
CONF = cfg.CONF
class IPv6TestCase(unit.TestCase):
def setUp(self):
self.skip_if_no_ipv6()
super(IPv6TestCase, self).setUp()
self.load_backends()
def test_ipv6_ok(self):
"""Make sure both public and admin API work with ipv6."""
paste_conf = self._paste_config('keystone')
# Verify Admin
with appserver.AppServer(paste_conf, appserver.ADMIN, host="::1"):
conn = environment.httplib.HTTPConnection(
'::1', CONF.eventlet_server.admin_port)
conn.request('GET', '/')
resp = conn.getresponse()
self.assertEqual(300, resp.status)
# Verify Public
with appserver.AppServer(paste_conf, appserver.MAIN, host="::1"):
conn = environment.httplib.HTTPConnection(
'::1', CONF.eventlet_server.public_port)
conn.request('GET', '/')
resp = conn.getresponse()
self.assertEqual(300, resp.status)

186
keystone/tests/unit/test_ssl.py
View File

@ -1,186 +0,0 @@
# Copyright 2012 OpenStack Foundation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import ssl
from oslo_config import cfg
from keystone.common import environment
from keystone.tests import unit
from keystone.tests.unit.ksfixtures import appserver
CONF = cfg.CONF
CERTDIR = unit.dirs.root('examples', 'pki', 'certs')
KEYDIR = unit.dirs.root('examples', 'pki', 'private')
CERT = os.path.join(CERTDIR, 'ssl_cert.pem')
KEY = os.path.join(KEYDIR, 'ssl_key.pem')
CA = os.path.join(CERTDIR, 'cacert.pem')
CLIENT = os.path.join(CERTDIR, 'middleware.pem')
class SSLTestCase(unit.TestCase):
def setUp(self):
super(SSLTestCase, self).setUp()
raise self.skipTest('SSL Version and Ciphers cannot be configured '
'with eventlet, some platforms have disabled '
'SSLv3. See bug 1381365.')
# NOTE(morganfainberg): It has been determined that this
# will not be fixed. These tests should be re-enabled for the full
# functional test suite when run against an SSL terminated
# endpoint. Some distributions/environments have patched OpenSSL to
# not have SSLv3 at all due to POODLE and this causes differing
# behavior depending on platform. See bug 1381365 for more information.
# NOTE(jamespage):
# Deal with more secure certificate chain verification
# introduced in python 2.7.9 under PEP-0476
# https://github.com/python/peps/blob/master/pep-0476.txt
self.context = None
if hasattr(ssl, '_create_unverified_context'):
self.context = ssl._create_unverified_context()
self.load_backends()
def get_HTTPSConnection(self, *args):
"""Simple helper to configure HTTPSConnection objects."""
if self.context:
return environment.httplib.HTTPSConnection(
*args,
context=self.context
)
else:
return environment.httplib.HTTPSConnection(*args)
def test_1way_ssl_ok(self):
"""Make sure both public and admin API work with 1-way SSL."""
paste_conf = self._paste_config('keystone')
ssl_kwargs =