Add MicroversionMiddleware

This change imports (with slight modification) the
MicroversionMiddleware created for the placement service.

For the time being the middleware relies on WebOb as that is
what it originally relied on. If there are requirements to do
otherwise, later, we can make adjustments then.

The README is updated with limited usage instructions.

Gabbi tests are added, requiring adjustments to the test configuration,
so the opportunity was taken to switch to using stestr instead of testr.

Though arguably "functional" tests because they are not strictly unit
tests, no distinction is made in the tests directory, for now, as it
is all nice and fast and we'd like to keep it that way.

Change-Id: I9d27b6fad35f7aea0085f08c0353f8e8acdb8b73

.gitignore

@@ -60,3 +60,6 @@ target/
 
 #Ipython Notebook
 .ipynb_checkpoints
+
+.stestr
+cover

.stestr.conf

@@ -0,0 +1,6 @@
+[DEFAULT]
+test_path=./microversion_parse/tests
+top_dir=./
+# This regex ensures each yaml file used by gabbi is run in only one
+# process.
+group_regex=microversion_parse\.tests\.test_middleware(?:\.|_)([^_]+)

.testr.conf

@@ -1,4 +0,0 @@
-[DEFAULT]
-test_command=${PYTHON:-python} -m subunit.run discover -t . ${OS_TEST_PATH:-microversion_parse} $LISTOPT $IDOPTION
-test_id_option=--load-list $IDFILE
-test_list_option=--list

README.rst

@@ -1,10 +1,13 @@
 microversion_parse
 ==================
 
-A small set of functions to manage OpenStack microversion headers that can
+A small set of functions to manage OpenStack `microversion`_ headers that can
 be used in middleware, application handlers and decorators to effectively
 manage microversions.
 
+Also included, in the ``middleware`` module, is a ``MicroversionMiddleware``
+that will process incoming microversion headers.
+
 get_version
 -----------
 
@@ -71,3 +74,42 @@ a microversion for a given service type in a collection of headers::
 If the found version is not in versions_list a ``ValueError`` is raised.
 
 Note that ``extract_version`` does not support ``legacy_headers``.
+
+MicroversionMiddleware
+----------------------
+
+A WSGI middleware that can wrap an application that needs to be microversion
+aware. The application will get a WSGI environ with a
+'SERVICE_TYPE.microversion' key that has a value of the microversion found at
+an 'openstack-api-version' header that matches SERVICE_TYPE.  If no header is
+found, the minimum microversion will be set. If the special keyword 'latest' is
+used, the maximum microversion will be set.
+
+If the requested microversion is not available a 406 response is returned.
+
+If there is an error parsing a provided header, a 400 response is returned.
+
+Otherwise the application is called.
+
+The middleware is configured when it is created. Three parameters are required:
+
+app
+  The next WSGI middleware or application in the stack.
+
+service_type
+  The service type of the application, used to identify microversion headers.
+
+versions_list
+  An ordered list of legitimate microversions (as strings) for the application.
+  It's assumed that any application that is using microversions will have such
+  a list for its own housekeeping and documentation.
+
+For example::
+
+    def app():
+        app = middleware.MicroversionMiddleware(
+            MyWSGIApp(), 'cats', ['1.0', '1.1', '1.2'])
+        return app
+
+
+.. _microversion: http://specs.openstack.org/openstack/api-wg/guidelines/microversion_specification.html

microversion_parse/middleware.py

@@ -0,0 +1,83 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""WSGI middleware for getting microversion info."""
+
+import webob
+import webob.dec
+
+import microversion_parse
+
+
+class MicroversionMiddleware(object):
+    """WSGI middleware for getting microversion info.
+
+    The application will get a WSGI environ with a
+    'SERVICE_TYPE.microversion' key that has a value of the microversion
+    found at an 'openstack-api-version' header that matches SERVICE_TYPE. If
+    no header is found, the minimum microversion will be set. If the
+    special keyword 'latest' is used, the maximum microversion will be
+    set.
+
+    If the requested microversion is not available a 406 response is
+    returned.
+
+    If there is an error parsing a provided header, a 400 response is
+    returned.
+
+    Otherwise the application is called.
+    """
+
+    def __init__(self, application, service_type, versions):
+        """Create the WSGI middleware.
+
+        :param application: The application hosting the service.
+        :param service_type: The service type (entry in keystone catalog)
+                             of the application.
+        :param versions: An ordered list of legitimate versions for the
+                         application.
+        """
+        self.application = application
+        self.service_type = service_type
+        self.microversion_environ = '%s.microversion' % service_type
+        self.versions = versions
+
+    @webob.dec.wsgify
+    def __call__(self, req):
+        try:
+            microversion = microversion_parse.extract_version(
+                req.headers, self.service_type, self.versions)
+        # TODO(cdent): These error response are not formatted according to
+        # api-sig guidelines.
+        except ValueError as exc:
+            raise webob.exc.HTTPNotAcceptable(
+                ('Invalid microversion: %(error)s') % {'error': exc})
+        except TypeError as exc:
+            raise webob.exc.HTTPBadRequest(
+                ('Invalid microversion: %(error)s') % {'error': exc})
+
+        req.environ[self.microversion_environ] = microversion
+        microversion_header = '%s %s' % (self.service_type, microversion)
+        standard_header = microversion_parse.STANDARD_HEADER
+
+        try:
+            response = req.get_response(self.application)
+        except webob.exc.HTTPError as exc:
+            # If there was an HTTPError in the application we still need
+            # to send the microversion header, so add the header and
+            # re-raise the exception.
+            exc.headers.add(standard_header, microversion_header)
+            raise exc
+
+        response.headers.add(standard_header, microversion_header)
+        response.headers.add('vary', standard_header)
+        return response

microversion_parse/tests/gabbits/middleware.yaml

@@ -0,0 +1,75 @@
+# Tests that the middleware does microversioning as we expect
+# The min version of the service is 1.0, the max is 1.2,
+# the service type is "cats" (because the internet is for cats).
+
+defaults:
+    request_headers:
+        # We must guard against webob requiring an accept header.
+        # We don't want to do this in the middleware itself as
+        # we don't know what the application would prefer as a
+        # default.
+        accept: application/json
+
+tests:
+
+- name: min default
+  GET: /good
+  response_headers:
+      openstack-api-version: cats 1.0
+
+- name: max latest
+  GET: /good
+  request_headers:
+      openstack-api-version: cats latest
+  response_headers:
+      openstack-api-version: cats 1.2
+
+- name: explict
+  GET: /good
+  request_headers:
+      openstack-api-version: cats 1.1
+  response_headers:
+      openstack-api-version: cats 1.1
+
+- name: out of range
+  GET: /good
+  request_headers:
+      openstack-api-version: cats 1.9
+  status: 406
+  response_strings:
+      - Unacceptable version header
+
+- name: invalid format
+  GET: /good
+  request_headers:
+      openstack-api-version: cats 1.9.5
+  status: 400
+  response_strings:
+      - invalid literal
+
+- name: different service
+  desc: end up with default microversion
+  GET: /good
+  request_headers:
+      openstack-api-version: dogs 1.9
+  response_headers:
+      openstack-api-version: cats 1.0
+
+- name: multiple services
+  GET: /good
+  request_headers:
+      openstack-api-version: dogs 1.9, cats 1.1
+  response_headers:
+      openstack-api-version: cats 1.1
+
+- name: header present on exception
+  GET: /bad
+  request_headers:
+      openstack-api-version: dogs 1.9, cats 1.1
+  response_headers:
+      openstack-api-version: cats 1.1
+  status: 404
+  response_strings:
+      - /bad not found
+
+

microversion_parse/tests/gabbits/simple.yaml

@@ -0,0 +1,15 @@
+# tests that the SimpleWSGI app is present
+
+tests:
+
+- name: get good
+  GET: /good
+  status: 200
+  response_strings:
+      - good
+
+- name: get bad
+  GET: /bad
+  status: 404
+  response_strings:
+      - not found

microversion_parse/tests/test_middleware.py

@@ -0,0 +1,57 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# The microversion_parse middlware is tests using gabbi to run real
+# http requests through it. To do that, we need a simple WSGI
+# application running under wsgi-intercept (handled by gabbi).
+
+import os
+
+from gabbi import driver
+import webob
+
+from microversion_parse import middleware
+
+
+TESTS_DIR = 'gabbits'
+SERVICE_TYPE = 'cats'
+VERSIONS = [
+    '1.0',  # initial version
+    '1.1',  # now with kittens
+    '1.2',  # added breeds
+]
+
+
+class SimpleWSGI(object):
+    """A WSGI application that can be contiained within a middlware."""
+
+    def __call__(self, environ, start_response):
+        path_info = environ['PATH_INFO']
+        if path_info == '/good':
+            start_response('200 OK', [('content-type', 'text/plain')])
+            return [b'good']
+
+        raise webob.exc.HTTPNotFound('%s not found' % path_info)
+
+
+def app():
+    app = middleware.MicroversionMiddleware(
+            SimpleWSGI(), SERVICE_TYPE, VERSIONS)
+    return app
+
+
+def load_tests(loader, tests, pattern):
+    """Provide a TestSuite to the discovery process."""
+    test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
+    return driver.build_tests(
+        test_dir, loader, test_loader_name=__name__, intercept=app)

requirements.txt

@@ -1 +1 @@
-
+WebOb>=1.2.3 # MIT

test-requirements.txt

@@ -2,6 +2,6 @@ hacking>=0.10.2,<0.11 # Apache-2.0
 coverage>=3.6 # Apache-2.0
 sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 # BSD
 oslosphinx!=3.4.0,>=2.5.0 # Apache-2.0
-testrepository>=0.0.18 # Apache-2.0/BSD
+stestr>=1.0.0 # Apache-2.0
 testtools>=1.4.0 # MIT
-WebOb>=1.2.3 # MIT
+gabbi>=1.35.0 # Apache-2.0

tox.ini

@@ -3,15 +3,14 @@ minversion = 2.0
 skipsdist = True
 # If you want pypy or pypy3, do 'tox -epypy,pypy3', it might work!
 # And you can get coverage with 'tox -ecover'.
-envlist = py27,py34,py35,pep8
+envlist = py27,py36,py35,pep8
 
 [testenv]
 deps = -r{toxinidir}/requirements.txt
        -r{toxinidir}/test-requirements.txt
 install_command = pip install -U {opts} {packages}
-setenv = OS_TEST_PATH=microversion_parse/tests/
 usedevelop = True
-commands = python setup.py testr --testr-args="{posargs}"
+commands = stestr run {posargs}
 
 [testenv:venv]
 deps = -r{toxinidir}/requirements.txt
@@ -25,7 +24,15 @@ commands =
     flake8
 
 [testenv:cover]
-commands = python setup.py testr --coverage --testr-args="{posargs}"
+setenv = PYTHON=coverage run --source microversion_parse --parallel-mode
+commands =
+    coverage erase
+    find . -type f -name "*.pyc" -delete
+    stestr run {posargs}
+    coverage combine
+    coverage html -d cover
+whitelist_externals =
+    find
 
 [testenv:docs]
 commands =