add support for auth_receipts and multi-method auth

- new exception when an auth receipt is returned.
- a new method for auth receipt.
- support to existing v3 Auth plugins to add additional methods.
- Added a new MultiFactor plugin with loading support which
  takes method names as strings.

Change-Id: Ie6601a50011118e3a07be9752f747c2298ff5230
Closes-Bug: #1839748

doc/source/authentication-plugins.rst

@@ -55,6 +55,9 @@ this V3 defines a number of different
   against a V3 identity service using a username and password.
 - :py:class:`~keystoneauth1.identity.v3.TokenMethod`: Authenticate against
   a V3 identity service using an existing token.
+- :py:class:`~keystoneauth1.identity.v3.ReceiptMethod`: Authenticate against
+  a V3 identity service using an existing auth-receipt. This method has to be
+  used in conjunction with at least one other method.
 - :py:class:`~keystoneauth1.identity.v3.TOTPMethod`: Authenticate against
   a V3 identity service using Time-Based One-Time Password (TOTP).
 - :py:class:`~keystoneauth1.identity.v3.TokenlessAuth`: Authenticate against
@@ -77,7 +80,19 @@ passed to the :py:class:`~keystoneauth1.identity.v3.Auth` plugin::
     ...                project_id='projectid')
     >>> sess = session.Session(auth=auth)
 
-As in the majority of cases you will only want to use one
+You can even add additional methods to an existing auth instance after it
+has been created::
+
+    >>> totp = v3.TOTPMethod(username='user',
+    ...                      passcode='123456',
+    ...                      user_domain_name='default')
+    >>> auth.add_method(totp)
+
+Or use the :py:class:`~keystoneauth1.identity.v3.MultiFactor` helper
+plugin to do it all simply in one go, an example of whichs exists in the
+section below.
+
+For the common cases where you will only want to use one
 :py:class:`~keystoneauth1.identity.v3.AuthMethod` there are also helper
 authentication plugins for the various
 :py:class:`~keystoneauth1.identity.v3.AuthMethod` which can be used more
@@ -107,6 +122,110 @@ This will have exactly the same effect as using the single
 V3 identity plugins must use an `auth_url` that points to the root of a V3
 identity server URL, i.e.: ``http://hostname:5000/v3``.
 
+Multi-Factor with V3 Identity Plugins
+-------------------------------------
+
+The basic example of multi-factor authentication is when you supply all the
+needed auth methods up front.
+
+This can be done by building an Auth class with method instances:
+
+.. code-block:: python
+
+    from keystoneauth1 import session
+    from keystoneauth1.identity import v3
+
+    auth = v3.Auth(
+        auth_url='http://my.keystone.com:5000/v3',
+        auth_methods=[
+            v3.PasswordMethod(
+                username='user',
+                password='password',
+                user_domain_id="default",
+            ),
+            v3.TOTPMethod(
+                username='user',
+                passcode='123456',
+                user_domain_id="default",
+            )
+        ],
+        project_id='projectid',
+    )
+    sess = session.Session(auth=auth)
+
+Or by letting the helper plugin do it for you:
+
+.. code-block:: python
+
+    from keystoneauth1 import session
+    from keystoneauth1.identity import v3
+
+    auth = v3.MultiFactor(
+        auth_url='http://my.keystone.com:5000/v3',
+        auth_methods=['v3password', 'v3totp'],
+        username='user',
+        password='password',
+        passcode='123456',
+        user_domain_id="default",
+        project_id='projectid',
+    )
+    sess = session.Session(auth=auth)
+
+**Note:** The :py:class:`~keystoneauth1.identity.v3.MultiFactor` helper
+does not support auth receipts as an option in auth_methods, but one can
+be added with `auth.add_method`.
+
+When you supply just one method when multiple are needed, a
+:py:class:`~keystoneauth1.exceptions.auth.MissingAuthMethods` error will
+be raised. This can be caught, and you can infer based on the error what
+the missing methods were, and from it extract the receipt to continue
+authentication:
+
+.. code-block:: python
+
+    auth = v3.Password(auth_url='http://my.keystone.com:5000/v3',
+                       username='username',
+                       password='password',
+                       project_id='projectid',
+                       user_domain_id='default')
+    sess = session.Session(auth=auth)
+    try:
+       sess.get_token()
+    except exceptions.MissingAuthMethods as e:
+        receipt = e.receipt
+        methods = e.methods
+        required_methods = e.required_auth_methods
+
+Once you know what auth methods are needed to continue, you can extend
+the existing auth plugin with additional methods:
+
+.. code-block:: python
+
+    auth.add_method(
+        v3.TOTPMethod(
+            username='user',
+            passcode='123456',
+            user_domain_id='default',
+        )
+    )
+    sess.get_token()
+
+Or if you do not have the existing auth method, but have the receipt
+you can continue as well:
+
+.. code-block:: python
+
+    auth = v3.TOTP(
+        auth_url='http://my.keystone.com:5000/v3',
+        username='user',
+        passcode='123456',
+        user_domain_id='default',
+        project_id='projectid',
+    )
+    auth.add_method(v3.ReceiptMethod(receipt=receipt))
+    sess = session.Session(auth=auth)
+    sess.get_token()
+
 Federation
 ==========
 

keystoneauth1/exceptions/auth.py

@@ -10,8 +10,23 @@
 # License for the specific language governing permissions and limitations
 # under the License.
 
+from keystoneauth1 import _utils as utils
 from keystoneauth1.exceptions import base
 
 
 class AuthorizationFailure(base.ClientException):
     message = "Cannot authorize API client."
+
+
+class MissingAuthMethods(base.ClientException):
+    message = "Not all required auth rules were satisfied"
+
+    def __init__(self, response):
+        self.response = response
+        self.receipt = response.headers.get("Openstack-Auth-Receipt")
+        body = response.json()
+        self.methods = body['receipt']['methods']
+        self.required_auth_methods = body['required_auth_methods']
+        self.expires_at = utils.parse_isotime(body['receipt']['expires_at'])
+        message = "%s: %s" % (self.message, self.required_auth_methods)
+        super(MissingAuthMethods, self).__init__(message)

keystoneauth1/exceptions/http.py

@@ -21,6 +21,7 @@
 import inspect
 import sys
 
+from keystoneauth1.exceptions import auth
 from keystoneauth1.exceptions import base
 
 
@@ -442,6 +443,11 @@ def from_response(response, method, url):
     elif content_type.startswith("text/"):
         kwargs["details"] = response.text
 
+    # we check explicity for 401 in case of auth receipts
+    if (response.status_code == 401
+            and "Openstack-Auth-Receipt" in response.headers):
+        return auth.MissingAuthMethods(response)
+
     try:
         cls = _code_map[response.status_code]
     except KeyError:

keystoneauth1/identity/__init__.py

@@ -58,6 +58,9 @@ V3TokenlessAuth = v3.TokenlessAuth
 V3ApplicationCredential = v3.ApplicationCredential
 """See :class:`keystoneauth1.identity.v3.ApplicationCredential`"""
 
+V3MultiFactor = v3.MultiFactor
+"""See :class:`keystoneauth1.identity.v3.MultiFactor`"""
+
 __all__ = ('BaseIdentityPlugin',
            'Password',
            'Token',
@@ -70,4 +73,5 @@ __all__ = ('BaseIdentityPlugin',
            'V3OidcAccessToken',
            'V3TOTP',
            'V3TokenlessAuth',
-           'V3ApplicationCredential')
+           'V3ApplicationCredential',
+           'V3MultiFactor')

keystoneauth1/identity/v3/__init__.py

@@ -14,8 +14,10 @@ from keystoneauth1.identity.v3.application_credential import *  # noqa
 from keystoneauth1.identity.v3.base import *  # noqa
 from keystoneauth1.identity.v3.federation import *  # noqa
 from keystoneauth1.identity.v3.k2k import *  # noqa
+from keystoneauth1.identity.v3.multi_factor import *  # noqa
 from keystoneauth1.identity.v3.oidc import *  # noqa
 from keystoneauth1.identity.v3.password import *  # noqa
+from keystoneauth1.identity.v3.receipt import *  # noqa
 from keystoneauth1.identity.v3.token import *  # noqa
 from keystoneauth1.identity.v3.totp import *  # noqa
 from keystoneauth1.identity.v3.tokenless_auth import *  # noqa
@@ -47,4 +49,8 @@ __all__ = ('ApplicationCredential',
            'TOTPMethod',
            'TOTP',
 
-           'TokenlessAuth')
+           'TokenlessAuth',
+
+           'ReceiptMethod',
+
+           'MultiFactor', )

keystoneauth1/identity/v3/base.py

@@ -110,6 +110,10 @@ class Auth(BaseAuth):
         super(Auth, self).__init__(auth_url=auth_url, **kwargs)
         self.auth_methods = auth_methods
 
+    def add_method(self, method):
+        """Add an additional initialized AuthMethod instance."""
+        self.auth_methods.append(method)
+
     def get_auth_ref(self, session, **kwargs):
         headers = {'Accept': 'application/json'}
         body = {'auth': {'identity': {}}}
@@ -117,10 +121,12 @@ class Auth(BaseAuth):
         rkwargs = {}
 
         for method in self.auth_methods:
-            name, auth_data = method.get_auth_data(session,
+            name, auth_data = method.get_auth_data(
-                                                   self,
+                session, self, headers, request_kwargs=rkwargs)
-                                                   headers,
+            # NOTE(adriant): Methods like ReceiptMethod don't
-                                                   request_kwargs=rkwargs)
+            # want anything added to the request data, so they
+            # explicitly return None, which we check for.
+            if name:
                 ident.setdefault('methods', []).append(name)
                 ident[name] = auth_data
 

keystoneauth1/identity/v3/multi_factor.py

@@ -0,0 +1,59 @@
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from keystoneauth1.identity.v3 import base
+from keystoneauth1 import loading
+
+
+__all__ = ('MultiFactor', )
+
+
+class MultiFactor(base.Auth):
+    """A plugin for authenticating with multiple auth methods.
+
+    :param string auth_url: Identity service endpoint for authentication.
+    :param string auth_methods: names of the methods to authenticate with.
+    :param string trust_id: Trust ID for trust scoping.
+    :param string system_scope: System information to scope to.
+    :param string domain_id: Domain ID for domain scoping.
+    :param string domain_name: Domain name for domain scoping.
+    :param string project_id: Project ID for project scoping.
+    :param string project_name: Project name for project scoping.
+    :param string project_domain_id: Project's domain ID for project.
+    :param string project_domain_name: Project's domain name for project.
+    :param bool reauthenticate: Allow fetching a new token if the current one
+                                is going to expire. (optional) default True
+
+    Also accepts various keyword args based on which methods are specified.
+    """
+
+    def __init__(self, auth_url, auth_methods, **kwargs):
+        method_instances = []
+        method_keys = set()
+        for method in auth_methods:
+            # Using the loaders we pull the related auth method class
+            method_class = loading.get_plugin_loader(
+                method).plugin_class._auth_method_class
+            # We build some new kwargs for the method from required parameters
+            method_kwargs = {}
+            for key in method_class._method_parameters:
+                # we add them to method_keys to pop later from global kwargs
+                # rather than here as other methods may need them too
+                method_keys.add(key)
+                method_kwargs[key] = kwargs.get(key, None)
+            # We initialize the method class using just required kwargs
+            method_instances.append(method_class(**method_kwargs))
+        # We now pop all the keys used for methods as otherwise they get passed
+        # to the super class and throw errors
+        for key in method_keys:
+            kwargs.pop(key, None)
+        super(MultiFactor, self).__init__(auth_url, method_instances, **kwargs)

keystoneauth1/identity/v3/receipt.py

@@ -0,0 +1,37 @@
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from keystoneauth1.identity.v3 import base
+
+
+__all__ = ('ReceiptMethod', )
+
+
+class ReceiptMethod(base.AuthMethod):
+    """Construct an Auth plugin to continue authentication with a receipt.
+
+    :param string receipt: Receipt for authentication.
+    """
+
+    _method_parameters = ['receipt']
+
+    def get_auth_data(self, session, auth, headers, **kwargs):
+        """Add the auth receipt to the headers.
+
+        We explicitly return None to avoid being added to the request
+        methods, or body.
+        """
+        headers['Openstack-Auth-Receipt'] = self.receipt
+        return (None, None)
+
+    def get_cache_id_elements(self):
+        return {'receipt_receipt': self.receipt}

keystoneauth1/loading/_plugins/identity/v3.py

@@ -202,7 +202,11 @@ class TOTP(loading.BaseV3Loader):
         _add_common_identity_options(options)
 
         options.extend([
-            loading.Opt('passcode', secret=True, help="User's TOTP passcode"),
+            loading.Opt(
+                'passcode',
+                secret=True,
+                prompt='TOTP passcode: ',
+                help="User's TOTP passcode"),
         ])
 
         return options
@@ -294,3 +298,43 @@ class ApplicationCredential(loading.BaseV3Loader):
             raise exceptions.OptionError(m)
 
         return super(ApplicationCredential, self).load_from_options(**kwargs)
+
+
+class MultiFactor(loading.BaseV3Loader):
+
+    def __init__(self, *args, **kwargs):
+        self._methods = None
+        return super(MultiFactor, self).__init__(*args, **kwargs)
+
+    @property
+    def plugin_class(self):
+        return identity.V3MultiFactor
+
+    def get_options(self):
+        options = super(MultiFactor, self).get_options()
+
+        options.extend([
+            loading.Opt(
+                'auth_methods',
+                required=True,
+                help="Methods to authenticate with."),
+        ])
+
+        if self._methods:
+            options_dict = {o.name: o for o in options}
+            for method in self._methods:
+                method_opts = loading.get_plugin_options(method)
+                for opt in method_opts:
+                    options_dict[opt.name] = opt
+            options = list(options_dict.values())
+        return options
+
+    def load_from_options(self, **kwargs):
+        _assert_identity_options(kwargs)
+
+        if 'auth_methods' not in kwargs:
+            raise exceptions.OptionError("methods is a required option.")
+
+        self._methods = kwargs['auth_methods']
+
+        return super(MultiFactor, self).load_from_options(**kwargs)

keystoneauth1/tests/unit/identity/test_identity_v3.py

@@ -218,6 +218,22 @@ class V3IdentityPlugin(utils.TestCase):
                 "application_credential_restricted": True
             },
         }
+        self.TEST_RECEIPT_RESPONSE = {
+            "receipt": {
+                "methods": ["password"],
+                "expires_at": "2020-01-01T00:00:10.000123Z",
+                "user": {
+                    "domain": {
+                        "id": self.TEST_DOMAIN_ID,
+                        "name": self.TEST_DOMAIN_NAME,
+                    },
+                    "id": self.TEST_USER,
+                    "name": self.TEST_USER,
+                },
+                "issued_at": "2013-05-29T16:55:21.468960Z",
+            },
+            "required_auth_methods": [["password", "totp"]],
+        }
 
     def stub_auth(self, subject_token=None, **kwargs):
         if not subject_token:
@@ -226,6 +242,19 @@ class V3IdentityPlugin(utils.TestCase):
         self.stub_url('POST', ['auth', 'tokens'],
                       headers={'X-Subject-Token': subject_token}, **kwargs)
 
+    def stub_receipt(self, receipt=None, receipt_data=None, **kwargs):
+        if not receipt:
+            receipt = self.TEST_RECEIPT
+
+        if not receipt_data:
+            receipt_data = self.TEST_RECEIPT_RESPONSE
+
+        self.stub_url('POST', ['auth', 'tokens'],
+                      headers={'Openstack-Auth-Receipt': receipt},
+                      status_code=401,
+                      json=receipt_data,
+                      **kwargs)
+
     def test_authenticate_with_username_password(self):
         self.stub_auth(json=self.TEST_RESPONSE_DICT)
         a = v3.Password(self.TEST_URL,
@@ -678,3 +707,92 @@ class V3IdentityPlugin(utils.TestCase):
         s = session.Session()
         self.assertEqual(self.TEST_TOKEN, a.get_token(s))  # updates expired
         self.assertEqual(initial_cache_id, a.get_cache_id())
+
+    def test_receipt_response_is_handled(self):
+        self.stub_receipt()
+
+        a = v3.Password(
+            self.TEST_URL,
+            username=self.TEST_USER,
+            password=self.TEST_PASS,
+            user_domain_id=self.TEST_DOMAIN_ID,
+            project_id=self.TEST_TENANT_ID,
+        )
+
+        s = session.Session(a)
+        self.assertRaises(exceptions.MissingAuthMethods,
+                          s.get_auth_headers, None)
+
+    def test_authenticate_with_receipt_and_totp(self):
+        self.stub_auth(json=self.TEST_RESPONSE_DICT)
+        passcode = "123456"
+        auth = v3.TOTP(
+            self.TEST_URL,
+            username=self.TEST_USER,
+            passcode=passcode
+        )
+        auth.add_method(v3.ReceiptMethod(receipt=self.TEST_RECEIPT))
+        self.assertFalse(auth.has_scope_parameters)
+        s = session.Session(auth=auth)
+
+        self.assertEqual({"X-Auth-Token": self.TEST_TOKEN},
+                         s.get_auth_headers())
+
+        # NOTE(adriant): Here we are confirming the receipt data isn't in the
+        # body or listed as a method
+        req = {
+            "auth": {
+                "identity": {
+                    "methods": ["totp"],
+                    "totp": {"user": {
+                        "name": self.TEST_USER, "passcode": passcode}},
+                }
+            }
+        }
+
+        self.assertRequestBodyIs(json=req)
+        self.assertRequestHeaderEqual("Openstack-Auth-Receipt",
+                                      self.TEST_RECEIPT)
+        self.assertRequestHeaderEqual("Content-Type", "application/json")
+        self.assertRequestHeaderEqual("Accept", "application/json")
+        self.assertEqual(s.auth.auth_ref.auth_token, self.TEST_TOKEN)
+
+    def test_authenticate_with_multi_factor(self):
+        self.stub_auth(json=self.TEST_RESPONSE_DICT)
+        passcode = "123456"
+        auth = v3.MultiFactor(
+            self.TEST_URL,
+            auth_methods=['v3password', 'v3totp'],
+            username=self.TEST_USER,
+            password=self.TEST_PASS,
+            passcode=passcode,
+            user_domain_id=self.TEST_DOMAIN_ID,
+            project_id=self.TEST_TENANT_ID,
+        )
+        self.assertTrue(auth.has_scope_parameters)
+        s = session.Session(auth=auth)
+
+        self.assertEqual({"X-Auth-Token": self.TEST_TOKEN},
+                         s.get_auth_headers())
+
+        req = {
+            "auth": {
+                "identity": {
+                    "methods": ["password", "totp"],
+                    "totp": {"user": {
+                        "name": self.TEST_USER, "passcode": passcode,
+                        'domain': {'id': self.TEST_DOMAIN_ID}
+                    }},
+                    'password': {'user': {
+                        'name': self.TEST_USER, 'password': self.TEST_PASS,
+                        'domain': {'id': self.TEST_DOMAIN_ID}
+                    }},
+                },
+                'scope': {'project': {'id': self.TEST_TENANT_ID}}
+            }
+        }
+
+        self.assertRequestBodyIs(json=req)
+        self.assertRequestHeaderEqual("Content-Type", "application/json")
+        self.assertRequestHeaderEqual("Accept", "application/json")
+        self.assertEqual(s.auth.auth_ref.auth_token, self.TEST_TOKEN)

keystoneauth1/tests/unit/loading/test_v3.py

@@ -427,3 +427,62 @@ class V3ApplicationCredentialTests(utils.TestCase):
                           application_credential_id=uuid.uuid4().hex,
                           username=uuid.uuid4().hex,
                           user_domain_id=uuid.uuid4().hex)
+
+
+class MultiFactorTests(utils.TestCase):
+
+    def setUp(self):
+        super(MultiFactorTests, self).setUp()
+
+        self.auth_url = uuid.uuid4().hex
+
+    def create(self, **kwargs):
+        kwargs.setdefault('auth_url', self.auth_url)
+        loader = loading.get_plugin_loader('v3multifactor')
+        return loader.load_from_options(**kwargs)
+
+    def test_password_and_totp(self):
+        username = uuid.uuid4().hex
+        password = uuid.uuid4().hex
+        user_domain_id = uuid.uuid4().hex
+        # passcode is 6 digits
+        passcode = ''.join(str(random.randint(0, 9)) for x in range(6))
+        project_name = uuid.uuid4().hex
+        project_domain_id = uuid.uuid4().hex
+
+        p = self.create(
+            auth_methods=['v3password', 'v3totp'],
+            username=username,
+            password=password,
+            user_domain_id=user_domain_id,
+            project_name=project_name,
+            project_domain_id=project_domain_id,
+            passcode=passcode)
+
+        password_method = p.auth_methods[0]
+        totp_method = p.auth_methods[1]
+
+        self.assertEqual(username, password_method.username)
+        self.assertEqual(user_domain_id, password_method.user_domain_id)
+        self.assertEqual(password, password_method.password)
+
+        self.assertEqual(username, totp_method.username)
+        self.assertEqual(user_domain_id, totp_method.user_domain_id)
+        self.assertEqual(passcode, totp_method.passcode)
+
+        self.assertEqual(project_name, p.project_name)
+        self.assertEqual(project_domain_id, p.project_domain_id)
+
+    def test_without_methods(self):
+        self.assertRaises(exceptions.OptionError,
+                          self.create,
+                          username=uuid.uuid4().hex,
+                          passcode=uuid.uuid4().hex)
+
+    def test_without_user_domain_for_password(self):
+        self.assertRaises(exceptions.OptionError,
+                          self.create,
+                          methods=['v3password'],
+                          username=uuid.uuid4().hex,
+                          project_name=uuid.uuid4().hex,
+                          project_domain_id=uuid.uuid4().hex)

keystoneauth1/tests/unit/utils.py

@@ -30,6 +30,7 @@ class TestCase(testtools.TestCase):
     TEST_ROLE_ID = uuid.uuid4().hex
     TEST_TENANT_ID = uuid.uuid4().hex
     TEST_TENANT_NAME = uuid.uuid4().hex
+    TEST_RECEIPT = uuid.uuid4().hex
     TEST_TOKEN = uuid.uuid4().hex
     TEST_TRUST_ID = uuid.uuid4().hex
     TEST_USER = uuid.uuid4().hex

releasenotes/notes/bug-1839748-5d8dfc99c43aaefc.yaml

@@ -0,0 +1,10 @@
+---
+features:
+  - |
+    [`bug 1839748 <https://bugs.launchpad.net/keystoneauth/+bug/1839748>`_]
+    Keystoneauth now supports MFA authentication and Auth Receipts.
+    Responses from Keystone containing and auth receipt will now
+    raise a ``MissingAuthMethods`` exception which will contain the
+    auth receipt itself, and information about the missing methods.
+    There are now also ways to easily do more than one method when
+    authenticating to Keystone and those have been documented.

setup.cfg

@@ -58,6 +58,7 @@ keystoneauth1.plugin =
     v3adfspassword = keystoneauth1.extras._saml2._loading:ADFSPassword
     v3samlpassword = keystoneauth1.extras._saml2._loading:Saml2Password
     v3applicationcredential = keystoneauth1.loading._plugins.identity.v3:ApplicationCredential
+    v3multifactor = keystoneauth1.loading._plugins.identity.v3:MultiFactor
 
 [build_sphinx]
 source-dir = doc/source