Add new ssl header into Listener for client certificate

Add new ssl headers:
'X-SSL-Client-Verify', 'X-SSL-Client-Has-Cert', 'X-SSL-Client-DN',
'X-SSL-Client-CN', 'X-SSL-Issuer', 'X-SSL-Client-SHA1',
'X-SSL-Client-Not-Before', 'X-SSL-Client-Not-After'

Allow users to send to the backend with multiple choices when
tls_terminated is enabled for client certificate.

Story: 2002165
Task: 20020

Change-Id: I112936ee85c9e0dcfb87b962176ba7d623989a30

api-ref/source/v2/listener.inc

@@ -172,25 +172,74 @@ Supported HTTP Header Insertions
    header insertions.
 
 
-+-------------------+--------+------------------------------------------------+
++-------------------------+--------+------------------------------------------------+
-| Key               | Value  | Description                                    |
+| Key                     | Value  | Description                                    |
-+===================+========+================================================+
++=========================+========+================================================+
-| X-Forwarded-For   | string | When "``true``" a ``X-Forwarded-For`` header   |
+| X-Forwarded-For         | string | When "``true``" a ``X-Forwarded-For`` header   |
-|                   |        | is inserted into the request to the backend    |
+|                         |        | is inserted into the request to the backend    |
-|                   |        | ``member`` that specifies the client IP        |
+|                         |        | ``member`` that specifies the client IP        |
-|                   |        | address.                                       |
+|                         |        | address.                                       |
-+-------------------+--------+------------------------------------------------+
++-------------------------+--------+------------------------------------------------+
-| X-Forwarded-Port  | string | When "``true``" a ``X-Forwarded-Port`` header  |
+| X-Forwarded-Port        | string | When "``true``" a ``X-Forwarded-Port`` header  |
-|                   |        | is inserted into the request to the backend    |
+|                         |        | is inserted into the request to the backend    |
-|                   |        | ``member`` that specifies the listener port.   |
+|                         |        | ``member`` that specifies the listener port.   |
-+-------------------+--------+------------------------------------------------+
++-------------------------+--------+------------------------------------------------+
-| X-Forwarded-Proto | string | When "``true``" a ``X-Forwarded-Proto`` header |
+| X-Forwarded-Proto       | string | When "``true``" a ``X-Forwarded-Proto`` header |
-|                   |        | is inserted into the request to the backend    |
+|                         |        | is inserted into the request to the backend    |
-|                   |        | ``member``. HTTP for the HTTP listener         |
+|                         |        | ``member``. HTTP for the HTTP listener         |
-|                   |        | protocol type, HTTPS for the TERMINATED_HTTPS  |
+|                         |        | protocol type, HTTPS for the TERMINATED_HTTPS  |
-|                   |        | listener protocol type.                        |
+|                         |        | listener protocol type.                        |
-|                   |        | **New in version 2.1**                         |
+|                         |        | **New in version 2.1**                         |
-+-------------------+--------+------------------------------------------------+
++-------------------------+--------+------------------------------------------------+
+| X-SSL-Client-Verify     | string | When "``true``" a ``X-SSL-Client-Verify``      |
+|                         |        | header is inserted into the request to the     |
+|                         |        | backend ``member`` that contains 0 if the      |
+|                         |        | client authentication was successful, or an    |
+|                         |        | result error number greater than 0 that align  |
+|                         |        | to the openssl veryify error codes.            |
++-------------------------+--------+------------------------------------------------+
+| X-SSL-Client-Has-Cert   | string | When "``true``" a ``X-SSL-Client-Has-Cert``    |
+|                         |        | header is inserted into the request to the     |
+|                         |        | backend ``member`` that is ''true'' if a client|
+|                         |        | authentication certificate was presented, and  |
+|                         |        | ''false'' if not. Does not indicate validity.  |
++-------------------------+--------+------------------------------------------------+
+| X-SSL-Client-DN         | string | When "``true``" a ``X-SSL-Client-DN`` header   |
+|                         |        | is inserted into the request to the backend    |
+|                         |        | ``member`` that contains the full              |
+|                         |        | Distinguished Name of the certificate          |
+|                         |        | presented by the client.                       |
++-------------------------+--------+------------------------------------------------+
+| X-SSL-Client-CN         | string | When "``true``" a ``X-SSL-Client-CN`` header   |
+|                         |        | is inserted into the request to the backend    |
+|                         |        | ``member`` that contains the Common Name from  |
+|                         |        | the full Distinguished Name of the certificate |
+|                         |        | presented by the client.                       |
++-------------------------+--------+------------------------------------------------+
+| X-SSL-Issuer            | string | When "``true``" a ``X-SSL-Issuer`` header is   |
+|                         |        | inserted into the request to the backend       |
+|                         |        | ``member`` that contains the full              |
+|                         |        | Distinguished Name of the client certificate   |
+|                         |        | issuer.                                        |
++-------------------------+--------+------------------------------------------------+
+| X-SSL-Client-SHA1       | string | When "``true``" a ``X-SSL-Client-SHA1`` header |
+|                         |        | is inserted into the request to the backend    |
+|                         |        | ``member`` that contains the SHA-1 fingerprint |
+|                         |        | of the certificate presented by the client in  |
+|                         |        | hex string format.                             |
++-------------------------+--------+------------------------------------------------+
+| X-SSL-Client-Not-Before | string | When "``true``" a ``X-SSL-Client-Not-Before``  |
+|                         |        | header is inserted into the request to the     |
+|                         |        | backend ``member`` that contains the start     |
+|                         |        | date presented by the client as a formatted    |
+|                         |        | string YYMMDDhhmmss[Z].                        |
++-------------------------+--------+------------------------------------------------+
+| X-SSL-Client-Not-After  | string | When "``true``" a ``X-SSL-Client-Not-After``   |
+|                         |        | header is inserted into the request to the     |
+|                         |        | backend ``member`` that contains the end date  |
+|                         |        | presented by the client as a formatted string  |
+|                         |        | YYMMDDhhmmss[Z].                               |
++-------------------------+--------+------------------------------------------------+
 
 Request Example
 ----------------

octavia/api/v2/controllers/listener.py

@@ -205,6 +205,27 @@ class ListenersController(base.BaseController):
         return (self._has_tls_container_refs(listener_dict) or
                 listener_dict.get('insert_headers'))
 
+    def _validate_insert_headers(self, insert_header_list, listener_protocol):
+        if list(set(insert_header_list) - (
+                set(constants.SUPPORTED_HTTP_HEADERS +
+                    constants.SUPPORTED_SSL_HEADERS))):
+            raise exceptions.InvalidOption(
+                value=insert_header_list,
+                option='insert_headers')
+        if not listener_protocol == constants.PROTOCOL_TERMINATED_HTTPS:
+            is_matched = len(
+                constants.SUPPORTED_SSL_HEADERS) > len(
+                list(set(constants.SUPPORTED_SSL_HEADERS) - set(
+                    insert_header_list)))
+            if is_matched:
+                headers = []
+                for header_name in insert_header_list:
+                    if header_name in constants.SUPPORTED_SSL_HEADERS:
+                        headers.append(header_name)
+                raise exceptions.InvalidOption(
+                    value=headers,
+                    option=('%s protocol listener.' % listener_protocol))
+
     def _validate_create_listener(self, lock_session, listener_dict):
         """Validate listener for wrong protocol or duplicate listeners
 
@@ -212,13 +233,9 @@ class ListenersController(base.BaseController):
         """
         listener_protocol = listener_dict.get('protocol')
 
-        if (listener_dict and
+        if listener_dict and listener_dict.get('insert_headers'):
-            listener_dict.get('insert_headers') and
+            self._validate_insert_headers(
-            list(set(listener_dict['insert_headers'].keys()) -
+                listener_dict['insert_headers'].keys(), listener_protocol)
-                 set(constants.SUPPORTED_HTTP_HEADERS))):
-            raise exceptions.InvalidOption(
-                value=listener_dict.get('insert_headers'),
-                option='insert_headers')
 
         # Check for UDP compatibility
         if (listener_protocol == constants.PROTOCOL_UDP and
@@ -441,6 +458,10 @@ class ListenersController(base.BaseController):
                 "container reference.") %
                 listener.client_authentication)
 
+        if listener.insert_headers:
+            self._validate_insert_headers(
+                list(listener.insert_headers.keys()), db_listener.protocol)
+
         sni_containers = listener.sni_container_refs or []
         tls_refs = [sni for sni in sni_containers]
         if listener.default_tls_container_ref:

octavia/common/constants.py

@@ -466,6 +466,12 @@ SUPPORTED_HTTP_HEADERS = ['X-Forwarded-For',
                           'X-Forwarded-Port',
                           'X-Forwarded-Proto']
 
+# List of SSL headers for client certificate
+SUPPORTED_SSL_HEADERS = ['X-SSL-Client-Verify', 'X-SSL-Client-Has-Cert',
+                         'X-SSL-Client-DN', 'X-SSL-Client-CN',
+                         'X-SSL-Issuer', 'X-SSL-Client-SHA1',
+                         'X-SSL-Client-Not-Before', 'X-SSL-Client-Not-After']
+
 FLOW_DOC_TITLES = {'AmphoraFlows': 'Amphora Flows',
                    'LoadBalancerFlows': 'Load Balancer Flows',
                    'ListenerFlows': 'Listener Flows',

octavia/common/jinja/haproxy/templates/macros.j2

@@ -286,6 +286,40 @@ backend {{ pool.id }}
     http-request set-header X-Forwarded-Proto https
         {% endif %}
     {% endif %}
+    {% if listener.protocol.lower() == constants.PROTOCOL_TERMINATED_HTTPS.lower() %}
+        {% if listener.insert_headers.get('X-SSL-Client-Verify',
+                                          'False').lower() == 'true' %}
+    http-request set-header X-SSL-Client-Verify %[ssl_c_verify]
+        {% endif %}
+        {% if listener.insert_headers.get('X-SSL-Client-Has-Cert',
+                                          'False').lower() == 'true' %}
+    http-request set-header X-SSL-Client-Has-Cert %[ssl_c_used]
+        {% endif %}
+        {% if listener.insert_headers.get('X-SSL-Client-DN',
+                                          'False').lower() == 'true' %}
+    http-request set-header X-SSL-Client-DN %{+Q}[ssl_c_s_dn]
+        {% endif %}
+        {% if listener.insert_headers.get('X-SSL-Client-CN',
+                                          'False').lower() == 'true' %}
+    http-request set-header X-SSL-Client-CN %{+Q}[ssl_c_s_dn(cn)]
+        {% endif %}
+        {% if listener.insert_headers.get('X-SSL-Issuer',
+                                          'False').lower() == 'true' %}
+    http-request set-header X-SSL-Issuer %{+Q}[ssl_c_i_dn]
+        {% endif %}
+        {% if listener.insert_headers.get('X-SSL-Client-SHA1',
+                                          'False').lower() == 'true' %}
+    http-request set-header X-SSL-Client-SHA1 %{+Q}[ssl_c_sha1,hex]
+        {% endif %}
+        {% if listener.insert_headers.get('X-SSL-Client-Not-Before',
+                                          'False').lower() == 'true' %}
+    http-request set-header X-SSL-Client-Not-Before %{+Q}[ssl_c_notbefore]
+        {% endif %}
+        {% if listener.insert_headers.get('X-SSL-Client-Not-After',
+                                          'False').lower() == 'true' %}
+    http-request set-header X-SSL-Client-Not-After %{+Q}[ssl_c_notafter]
+        {% endif %}
+    {% endif %}
     {% if listener.connection_limit is defined %}
     fullconn {{ listener.connection_limit }}
     {% endif %}

octavia/tests/functional/api/v2/test_listener.py

@@ -2031,7 +2031,11 @@ class TestListener(base.BaseAPITest):
         get_listener = self.get(listener_path).json['listener']
         self.assertEqual([], get_listener.get('sni_container_refs'))
 
-    def test_create_with_valid_insert_headers(self):
+    # TODO(johnsom) Fix this when there is a noop certificate manager
+    @mock.patch('octavia.common.tls_utils.cert_parser.load_certificates_data')
+    def test_create_with_valid_insert_headers(self, mock_cert_data):
+        cert1 = data_models.TLSContainer(certificate='cert 1')
+        mock_cert_data.return_value = {'tls_cert': cert1}
         lb_listener = {'protocol': 'HTTP',
                        'protocol_port': 80,
                        'loadbalancer_id': self.lb_id,
@@ -2039,14 +2043,98 @@ class TestListener(base.BaseAPITest):
         body = self._build_body(lb_listener)
         self.post(self.LISTENERS_PATH, body, status=201)
 
+        # test client certificate http headers
+        self.set_lb_status(self.lb_id)
+        header = {}
+        for name in constants.SUPPORTED_SSL_HEADERS:
+            header[name] = 'true'
+        lb_listener = {'protocol': constants.PROTOCOL_TERMINATED_HTTPS,
+                       'protocol_port': 1801,
+                       'loadbalancer_id': self.lb_id,
+                       'insert_headers': header,
+                       'default_tls_container_ref': uuidutils.generate_uuid()}
+        body = self._build_body(lb_listener)
+        self.post(self.LISTENERS_PATH, body, status=201)
+
     def test_create_with_bad_insert_headers(self):
-        lb_listener = {'protocol': 'HTTP',
+        lb_listener = {'protocol': constants.PROTOCOL_HTTP,
                        'protocol_port': 80,
                        'loadbalancer_id': self.lb_id,
                        'insert_headers': {'X-Forwarded-Four': 'true'}}
         body = self._build_body(lb_listener)
         self.post(self.LISTENERS_PATH, body, status=400)
 
+        # test client certificate http headers
+        for name in constants.SUPPORTED_SSL_HEADERS:
+            header = {}
+            header[name] = 'true'
+            lb_listener['insert_headers'] = header
+            body = self._build_body(lb_listener)
+            listener = self.post(self.LISTENERS_PATH, body, status=400).json
+            self.assertIn('{0} is not a valid option for {1}'.format(
+                [name],
+                '%s protocol listener.' % constants.PROTOCOL_HTTP),
+                listener.get('faultstring'))
+
+    @mock.patch('octavia.common.tls_utils.cert_parser.load_certificates_data')
+    def test_update_with_valid_insert_headers(self, mock_cert_data):
+        cert1 = data_models.TLSContainer(certificate='cert 1')
+        mock_cert_data.return_value = {'tls_cert': cert1}
+        listener = self.create_listener(
+            constants.PROTOCOL_HTTP, 80, self.lb_id)
+        self.set_lb_status(self.lb_id)
+        new_listener = self._build_body(
+            {'insert_headers': {'X-Forwarded-For': 'true'}})
+        listener_path = self.LISTENER_PATH.format(
+            listener_id=listener['listener'].get('id'))
+        update_listener = self.put(
+            listener_path, new_listener, status=200).json
+        self.assertNotEqual(
+            listener[self.root_tag]['insert_headers'],
+            update_listener[self.root_tag]['insert_headers'])
+
+        self.set_lb_status(self.lb_id)
+        # test client certificate http headers
+        cert1_id = uuidutils.generate_uuid()
+        listener = self.create_listener(
+            constants.PROTOCOL_TERMINATED_HTTPS, 443, self.lb_id,
+            default_tls_container_ref=cert1_id)
+        self.set_lb_status(self.lb_id)
+        header = {}
+        for name in constants.SUPPORTED_SSL_HEADERS:
+            header[name] = 'true'
+        new_listener[self.root_tag]['insert_headers'] = header
+        listener_path = self.LISTENER_PATH.format(
+            listener_id=listener['listener'].get('id'))
+        update_listener = self.put(
+            listener_path, new_listener, status=200).json
+        self.assertNotEqual(
+            listener[self.root_tag]['insert_headers'],
+            update_listener[self.root_tag]['insert_headers'])
+
+    def test_update_with_bad_insert_headers(self):
+        listener = self.create_listener(
+            constants.PROTOCOL_HTTP, 80, self.lb_id)
+        self.set_lb_status(self.lb_id)
+        new_listener = self._build_body(
+            {'insert_headers': {'X-Bad-Header': 'true'}})
+        listener_path = self.LISTENER_PATH.format(
+            listener_id=listener['listener'].get('id'))
+        update_listener = self.put(
+            listener_path, new_listener, status=400).json
+        self.assertIn('{0} is not a valid option for {1}'.format(
+            '[\'X-Bad-Header\']', 'insert_headers'),
+            update_listener.get('faultstring'))
+
+        # test client certificate http headers
+        header = {}
+        for name in constants.SUPPORTED_SSL_HEADERS:
+            header[name] = 'true'
+        new_listener[self.root_tag]['insert_headers'] = header
+        # as the order of output faultstring is not stable, so we just check
+        # the status.
+        self.put(listener_path, new_listener, status=400).json
+
     def _getStats(self, listener_id):
         res = self.get(self.LISTENER_PATH.format(
             listener_id=listener_id + "/stats"))

releasenotes/notes/Add-TLS-client-auth-header-insertion-039debc7e6f06474.yaml

@@ -0,0 +1,7 @@
+---
+features:
+  - |
+    When using TLS client authentication on TERMINATED_HTTPS listeners, you can now insert the
+    following headers for backend members\: 'X-SSL-Client-Verify', 'X-SSL-Client-Has-Cert',
+    'X-SSL-Client-DN', 'X-SSL-Client-CN', 'X-SSL-Issuer', 'X-SSL-Client-SHA1',
+    'X-SSL-Client-Not-Before', 'X-SSL-Client-Not-After'.