Browse Source

External facing K8s API w/ Keystone auth/z

- Deploy a second set of Kubernetes apiservers configured
  to only support auth/authz via Keystone webhook. Do not
  include these apiservers in the endpoint list for the standard
  kubernetes API service
- Create an ingress entrypoint for the Kubernetes API so that
  it is reachable from external clients.

Change-Id: Id8ae3060878a91c018a4134dc6eafda449cf04e3
Scott Hussey 8 months ago
parent
commit
5bb092810d
1 changed files with 163 additions and 0 deletions
  1. 163
    0
      specs/approved/k8s_external_facing_api.rst

+ 163
- 0
specs/approved/k8s_external_facing_api.rst View File

@@ -0,0 +1,163 @@
1
+..
2
+  This work is licensed under a Creative Commons Attribution 3.0 Unported
3
+  License.
4
+
5
+  http://creativecommons.org/licenses/by/3.0/legalcode
6
+
7
+.. index::
8
+   single: Kubernetes
9
+   single: Promenade
10
+   single: Security
11
+
12
+============================================================
13
+Deploy Kubernetes API Server w/ Ingress and Keystone Webhook
14
+============================================================
15
+
16
+OpenStack Keystone_ will the be single authentication mechanism for Airship
17
+users. As such, we need to deploy a Kubernetes API server endpoint that
18
+can utilize Keystone for authentication and authorization. The avenue to support
19
+this is using the Kubernetes `webhook admission controller`_ with a webhook
20
+supporting Keystone.
21
+
22
+Links
23
+=====
24
+
25
+None
26
+
27
+Problem description
28
+===================
29
+
30
+While Airship component APIs should care for most lifecycle tasks for the
31
+Kubernetes cluster, there will be some maintenance and recovery operations
32
+that will require direct access to the Kubernetes API. To properly secure
33
+this API, it needs to utilize the common single sign-on that operators use
34
+for accessing Airship APIs, i.e. Keystone. However, the external facing API
35
+should minimize risk to the core Kubernetes API servers used by other
36
+Kubernetes core components. This specification proposes a design to maximize
37
+the security of this external facing API endpoint and minimizes the
38
+risk to the core operations of the cluster by avoiding the need to add
39
+complexity to core apiserver configuration or sending extra traffic through
40
+the core apiservers and Keystone.
41
+
42
+Impacted components
43
+===================
44
+
45
+The following Airship components would be impacted by this solution:
46
+
47
+#. Promenade - Maintenance of the chart for external facing Kubernetes API
48
+servers
49
+
50
+Proposed change
51
+===============
52
+
53
+Create a chart, ``webhook_apiserver``, for an external facing Kubernetes API server that would
54
+create a Kubernetes Ingress entrypoint for the API server and, optionally, also spin up a
55
+webhook side-car for each API server (i.e. ``sidecar`` mode). The other mode of operation
56
+is ``federated`` mode where the webhook will be accessed over a Kubernetes service.
57
+
58
+A new chart is needed because the `standard apiserver chart <https://github.com/openstack/airship-promenade/tree/master/charts/apiserver>`
59
+relies on the anchor pattern creating static pods. The ``webhook_apiserver`` chart
60
+should be based on the standard apiserver chart and use helm_toolkit_ standards.
61
+
62
+The chart would provide for configuration of the `Keystone webhook`_ (also
63
+`Keystone webhook addl`_ and `Keystone webhook chart`_) in ``sidecar`` mode and allow for configuring
64
+the webhook service address in ``federated``` mode. The Kubernetes apiserver
65
+would be configured to only allow for authentication/authorization via webhook.
66
+No other authorization modes would be enabled. All ``kube-apiserver`` command line options
67
+should match the with the following exceptions:
68
+
69
+  - authorization-mode: ``Webhook``
70
+  - audit-log-path: ``-``
71
+  - authentication-token-webhook-config-file: path to configuration file for accessing the webhook.
72
+  - authorization-webhook-config-file: path to configuration file for accessing the webhook.
73
+  - apiserver-count: omit
74
+  - endpoint-reconciler-type: ``none``
75
+
76
+Webhook Configuration
77
+---------------------
78
+
79
+The configuration for how the Kubernetes API server will contact the webhook service is
80
+stored in a YAML configuration file based on the `kubeconfig file format`_. The below
81
+example would be used in ``sidecar`` mode.
82
+
83
+.. code:: yaml
84
+  :number-lines:
85
+
86
+  # clusters refers to the remote service.
87
+  clusters:
88
+    - name: keystone-webhook
89
+      cluster:
90
+        # CA for verifying the remote service.
91
+        certificate-authority: /path/to/webhook_ca.pem
92
+        # URL of remote service to query. Must use 'https'. May not include parameters.
93
+        server: https://localhost:4443/
94
+
95
+  # users refers to the API Server's webhook configuration.
96
+  users:
97
+    - name: external-facing-api
98
+      user:
99
+        client-certificate: /path/to/apiserver_webhook_cert.pem # cert for the webhook plugin to use
100
+        client-key: /path/to/apiserver_webhook_key.pem          # key matching the cert
101
+
102
+  # kubeconfig files require a context. Provide one for the API Server.
103
+  current-context: webhook
104
+  contexts:
105
+  - context:
106
+      cluster: keystone-webhook
107
+      user: external-facing-api
108
+    name: webhook
109
+
110
+Documentation impact
111
+--------------------
112
+
113
+Documentation of the overrides to this chart for controlling
114
+webhook authorization mapping policy.
115
+
116
+Security impact
117
+---------------
118
+
119
+- Additional TLS certificates for apiserver <-> webhook connections
120
+- Keystone webhook must have an admin-level Keystone account
121
+- Optionally, the Keystone webhook minimizes attack surface by becoming a sidecar without external facing service.
122
+
123
+Performance impact
124
+------------------
125
+
126
+This should not have any performance impacts as the only traffic handled by the webhook
127
+will be from users specifically using Keystone for authentication and authorization.
128
+
129
+Testing impact
130
+--------------
131
+
132
+The chart should include a Helm test that validates a valid Keystone token
133
+is usable with ``kubectl`` to successfully get a respond from the Kubernetes
134
+API.
135
+
136
+Implementation
137
+==============
138
+
139
+Milestone 1
140
+-----------
141
+
142
+Chart support for ``sidecar`` mode
143
+
144
+Milestone 2
145
+-----------
146
+
147
+Addition of ``federated`` mode
148
+
149
+Dependencies
150
+============
151
+
152
+None
153
+
154
+References
155
+==========
156
+
157
+.. _Keystone: https://docs.openstack.org/keystone
158
+.. _`webhook admission controller`: https://kubernetes.io/docs/reference/access-authn-authz/webhook/
159
+.. _`Keystone webhook`: https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-keystone-webhook-authenticator-and-authorizer.md
160
+.. _`Keystone webhook addl`: https://github.com/dims/k8s-keystone-auth
161
+.. _`kubeconfig file format`: https://v1-10.docs.kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/
162
+.. _`Keystone webhook chart`: https://github.com/openstack/openstack-helm-infra/tree/master/kubernetes-keystone-webhook
163
+.. _helm_toolkit: https://docs.openstack.org/openstack-helm/latest/devref/index.html#

Loading…
Cancel
Save