Browse Source

Add Federation API specification

Add a new spec covering the impact and implementation details for a new
Magnum API managing federation and federated clusters.

Implements: blueprint federation-api
Co-Authored-By: Clenimar Filemon <clenimar.filemon@gmail.com>

Change-Id: I1b58f44520c6d968dd0c930a54dbd0ef36c2cb4e
master
Ricardo Rocha 1 year ago
parent
commit
1315420afe
2 changed files with 304 additions and 0 deletions
  1. 8
    0
      doc/source/index.rst
  2. 296
    0
      specs/queens/federation-api.rst

+ 8
- 0
doc/source/index.rst View File

@@ -6,6 +6,14 @@
6 6
 magnum-specs Design Specifications
7 7
 ==================================================
8 8
 
9
+Queens approved specs:
10
+
11
+.. toctree::
12
+   :glob:
13
+   :maxdepth: 2
14
+
15
+   specs/queens/*
16
+
9 17
 Ocata approved specs:
10 18
 
11 19
 .. toctree::

+ 296
- 0
specs/queens/federation-api.rst View File

@@ -0,0 +1,296 @@
1
+Magnum Federation API
2
+========================
3
+
4
+Launchpad blueprint:
5
+
6
+https://blueprints.launchpad.net/magnum/+spec/federation-api
7
+
8
+This is a proposal to extend the Magnum API adding support for
9
+federating existing clusters.
10
+
11
+
12
+Problem Description
13
+-------------------
14
+
15
+Magnum provides the means to deploy individual container clusters, supporting
16
+multiple container orchestration engines (COE) - currently Swarm, Kubernetes
17
+and Mesos/DCOS. Each of these clusters is independent from all others, with
18
+its master node(s), worker node(s) and certificate authorities from which client
19
+certificates are issued.
20
+
21
+In the case of Kubernetes the COE has the ability to federate independent
22
+clusters, joining them under a single API endpoint and distributing workloads
23
+among all participants. We can envision that other COEs will add similar
24
+functionality in the future.
25
+
26
+This proposal adds a new API endpoint to Magnum to setup, persist and manage
27
+federations of local or external clusters.
28
+
29
+
30
+Use Cases
31
+---------
32
+
33
+Below are some of the use cases that this API addresses:
34
+
35
+1. A set of Magnum clusters is created in independent projects so that each
36
+   is accounted for its own resources, but a single endpoint is desired where
37
+   policies on workload scheduling can be enforced.
38
+
39
+2. A very large container infrastructure is desired, composed of thousands or
40
+   tens of thousands of nodes and growing regularly - as an example, version
41
+   1.7 of Kubernetes supports up to 5000 nodes [3]. A project administrator can
42
+   create a new cluster for each set of new resources and simply add this
43
+   new cluster to an existing federation, easing the management of the overall
44
+   resources and increasing scalability.
45
+
46
+3. A set of Magnum clusters is created, each with different characteristics:
47
+   node flavor, storage setup, etc. Federating them together forms a
48
+   heterogeneous cluster.
49
+
50
+4. An existing Magnum cluster in an OpenStack environment is to be extended
51
+   using external resources. An external cluster endpoint (deployed in AWS,
52
+   Azure, GKE, another OpenStack or cloud) can be added to an existing
53
+   Magnum federated cluster, including the complex setup and management of
54
+   cluster credentials.
55
+
56
+5. A project has several existing clusters which it would like to expose to a
57
+   set of users in a single endpoint, without disrupting existing users of
58
+   each cluster.
59
+
60
+
61
+Proposed Changes
62
+----------------
63
+
64
+The proposed change includes:
65
+
66
+* Adding a new '/federation' REST API endpoint to Magnum providing management
67
+  of federated clusters. This includes federation creation, update and
68
+  deletion, as well as adding and removing members from a federation.
69
+
70
+* Adding a new entity to the data model (federation) to represent a federated
71
+  cluster and its metadata.
72
+
73
+Check sections 'Data Model Impact' and 'REST API Impact' for more details.
74
+
75
+
76
+Alternatives
77
+------------
78
+
79
+A valid alternative would be to manage the setup of the federated control
80
+plane on the client side only. This is possible but has a few drawbacks:
81
+
82
+* The logic for the federation setup is in some cases contained in a tool
83
+  supported by the COE. Relying on it would mean adding a client side
84
+  dependency on each of the supported COE clients (hard to manage) or
85
+  replicating the full functionality in the Magnum client code (hard to keep
86
+  up with upstream COE developments).
87
+
88
+* Managing the cluster client access context is COE specific and required to
89
+  setup the federation control plane. Asking the user to do this herself
90
+  would greatly reduce the added value of Magnum on managing federations, and
91
+  miss the opportunity to (re)use a lot of the cluster metadata information
92
+  that Magnum already stores.
93
+
94
+
95
+Data Model Impact
96
+-----------------
97
+
98
+A new entity would be added (corresponding tables will be added):
99
+
100
+* **federation**
101
+
102
+  * uuid
103
+  * name
104
+  * project_id
105
+  * hostcluster_id (the cluster hosting the federation control plane)
106
+  * member_ids (the clusters that are members of the federation)
107
+  * status (the current status of the federation)
108
+  * status_reason (a message with more info regarding the status)
109
+  * properties (additional metadata, coe specific in some cases)
110
+
111
+We chose to add a new entity so that we:
112
+
113
+* decouple the federation entity from the existing resources
114
+* minimize the impact on the current interfaces
115
+* more easily extend the federation functionality in the future
116
+
117
+
118
+REST API Impact
119
+---------------
120
+
121
+This change leads to a minor version increase in the Magnum API, the
122
+addition of a new REST endpoint and a new set of CLI commands.
123
+
124
+Below is a description of the commands to manage federations:
125
+
126
+* federation create: creates a new federation, in an existing cluster::
127
+
128
+    openstack coe federation create myfederation --hostcluster cluster1
129
+
130
+* federation delete: deletes an existing federation::
131
+
132
+    openstack coe federation delete myfederation
133
+
134
+* federation update: updates the metadata of an existing federation::
135
+
136
+    openstack coe federation update --property dns='cluster.local' myfederation
137
+
138
+* federation list: list existing federations, with metadata and membership::
139
+
140
+    openstack coe federation list
141
+
142
+    +------+--------------+---------+-----------------+
143
+    | uuid | name         | members | status          |
144
+    +------+--------------+---------+-----------------+
145
+    | ...  | myfederation | 3       | UPDATE_COMPLETE |
146
+    +------+--------------+---------+-----------------+
147
+
148
+
149
+* federation show: show details of an existing federation::
150
+
151
+    openstack coe federation show myfederation
152
+
153
+    +---------------------+-------------------------------------------+
154
+    | Property            | Value                                     |
155
+    +---------------------+-------------------------------------------+
156
+    | uuid                | 5b2ee3b5-2f85-4917-be7c-11a2c82031ad      |
157
+    | name                | myfederation                              |
158
+    | hostcluster         | <uuid-cluster1>                           |
159
+    | members             | ['<uuid-cluster2>', '<uuid-cluster3>']    |
160
+    | project_id          | ae72a4f3-30cf-4406-83b4-40b16bb480d6      |
161
+    | properties          | dns=cluster.local                         |
162
+    | status              | UPDATE_COMPLETE                           |
163
+    | status_reason       | Federation UPDATE completed successfully  |
164
+    +---------------------+-------------------------------------------+
165
+
166
+* federation join: adds an existing cluster to a federation::
167
+
168
+    openstack coe federation join myfederation cluster2 cluster3
169
+
170
+* federation unjoin: parts a cluster from a federation::
171
+
172
+    openstack coe federation unjoin myfederation cluster3
173
+
174
+
175
+Other Implementation Options
176
+----------------------------
177
+
178
+See Alternatives.
179
+
180
+
181
+Security Impact
182
+---------------
183
+
184
+Management of federations implies the same level of security as for the
185
+existing cluster management functionality - creation, deletion, update based
186
+on a policy.
187
+
188
+Joining/unjoining a federation requires special care and a special policy
189
+evaluating permissions on both source and destination clusters - the host
190
+and the additional members. An example policy is shown below:
191
+
192
+
193
+Notifications Impact
194
+--------------------
195
+
196
+New notifications will be added for:
197
+* federation creation
198
+* federation deletion
199
+* federation update (including metadata and members)
200
+
201
+
202
+Other End User Impact
203
+---------------------
204
+
205
+Users will not be able to delete a cluster that is part of a federation, and
206
+will have to unjoin first.
207
+
208
+New subcommands will be added to the openstack client as described above.
209
+
210
+
211
+Developer Impact
212
+----------------
213
+
214
+This functionality will be added via a separate endpoint and metadata stored
215
+in a new entity. Impact in the existing Magnum code will be minimal.
216
+
217
+
218
+Implementation
219
+--------------
220
+
221
+The implementation will be done in 5 phases.
222
+
223
+1. Add the new API endpoint and data model entity, and the corresponding
224
+   controller implementation linked to each driver. At this point we will
225
+   have all drivers declaring this functionality as 'Not Implemented'.
226
+
227
+2. Implement the federation functionality for the Kubernetes Atomic driver.
228
+   This includes all the described setup in [2] (dns, context, credentials).
229
+   The initial implementation will only support federating Magnum clusters
230
+   in the same OpenStack deployment.
231
+
232
+3. Add the new command line tools to the openstack client.
233
+
234
+4. Add support for including external Kubernetes clusters (deployed in GKE,
235
+   ACS, AWS, ...) in a Magnum federation. Given this is all set at the level
236
+   of Kubernetes the source environment of the cluster should not be
237
+   relevant, but a way for the user to provide the cluster context to Magnum
238
+   will be added to the command line tools and API.
239
+
240
+5. Implement the Magnum federation notifications, for creation, deletion and
241
+   update.
242
+
243
+Implementation of this functionality for drivers other than Kubernetes will be
244
+left for future specs, as there is currently no support in other COEs.
245
+
246
+
247
+Assignee(s)
248
+-----------
249
+
250
+See blueprint:
251
+
252
+https://blueprints.launchpad.net/magnum/+spec/federation-api
253
+
254
+
255
+Work Items
256
+----------
257
+
258
+See blueprint:
259
+
260
+https://blueprints.launchpad.net/magnum/+spec/federation-api
261
+
262
+
263
+Dependencies
264
+------------
265
+
266
+Setting up federations is often done by a COE specific tool - kubefed in the
267
+case of Kubernetes. We will rely on these tools for the driver specific
268
+implementation, thus adding new dependencies on them.
269
+
270
+
271
+Testing
272
+-------
273
+
274
+A new set of tests covering creation, deletion, update of federations and
275
+join/unjoin of clusters needs to be added to both unit and functional tests.
276
+
277
+
278
+Documentation Impact
279
+--------------------
280
+
281
+New documentation will be added to describe the new API endpoint and its
282
+functionality. The quickstart guide needs an update to include information
283
+regarding cluster federation.
284
+
285
+
286
+References
287
+----------
288
+
289
+[1] - Magnum Federation API Blueprint:
290
+https://blueprints.launchpad.net/magnum/+spec/federation-api
291
+
292
+[2] - Tutorial on Cluster Federation setup with Kubefed
293
+https://kubernetes.io/docs/tasks/federation/set-up-cluster-federation-kubefed
294
+
295
+[3] - Kubernetes - Building Large Cluster:
296
+https://kubernetes.io/docs/admin/cluster-large/

Loading…
Cancel
Save