Browse Source

Add cluster driver encapsulation spec

Moving this to the spec repo. Previous patch is
https://review.openstack.org/#/c/389835/

Change-Id: I9de7b59e6737ae0fe01e3a6f33ec5fafcaa7984a
Partial-Blueprint: bp-driver-consolodation
master
Randall Burt 2 years ago
parent
commit
9fc6ac0b8c
1 changed files with 285 additions and 0 deletions
  1. 285
    0
      specs/ocata/driver-encapsulation.rst

+ 285
- 0
specs/ocata/driver-encapsulation.rst View File

@@ -0,0 +1,285 @@
1
+..
2
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
3
+ License.
4
+
5
+ http://creativecommons.org/licenses/by/3.0/legalcode
6
+
7
+================================================
8
+Cluster Driver Encapsulation and Lifecycle Hooks
9
+================================================
10
+
11
+Launchpad blueprint:
12
+
13
+https://blueprints.launchpad.net/magnum/+spec/bp-driver-consolodation
14
+
15
+By encapsulating all operations and monitoring performed on a cluster into the
16
+driver interface, operators have a broader choice in which services they chose
17
+to install and maintain in order to most efficiently operate Magnum according
18
+to their own constraints and service expertise.
19
+
20
+Also, extending this driver interface to include the definition of pre and post
21
+operation hooks can allow operators to more easily integrate Magnum with other
22
+parts of their infrastructure which could possibly exist outside of the
23
+OpenStack cloud.
24
+
25
+Problem Description
26
+===================
27
+
28
+Magnum currently only supports drivers that leverage Heat as the underlying
29
+cluster orchestration engine. Because Magnum makes direct calls to Heat during
30
+creation and deletion of a cluster, operators and other contributors who wish
31
+to leverage alternative orchestration engines are unable to do so.
32
+
33
+Additionally, should the operator need to customise this orchestration, they
34
+are limited to only those mechanisms directly supported by Heat.
35
+
36
+Use Cases
37
+=========
38
+
39
+1. Leverage one or more alternative OpenStack services for cluster
40
+   orchestration such as Senlin or Mistral.
41
+
42
+2. Allow the operator to define complex lifecycle management hooks outside of
43
+   those provided by Heat directly in their driver implementation.
44
+
45
+3. Drivers that leverage Heat should be even simpler to implement and only
46
+   require the implementor to define a Heat template with little or no driver
47
+   code.
48
+
49
+Proposed Change
50
+===============
51
+
52
+1. The driver class hierarchy shall be extended to include a base driver class
53
+   for defining the (mostly existing) contract for a driver independent of
54
+   orchestration engine while adding methods to cover those operations for
55
+   which Magnum currently calls Heat directly.
56
+
57
+2. Magnum code shall be refactored to defer any cluster operations to this new
58
+   driver interface.
59
+
60
+3. An abstract implementation of this interface based on common Heat operations
61
+   will be created and the current driver code refactored to be based on the
62
+   new implementation.
63
+
64
+Proposed Driver Interface
65
+-------------------------
66
+Attributes and methods of the existing classes keep their current definitions
67
+unless specifically called out below. The ``osc`` argument has been removed
68
+from the applicable driver methods to allow for non-OpenStack drivers to manage
69
+their own clients.
70
+
71
+.. code-block:: python
72
+
73
+    @six.add_metaclass(abc.ABCMeta)
74
+    class Driver(object):
75
+        # magnum.drivers.common.driver.Driver
76
+
77
+        @abc.abstractproperty
78
+        def provides(self):
79
+            # return a list of (server_type, os, coe) tuples supported by this
80
+            # driver
81
+
82
+        @abc.abstractmethod
83
+        def create_cluster(self, context, cluster, cluster_create_timeout):
84
+            # ... renamed create_stack method
85
+
86
+        @abc.abstractmethod
87
+        def update_cluster(self, context, cluster, scale_manager=None,
88
+                           rollback=False):
89
+            # ... renamed update_stack method; For now, this will do what it
90
+            # does today. The intent is to avoid too many disruptive changes in
91
+            # the current interface and defer extensions and/or new features
92
+            # to subsequent specifications. In the future, this method should
93
+            # probably be broken down into more granular operations like scale,
94
+            # upgrade, rename, etc.
95
+
96
+        @abc.abstractmethod
97
+        def delete_cluster(self, context, cluster):
98
+            # ... add this method to the driver
99
+
100
+Proposed Heat Driver
101
+--------------------
102
+These classes essentially consolidate existing Heat-based driver logic into
103
+easily extensible classes that most existing drivers can implement.
104
+
105
+.. code-block:: python
106
+
107
+    @six.add_metaclass(abc.ABCMeta)
108
+    class HeatDriver(magnum.drivers.common.driver.Driver):
109
+        # magnum.drivers.common.driver.heat.HeatDriver
110
+
111
+        @abc.abstractmethod
112
+        def get_template_definition(self):
113
+            # return an implementation of
114
+            # magnum.drivers.common.driver.heat.TemplateDefinition
115
+
116
+        def create_cluster(self, context, osc, cluster, cluster_create_timeout):
117
+            # per the current driver implementation
118
+
119
+        def update_cluster(self, context, osc, cluster, scale_manager=None,
120
+                           rollback=False):
121
+            # per the current driver implementation
122
+
123
+        def delete_cluster(self, context, cluster):
124
+            # ... move the existing delete logic into this driver
125
+
126
+    @six.add_metaclass(abc.ABCMeta)
127
+    class TemplateDefinition(object):
128
+        # move magnum.drivers.common.template_def.TemplateDefinition to
129
+        # magnum.drivers.common.driver.heat.TemplateDefinition
130
+
131
+
132
+    class HeatPoller(object):
133
+        # move existing
134
+        # magnum.conductor.handlers.cluster_conductor.HeatPoller to
135
+        # magnum.drivers.common.driver.heat.HeatPoller
136
+
137
+Design Principles
138
+=================
139
+
140
+Magnum should never have to make assumptions about the underlying orchestration
141
+system and should defer all calls for information to the specified driver for a
142
+cluster. To do this the driver interface should be designed to sufficiently
143
+allow Magnum to enact all aspects of a cluster's lifecycle.
144
+
145
+A sub-class of driver should be created such that a user or operator wishing to
146
+continue to use Heat as the underlying orchestration engine would only need to
147
+extend this class to specify a template and at best some basic mappings between
148
+driver parameters and parameters in the underlying template.
149
+
150
+Alternatives
151
+============
152
+
153
+1. Force users that wish to use other OpenStack services for orchestration to
154
+   do so via Heat resources for those services.
155
+
156
+       a. This forces operators into deploying, supporting, and maintaining
157
+          Heat in addition to whichever other services they may actually
158
+          require.
159
+
160
+       b. This assumes 100% coverage by Heat of other applicable services and/or
161
+          limits the aspects of the target service that a provider can take
162
+          advantage of to those exposed by the Heat resources.
163
+
164
+       c. This requires users who wish to integrate custom or existing services
165
+          and infrastructure outside of OpenStack to implement custom Heat
166
+          resources in addition to having to implement a custom Magnum driver.
167
+
168
+2. Force users wishing to apply something other than Heat alone for cluster
169
+   orchestration to fork or manage a patched version of Magnum. This would mean
170
+   patching or overriding the conductor and driver mechanisms.
171
+
172
+       a. Patching leads to anger. Anger leads to hate. Hate leads to
173
+          suffering.
174
+
175
+       b. Leaving operators the only reasonable option to fork the project
176
+          should be avoided if at all possible.
177
+
178
+Data Model Impact
179
+=================
180
+
181
+Refactoring the data model to rename or move the ``stack_id`` column from the
182
+cluster data model could be considered, however it is probably ok to leave this
183
+for now and just make it nullable to remove the need for data migration and
184
+upgrade downtime.
185
+
186
+REST API Impact
187
+===============
188
+
189
+This proposal does not anticipate requiring any changes to the current v1 api
190
+nor to the proposed v2 API allowing for heterogeneous clusters.
191
+
192
+Security Impact
193
+===============
194
+
195
+None identified.
196
+
197
+Notifications Impact
198
+====================
199
+
200
+None identified.
201
+
202
+Other End-user Impact
203
+=====================
204
+
205
+None identified.
206
+
207
+Performance Impact
208
+==================
209
+
210
+No change in current performance is anticipated as this is largely a refactor
211
+of existing code.
212
+
213
+Other Deployer Impact
214
+=====================
215
+
216
+By default, the drivers and dependencies won't change as this proposal would
217
+only refactor code and keep the existing drivers functionally intact.
218
+
219
+Deployers wishing to customize cluster orchestraion will have significantly
220
+increased flexibility in how to implement and a broader choice in what they can
221
+implement without having to patch Magnum directly.
222
+
223
+Developer Impact
224
+================
225
+
226
+Developers wishing to create customised drivers based on Heat should have a
227
+much easier time given that in many cases, all they need to do is subclass the
228
+base Heat driver and define a template location.
229
+
230
+Developers who wish to leverage a service other than Heat will be able to
231
+develop those drivers without having to touch or patch other parts of Magnum.
232
+
233
+Implementation
234
+==============
235
+
236
+Assignee(s)
237
+-----------
238
+
239
+TBD
240
+
241
+Work Items
242
+----------
243
+
244
+1. Refactor current driver code into an abstract Driver class that defines a
245
+   driver's contract
246
+
247
+2. Create a base HeatDriver class that implements most common methods for
248
+   leveraging Heat for cluster operations independent of a specific template
249
+
250
+3. Identify and refactor references to the template_def object in favor of new
251
+   driver methods
252
+
253
+4. Refactor the current drivers to leverage the generic Heat driver
254
+
255
+5. Refactor Magnum code to defer all orchestration operations to the driver
256
+
257
+
258
+Dependencies
259
+============
260
+
261
+None identified.
262
+
263
+Testing
264
+=======
265
+
266
+1. Add unit tests to validate each new class in the driver hierarchy
267
+
268
+2. All other functional unit tests should pass without modification.
269
+
270
+Documentation Impact
271
+====================
272
+
273
+1. The cluster drivers section of the users guide will need to be updated to
274
+   include both the new interface as well as implementing a driver via the Heat
275
+   interface.
276
+   (http://docs.openstack.org/developer/magnum/userguide.html#cluster-drivers)
277
+
278
+2. The documentation will need to make it clear that existing diagnostic
279
+   techniques apply to Heat-based drivers, and that other driver types will
280
+   have a driver-specific troubleshooting process.
281
+
282
+References
283
+==========
284
+
285
+None identified.

Loading…
Cancel
Save