Browse Source

Neutron-Neutron Interconnections

Proposed specifications for the Neutron-Neutron Interconnections
RFE.

Change-Id: I3860a62420ab4983e2b741dff04498fbb0432d00
Related-Bug: 1750368
Thomas Morin 1 year ago
parent
commit
2a34abbd25
1 changed files with 544 additions and 0 deletions
  1. 544
    0
      specs/rocky/neutron-inter.rst

+ 544
- 0
specs/rocky/neutron-inter.rst View File

@@ -0,0 +1,544 @@
1
+..
2
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
3
+ License.
4
+
5
+ http://creativecommons.org/licenses/by/3.0/legalcode
6
+
7
+================================
8
+Neutron-Neutron Interconnections
9
+================================
10
+
11
+Launchpad RFE: https://bugs.launchpad.net/neutron/+bug/1750368
12
+
13
+Problem Description
14
+===================
15
+
16
+Today, to realize connectivity between two OpenStack clouds or more (e.g.
17
+between distinct OpenStack deployments, or between OpenStack regions,
18
+for instance) some options are available, such as floating IPs, VPNaaS
19
+(IPSec-based), and BGPVPNs.
20
+
21
+However, none of these options are appropriate to address use cases where all
22
+the following properties are desired:
23
+
24
+1. interconnection consumable on-demand, without admin intervention [#f1]_
25
+
26
+2. have network isolation and allow the use of private IP addressing
27
+   end-to-end [#f2]_
28
+
29
+3. avoid the overhead of packet encryption [#f3]_
30
+
31
+An additional design requirement to the solution is to avoid introducing a
32
+component that would require having admin rights on all the OpenStack
33
+clouds involved.
34
+
35
+The goal of this spec is a solution to provide network connectivity
36
+between two or more OpenStack deployments or regions, respecting these
37
+requirements.
38
+
39
+Use cases
40
+---------
41
+
42
+Example 1
43
+~~~~~~~~~
44
+
45
+User Foo has credentials for OpenStack cloud A and OpenStack cloud B, one
46
+Router X in OpenStack A, one Router Y in OpenStack B, both using a distinct
47
+subnet in the private IPv4 address space.
48
+
49
+User Foo would like to consume via an API a service that will result in
50
+establishing IP connectivity between Router A and Router B.
51
+
52
+Example 2
53
+~~~~~~~~~
54
+
55
+Same as example 1, but this time L2 connectivity is desired between Network X
56
+in OpenStack A, and Network Y in OpenStack B.
57
+
58
+Example 3
59
+~~~~~~~~~
60
+
61
+Same as example 1 or 2, but with 3 OpenStack deployments involved.
62
+
63
+Proposed Change
64
+===============
65
+
66
+Overview
67
+--------
68
+
69
+The proposition consists in introducing a service plugin and a corresponding
70
+API extension involving a new 'interconnection' resource. The 'interconnection'
71
+resource on a Neutron instance will refer to both a local resource (e.g.
72
+Router A) and a remote resource (OpenStack B:Router B), and will have the
73
+semantic that connectivity is desired between the two.
74
+
75
+This resource will be part of two sort of API calls:
76
+
77
+A. calls between the user having the connectivity need and each Neutron
78
+   instance involved; the role of these calls is to let the need for
79
+   connectivity be known by all Neutron instances
80
+
81
+B. API calls between a Neutron instance and another Neutron instance; the role
82
+   of these calls is to:
83
+
84
+   * let each Neutron instance validate the consistency between the
85
+     "interconnection" resources defined locally and the "interconnection"
86
+     resources made in other Neutron instances
87
+
88
+   * after this validation, let two Neutron instances identify the mechanism
89
+     to use and the per-interconnection parameters (depending on the
90
+     mechanism; could be a pair of VLANs on a interconnection box, a BGPVPN
91
+     RT identifiers, VXLAN ids, etc.)
92
+
93
+::
94
+
95
+                                .-------------.
96
+                                | tenant user |
97
+                                '----+---+----'
98
+                                     |   |
99
+    .--------------------------------'   '--------------------.
100
+    | A1. create "interconnection"                            |
101
+    |     between local net X,                                |
102
+    |     and "Neutron B: net Y"                              |
103
+    |                                          A2. create "interconnection"
104
+    |                                              between local net Y,
105
+    |                                              and "Neutron A: net X"
106
+    |                                                         |
107
+    |                                                         |
108
+    |                                                         |
109
+    |                                                         |
110
+    V                         B1. check symmetric inter.      V
111
+ .-------------------------.                      (fail) .--------------------.
112
+ |                         +---------------------------> |                    |
113
+ |   Neutron A             |                             |   Neutron B        |
114
+ |                         | <---------------------------+                    |
115
+ |                         |  B2. check symmetric inter. |                    |
116
+ |                         |                       (ok!) |                    |
117
+ |                         |                             |                    |
118
+ |                         +---------------------------> |                    |
119
+ |                         | <---------------------------+                    |
120
+ '-------------------------'  B3. exchange info to build '--------------------'
121
+                    net X        interconnection           net Y
122
+                      |                                      |
123
+                      |                                      |
124
+                    --+                                      +--
125
+                      |      C. interconnection is built     |
126
+                      +--    - - - - - - - - - - - - - -   --+
127
+                      |                                      |
128
+                      |                                      |
129
+
130
+
131
+Note that the order between A1/A2/B1/B2 can vary, but the result is
132
+unchanged: at least one of the two Neutron instances will eventually confirm
133
+that the interconnection has been defined on both side symmetrically, and the
134
+interconnection setup phase will ultimate proceed on both sides (see
135
+:ref:`details`).
136
+
137
+When more than two OpenStack deployments, or more than two OpenStack regions,
138
+are involved, these API calls will happen for each pair of region/deployments.
139
+
140
+Base assumptions, trust model
141
+-----------------------------
142
+
143
+The base assumption underlying the trust model in this proposal is that the end
144
+user requesting connectivity delegates trust to each OpenStack deployment to
145
+provide only the interconnection requested, i.e. to not create connectivity
146
+between resources unless requested.
147
+
148
+To respect this contract each OpenStack deployment needs, by definition, a
149
+trust relationship with the other(s) OpenStack deployment(s) involved in these
150
+interconnections; practically speaking it cannot do better, when receiving
151
+a packet from another deployment, identified as intended for own of its local
152
+network A (VLAN, VXLAN ID, MPLS label, etc.) to trust that this identifier was
153
+pushed by the OpenStack deployment by mechanisms ultimately respecting the
154
+contract of these specifications.
155
+
156
+Another aspect, obvious but better made explicit, is that the choice and
157
+definition of the network identifiers that will be used for an interconnection
158
+and to keep interconnections isolated from one another, are not controlled
159
+by consumers of this API. In this proposal these consumers do not and cannot
160
+write, or even read, these identifiers.
161
+
162
+Note that only the API calls to the 'interconnection' resources at step A
163
+require write access to the "interconnection" resources by tenant users (but
164
+not to the attributes related to the network mechanism to use).
165
+
166
+The calls at steps B, only require read-only access to these resources;
167
+this can be achieved by introducing an "interconnection" role with read-only
168
+access to all "interconnection" resources, and having each OpenStack deployment
169
+having credentials for a user with this role in other OpenStack deployments.
170
+
171
+With the above in mind, Keystone federation is not required for the calls at
172
+step 1, nor for the calls at step 2. However, using Keystone Federation for
173
+the user(s) used at step 2 will certainly be useful and avoid requiring the
174
+management in each Neutron instance of the credentials to use to each other
175
+OpenStack.
176
+
177
+Interconnection mechanisms
178
+--------------------------
179
+
180
+Although these specifications try to be agnostic to the network technique
181
+ultimately used to realize an interconnection, the assumption is made that
182
+for each 'interconnection', there is a technique common to the two OpenStack
183
+deployments involved.
184
+
185
+The approach proposed is a simple approach where each OpenStack deployment
186
+determines based on a configuration file, which technique to use when
187
+establishing an interconnection with a given OpenStack deployment.
188
+
189
+Note that only the parameters that do not differ between two interconnection
190
+would sit in a configuration file. The API exchange between two Neutron
191
+instances is used to exchange parameters that are specific to each
192
+interconnection.
193
+
194
+Example interconnection techniques
195
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
196
+
197
+The following techniques can be considered for realizing interconnections:
198
+
199
+* BGP-based VPNs: already supported via the Neutron BGPVPN Interconnection
200
+  service (see networking-bgpvpn_), it allows to create L2 (with EVPN) or L3
201
+  connectivity (with BGP/MPLS IP VPNs)
202
+
203
+* VXLAN stitching with networking-l2gw_ (details remain to be investigated)
204
+
205
+* VLAN stitching (details remain to be investigated)
206
+
207
+It is expected that the first implementation will provide at least support
208
+for the BGPVPN interconnection technique, which is already supported across
209
+an interesting range of Neutron backends (Neutron reference drivers,
210
+OpenDaylight, OpenContrail, Nuage Networks), and hence would allow this API
211
+extension to be implemented on day one with a support for all these controllers
212
+without any further per-controller driver development.
213
+
214
+.. _details:
215
+
216
+Details on operations
217
+---------------------
218
+
219
+When an "interconnection" resource is created, the Neutron instance will check
220
+that the symmetric interconnection exists on the remote Neutron instance
221
+designated in the interconnection, and will not proceed further until this
222
+becomes true.
223
+
224
+This check is what establishes the end-to-end trust, that on both sides the
225
+connectivity has been requested.
226
+
227
+Once a Neutron instance determines that an interconnection is symmetrically
228
+defined, further exchanges happen to determine the network parameters to use
229
+to realize the interconnection:
230
+
231
+* the Neutron (e.g. Neutron B) that just confirmed the symmetricity allocates
232
+  the required network identifiers, and asks the remote Neutron instance (A)
233
+  to refresh its state
234
+
235
+* Neutron A refreshes its state: checks symmetricity again (which now succeeds)
236
+  retrieves at the same time the network identifiers allocated by B,
237
+  and asks Neutron B to refresh
238
+
239
+* Neutron B refreshes again, this time retrieves at the same time the
240
+  network identifiers allocated by A
241
+
242
+In the above, "ask the remote Neutron instance to refresh its state" is
243
+done with a PUT on a specific ``refresh`` action on the ``interconnection``.
244
+
245
+.. _lifecycle:
246
+
247
+Interconnection lifecycle
248
+-------------------------
249
+
250
+In the previous section, it is implicit that an ``interconnection`` is along
251
+its life in different states before it is ultimately realized.  When an
252
+``interconnection`` resource is deleted on one side, the other side need also
253
+to ultimately be able to update its own state (if only for cleanup purpose
254
+or giving proper feedback to end users).
255
+
256
+Additionally, the interaction between a Neutron instance with another Neutron
257
+instance needs to happen out of the API call processing path, because it is not
258
+desirable that the success of a local API call would depend on the success of
259
+an operation with an external component which possibly would not be available
260
+at the moment.
261
+
262
+For all these reasons, a state machine will be introduced to handle the
263
+lifecycle of an interconnection resource, with triggered and periodic operation
264
+being done out-of-band of the API calls, to handle the operations for each
265
+state.
266
+
267
+Exposing this state in the API will allow:
268
+
269
+* end users to have feedback on how close they are to having something working
270
+
271
+* each Neutron instance to possibly identify that the remote state
272
+  is inconsistent with the local state
273
+
274
+State machine summary:
275
+
276
+TO_VALIDATE
277
+  interconnection resource has been created, but the existence of the
278
+  symmetric interconnection hasn't been validated yet
279
+
280
+VALIDATED
281
+  the existence of the symmetric interconnection has been validated
282
+
283
+ALLOCATED
284
+  local interconnection parameters have been defined (remote parameters are
285
+  still unknown)
286
+
287
+CONFIGURED
288
+  both local parameters and remote parameters are known, interconnection is
289
+  being built
290
+
291
+ACTIVE
292
+  interconnection has been setup, it should work
293
+
294
+TEARDOWN
295
+  local action taken to delete this interconnection, action
296
+  is being taken to have the remote state get in sync
297
+
298
+(DELETED)
299
+  implicit state corresponding to the resource not existing anymore
300
+
301
+.. blockdiag::
302
+
303
+    blockdiag {
304
+       default_shape = roundedbox
305
+
306
+       BEFORE_CREATION [shape = beginpoint, label=""]
307
+       TO_VALIDATE
308
+       VALIDATED
309
+       ALLOCATED
310
+       CONFIGURED
311
+       ACTIVE
312
+       TEARDOWN
313
+       DELETED [shape = endpoint, label=""]
314
+
315
+       BEFORE_CREATION -> TO_VALIDATE [folded]
316
+       TO_VALIDATE -> VALIDATED
317
+       TO_VALIDATE -> TEARDOWN [folded]
318
+       VALIDATED -> ALLOCATED
319
+       VALIDATED -> TEARDOWN [folded]
320
+       ALLOCATED -> CONFIGURED
321
+       ALLOCATED -> TEARDOWN [folded]
322
+       CONFIGURED -> ACTIVE
323
+       CONFIGURED -> TEARDOWN [folded]
324
+       ACTIVE -> TEARDOWN [folded]
325
+       TEARDOWN -> DELETED [folded]
326
+       ACTIVE -> TO_VALIDATE
327
+    }
328
+
329
+REST API Impact
330
+---------------
331
+
332
+The proposal is to introduce an API extension ``inter``, with a new
333
+``interconnection`` resource.
334
+
335
+Interconnection resource
336
+~~~~~~~~~~~~~~~~~~~~~~~~
337
+
338
+The new ``interconnection`` API resource will be introduce under the
339
+``inter`` API prefix, and having the following attributes:
340
+
341
++------------------+--------+----------------+--------------------------------+
342
+|Attribute Name    |Type    |Access          | Comment                        |
343
++==================+========+================+================================+
344
+|id                | uuid   | RO             |                                |
345
++------------------+--------+----------------+--------------------------------+
346
+|project_id        | uuid   | RO             |                                |
347
++------------------+--------+----------------+--------------------------------+
348
+|type              | enum   | RO             | ``router``, ``network_l2``,    |
349
+|                  |        |                | ``network_l3``                 |
350
++------------------+--------+----------------+--------------------------------+
351
+|state             | enum   | RO             | see states in :ref:`lifecycle` |
352
+|                  |        | will be updated|                                |
353
+|                  |        | by Neutron     |                                |
354
+|                  |        | along the life |                                |
355
+|                  |        | of the resource|                                |
356
++------------------+--------+----------------+--------------------------------+
357
+|name              | string | RW             |                                |
358
++------------------+--------+----------------+--------------------------------+
359
+|local_resource_id | uuid   | RO             | router or network UUID         |
360
++------------------+--------+----------------+--------------------------------+
361
+|remote_resource_id| uuid   | RO             | router or network UUID         |
362
++------------------+--------+----------------+--------------------------------+
363
+|remote_keystone   | string | RO             | AUTH_URL of remote             |
364
+|                  |        |                | keystone                       |
365
++------------------+--------+----------------+--------------------------------+
366
+|remote_region     | string | RO             | region in remote keystone      |
367
++------------------+--------+----------------+--------------------------------+
368
+|local_parameters  | dict   | RO             |                                |
369
++------------------+--------+                |                                |
370
+|remote_parameters | dict   | will be updated|                                |
371
+|                  |        | by Neutron     |                                |
372
+|                  |        | along the life |                                |
373
+|                  |        | of the resource|                                |
374
++------------------+--------+----------------+--------------------------------+
375
+
376
+This resource will be used with typical CRUD operations:
377
+
378
+* ``POST /v2.0/inter/interconnections``
379
+
380
+* ``GET /v2.0/inter/interconnections``
381
+
382
+* ``GET /v2.0/inter/interconnections/<uuid>``
383
+
384
+* ``PUT /v2.0/inter/interconnections/<uuid>``
385
+
386
+* ``DELETE /v2.0/inter/interconnections/<uuid>``
387
+
388
+Additionally, an additional REST operation is introduced to trigger a
389
+``refresh`` action on an ``interconnection`` resource:
390
+
391
+* ``PUT /v2.0/inter/interconnections/<uuid>/refresh``
392
+
393
+When this action is triggered the neutron instance on which the call is made
394
+will try to retrieve (``GET``) an ``interconnection`` resource on the remote
395
+neutron instance having same ``type`` as the local resource,
396
+as ``local_resource_id`` the local resource ``remote_resource_id``, and
397
+as ``remote_resource_id`` the local resource ``local_resource_id``.  Depending
398
+on the current local state, and depending on success or failure to find such
399
+a resource, the local state machine will transition.
400
+
401
+Example
402
+~~~~~~~
403
+
404
+This shows an example of API exchanges for a Neutron-Neutron interconnection
405
+between two Networks.
406
+
407
+API Call A1, from tenant user to Neutron A::
408
+
409
+    POST /v2.0/inter/interconnections
410
+         {'interconnection':
411
+             'type': 'network'
412
+             'local_resource_id': <uuid of network X>
413
+             'remote_keystone': "http//<keystone-B>/identity",
414
+             'remote_region': 'RegionOne',
415
+             'remote_resource_id': <uuid of network Y>
416
+             }
417
+         }
418
+
419
+    Response: 200 OK
420
+
421
+    {'interconnection': {
422
+         'id': <uuid 1>
423
+         ...
424
+     }}
425
+
426
+API Call B1, from Neutron A to Neutron B::
427
+
428
+    GET /v2.0/inter/interconnections?local_resource_id=<uuid of network Y>&remote_resource_id=<uuid of network X>
429
+
430
+    Response: 404
431
+
432
+API Call A2, from tenant user to Neutron B::
433
+
434
+    POST /v2.0/inter/interconnections
435
+         {'interconnection':
436
+             'type': 'network'
437
+             'local_resource_id': <uuid of network Y>
438
+             'remote_keystone': "http//<keystone-A>/identity",
439
+             'remote_region': 'RegionOne',
440
+             'remote_resource_id': <uuid of network X>
441
+             }
442
+         }
443
+
444
+    Response: 200 OK
445
+
446
+    {'interconnection': {
447
+         'id': <uuid 2>
448
+         ...
449
+     }}
450
+
451
+API Call B2, from Neutron B to Neutron A::
452
+
453
+    GET /v2.0/inter/interconnections/local_resource_id=<uuid of network X>&remote_resource_id=<uuid of network Y>
454
+
455
+    Response: 200 OK
456
+
457
+    {'interconnection': {
458
+         'id': <uuid 1>
459
+         ...
460
+         'local_parameters": {}
461
+     }}
462
+
463
+API Call B3' from Neutron B to Neutron A::
464
+
465
+    PUT /v2.0/inter/interconnections/<uuid 1>/refresh
466
+
467
+    {'interconnection': {
468
+         'id': <uuid 2>
469
+         'local_parameters": {}
470
+     }}
471
+
472
+    Response: 200 OK
473
+
474
+
475
+API Call B3'', from Neutron A to Neutron B ::
476
+
477
+    PUT /v2.0/inter/interconnections/<uuid 2>/refresh
478
+
479
+    Response: 200 OK
480
+
481
+    {'interconnection': {
482
+         'id': <uuid 2>
483
+         'local_parameters": {
484
+             'foo': '42'
485
+         }
486
+     }}
487
+
488
+API Call B3''', from Neutron B to Neutron A ::
489
+
490
+    PUT /v2.0/inter/interconnections/<uuid 1>/refresh
491
+
492
+    Response: 200 OK
493
+
494
+    {'interconnection': {
495
+         'id': <uuid 1>
496
+         'remote_parameters': {
497
+             'foo': '42'
498
+         }
499
+         'local_parameters": {
500
+             'bar': '43'
501
+         }
502
+     }}
503
+
504
+
505
+Command Line Client Impact
506
+--------------------------
507
+
508
+python-neutronclient will be updated to introduce an OSC extension to
509
+create/remove/update/delete ``interconnection`` resources.
510
+
511
+Client Libraries Impact
512
+--------------------------
513
+
514
+python-neutronclient and openstacksdk will need to be updated to support
515
+create/remove/update/delete operations on ``interconnection`` resources.
516
+
517
+Credits
518
+-------
519
+
520
+Przemyslaw Jasek contributed to exploring ideas that lead to this proposal
521
+during a six-months internship at Orange.
522
+
523
+
524
+References
525
+==========
526
+
527
+-  OpenStack Summit Sydney, lightning talk
528
+   https://www.openstack.org/videos/sydney-2017/neutron-neutron-interconnections
529
+
530
+.. _networking-bgpvpn: https://docs.openstack.org/networking-bgpvpn
531
+.. _networking-l2gw: https://docs.openstack.org/networking-l2gw
532
+
533
+Footnotes
534
+=========
535
+
536
+.. [#f1] possible with floating IPs, VPNaaS, but not with the BGP VPN
537
+   interconnections API extension (using a BGPVPN does not require admin
538
+   right, but creating a new BGPVPN does require admin rights)
539
+
540
+.. [#f2] possible with VPNaaS, and BGP VPN interconnections, but not with
541
+   floating IPs
542
+
543
+.. [#f3] possible with floating IPs and BGP VPN interconnections, but by
544
+   definition not with VPNaaS

Loading…
Cancel
Save