Merge "Network Custom Resource Definition De-facto Standard support"
This commit is contained in:
Zuul
Gerrit Code Review
commit
6aade4cbfb
@ -34,6 +34,7 @@ Design Specs
|
||||
specs/pike/fuxi_kubernetes
|
||||
specs/pike/sriov
|
||||
specs/queens/network_policy
|
||||
specs/rocky/npwg_spec_support
|
||||
|
||||
Indices and tables
|
||||
------------------
|
||||
|
||||
190
doc/source/specs/rocky/npwg_spec_support.rst
Normal file
190
doc/source/specs/rocky/npwg_spec_support.rst
Normal file
@ -0,0 +1,190 @@
|
||||
==========================================================================
|
||||
Kubernetes Network Custom Resource Definition De-facto Standard v1 Support
|
||||
==========================================================================
|
||||
|
||||
https://blueprints.launchpad.net/kuryr-kubernetes/+spec/kuryr-npwg-spec-support
|
||||
|
||||
This spec proposes an approach to support the mechanism defined in Kubernetes
|
||||
Network Custom Resource Definition De-facto Standard Version 1 [#]_, which is
|
||||
used to attach multiple VIFs to Pods.
|
||||
|
||||
Problem Description
|
||||
-------------------
|
||||
|
||||
There is always a desire for Pods to be able to be attached to multiple
|
||||
interfaces in NFV use cases. However CNI plugins which have implemented this
|
||||
functionality are using different way of defining the additional interfaces in
|
||||
Pods. There is no standard approach among those CNI plugins.
|
||||
|
||||
Therefore, the Networking Plumbing Working Group [#]_ drafted a spec (the NPWG
|
||||
spec) trying to standardize the the way of attaching Pods to multiple networks.
|
||||
|
||||
Proposed Change
|
||||
---------------
|
||||
|
||||
The NPWG spec defines a "Network" Custom Resource object which describes how to
|
||||
attach a Pod to the logical or physical network referenced by the object.
|
||||
|
||||
The proposed change is based on VIF-Handler And Vif Drivers Design [#]_. A new
|
||||
VIF driver 'npwg_multiple_interfaces' will be created to parse the annotation
|
||||
of Pods and Network CRDs. The new VIF driver will be invoked by the multi-vif
|
||||
driver as another sub-drivers. It should return a list of VIF objects. The
|
||||
'npwg_multiple_interfaces' should invoke other VIF driver to create the vif
|
||||
objects if it is necessary.
|
||||
|
||||
The VIFHandler then updates the Pod annotation of 'openstack.org/kuryr-vif'
|
||||
with the VIF objects. So that the Kuryr CNI can read these VIFs, and attaches
|
||||
each of them to Pods namespace. If any of the additional interfaces failed to
|
||||
be attached to the Pod, or any error happens during attachment, the CNI shall
|
||||
return with error.
|
||||
|
||||
Option in config file might look like this:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[kubernetes]
|
||||
|
||||
enabled_vif_drivers = npwg_multiple_interfaces
|
||||
|
||||
To define additional network in Pods, NPWG spec defines format of annotation.
|
||||
Here's how a Pod Spec with additional networks requests might look like:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: my-pod
|
||||
namespace: my-namespace
|
||||
annotations:
|
||||
kubernetes.v1.cni.cncf.io/networks: net-a,net-b,other-ns/net-c
|
||||
|
||||
Or in JSON format like:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: my-pod
|
||||
namespace: my-namespace
|
||||
annotations:
|
||||
kubernetes.v1.cni.cncf.io/networks: |
|
||||
[
|
||||
{"name":"net-a"},
|
||||
{"name":"net-b"},
|
||||
{
|
||||
"name":"net-c",
|
||||
"namespace":"other-ns"
|
||||
}
|
||||
]
|
||||
|
||||
Then the VIF driver can parse the network information defined in 'Network'
|
||||
objects. In NPWG spec, the 'Network' object definition is very flexible.
|
||||
Implementations that are not CNI delegating plugins can add annotations to the
|
||||
Network object and use those to store non-CNI configuration. And it is up to
|
||||
the implementation to define the content it requires.
|
||||
|
||||
Here is how 'Network' CRD specified in the NPWG spec.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: networks.kubernetes.cni.cncf.io
|
||||
spec:
|
||||
group: kubernetes.cni.cncf.io
|
||||
version: v1
|
||||
scope: Namespaced
|
||||
names:
|
||||
plural: networks
|
||||
singular: network
|
||||
kind: Network
|
||||
shortNames:
|
||||
- net
|
||||
validation:
|
||||
openAPIV3Schema:
|
||||
properties:
|
||||
spec:
|
||||
properties:
|
||||
config:
|
||||
type: string
|
||||
|
||||
For Kuryr-kubernetes, users should define the 'Network' object with a Neutron
|
||||
subnet created previously like:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
apiVersion: "kubernetes.cni.cncf.io/v1"
|
||||
kind: Network
|
||||
metadata:
|
||||
name: a-bridge-network
|
||||
annotations:
|
||||
openstack.org/kuryr-config: '{
|
||||
"subnetId": "id_of_neutron_subnet_created_previously"
|
||||
}'
|
||||
|
||||
With information read from Pod annotation kubernetes.v1.cni.cncf.io/networks
|
||||
and 'Network' objects, the Neutron ports could either be created or retrieved.
|
||||
Then the Pod annotation openstack.org/kuryr-vif will be updated accordingly.
|
||||
|
||||
Here's how openstack.org/kuryr-vif annotation with additional networks might
|
||||
look like:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: my-pod
|
||||
namespace: my-namespace
|
||||
annotations:
|
||||
openstack.org/kuryr-vif: {
|
||||
# default interface remains intact
|
||||
"eth0": {
|
||||
... Neutron vif object from default subnet ...
|
||||
}
|
||||
# additional interfaces appended by driver 'npwg_multiple_interfaces'
|
||||
"eth1": {
|
||||
... Neutron vif object ...
|
||||
}
|
||||
"eth2": {
|
||||
... Neutron vif object ...
|
||||
}
|
||||
}
|
||||
|
||||
Alternatives
|
||||
~~~~~~~~~~~~
|
||||
|
||||
Currently, Kuryr-Kubernetes has already designed a way of defining additional
|
||||
VIF. This spec will not change that part. Users can choose using which
|
||||
format they want by configuring 'enabled_vif_drivers'.
|
||||
|
||||
Other end user impact
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
Pods always attach the default Kubernetes network as how Kuryr-Kubernetes works
|
||||
today, and all networks specified in the Pod annotation are sidecars.
|
||||
|
||||
Assignee(s)
|
||||
~~~~~~~~~~~
|
||||
|
||||
Primary assignee:
|
||||
Peng Liu
|
||||
|
||||
Work Items
|
||||
~~~~~~~~~~
|
||||
|
||||
* Implement a new NPWG spec compatible VIF driver.
|
||||
* Document the procedure of using this new VIF driver.
|
||||
|
||||
Possible Further Work
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* To keep on track of the subsequent releases of NPWG spec.
|
||||
* To allow defining new neutron network/subnet in 'Network' objects, so that
|
||||
kuryr can create them in Neutron first, then attach Pod to it.
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
.. [#] https://docs.google.com/document/d/1Ny03h6IDVy_e_vmElOqR7UdTPAG_RNydhVE1Kx54kFQ/edit?usp=sharing
|
||||
.. [#] https://groups.google.com/forum/?_escaped_fragment_=topic/kubernetes-sig-network/ANAjTyqVosw
|
||||
.. [#] https://docs.openstack.org/kuryr-kubernetes/latest/devref/vif_handler_drivers_design.html
|
||||
Loading…
Reference in New Issue
Block a user