diff --git a/doc/source/devref/kuryr_kubernetes_design.rst b/doc/source/devref/kuryr_kubernetes_design.rst index dd72e8b62..e35849eaf 100644 --- a/doc/source/devref/kuryr_kubernetes_design.rst +++ b/doc/source/devref/kuryr_kubernetes_design.rst @@ -55,16 +55,16 @@ Please see below the component view of the integrated system: Design Principles ----------------- -1. Loose coupling between integration components. -2. Flexible deployment options to support different project, subnet and +#. Loose coupling between integration components. +#. Flexible deployment options to support different project, subnet and security groups assignment profiles. -3. The communication of the pod binding data between Controller and CNI driver +#. The communication of the pod binding data between Controller and CNI driver should rely on existing communication channels, currently added to the pod metadata via annotations. -4. CNI Driver should not depend on Neutron. It gets all required details +#. CNI Driver should not depend on Neutron. It gets all required details from Kubernetes API server (currently through Kubernetes annotations), therefore depending on Controller to perform its translation tasks. -5. Allow different neutron backends to bind Kubernetes pods without code +#. Allow different neutron backends to bind Kubernetes pods without code modification. This means that both Controller and CNI binding mechanism should allow loading of the vif management and binding components, manifested via configuration. If some vendor requires some extra code, it diff --git a/doc/source/devref/kuryr_kubernetes_ocp_route_design.rst b/doc/source/devref/kuryr_kubernetes_ocp_route_design.rst index 05f9f9bb7..2ca80b9d7 100644 --- a/doc/source/devref/kuryr_kubernetes_ocp_route_design.rst +++ b/doc/source/devref/kuryr_kubernetes_ocp_route_design.rst @@ -95,17 +95,17 @@ B. Create Service/Endpoints, create OCP-Route, delete OCP-Route. * OCP-Route details : - .. code-block:: yaml + .. code-block:: yaml - apiVersion: v1 - kind: Route - metadata: - name: test - spec: - host: www.example.com - to: - kind: Service - name: s1 + apiVersion: v1 + kind: Route + metadata: + name: test + spec: + host: www.example.com + to: + kind: Service + name: s1 * Since it's the first route pointing to this Service, the OCP-Route handler will create LbaaS pool (attached to L7 router)- named diff --git a/doc/source/devref/port_crd_usage.rst b/doc/source/devref/port_crd_usage.rst index 131f5254c..1c2017157 100644 --- a/doc/source/devref/port_crd_usage.rst +++ b/doc/source/devref/port_crd_usage.rst @@ -179,9 +179,9 @@ not be completely waterproof in all situations (e.g., if there is another entity using the same device owner name). Consequently, by storing the information into K8s CRD objects we have the benefit of: - * Calling K8s API instead of Neutron API - * Being sure the recovered ports into the pools were created by - kuryr-controller +* Calling K8s API instead of Neutron API +* Being sure the recovered ports into the pools were created by + kuryr-controller In addition to these advantages, moving to CRDs will easier the transition for kuryr-cni handling the ports pools as kuryr-cni has access to the K8S API but diff --git a/doc/source/installation/devstack/dragonflow_support.rst b/doc/source/installation/devstack/dragonflow_support.rst index bb7769510..866daf643 100644 --- a/doc/source/installation/devstack/dragonflow_support.rst +++ b/doc/source/installation/devstack/dragonflow_support.rst @@ -31,72 +31,72 @@ and then cover a nested environment where containers are created inside VMs. Single Node Test Environment ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. Create a test system. +#. Create a test system. -It's best to use a throwaway dev system for running DevStack. Your best bet is -to use either Fedora 25 or the latest Ubuntu LTS (16.04, Xenial). + It's best to use a throwaway dev system for running DevStack. Your best bet + is to use either Fedora 25 or the latest Ubuntu LTS (16.04, Xenial). -2. Create the ``stack`` user. +#. Create the ``stack`` user. -.. code-block:: console + .. code-block:: console - $ git clone https://opendev.org/openstack-dev/devstack.git - $ sudo ./devstack/tools/create-stack-user.sh + $ git clone https://opendev.org/openstack-dev/devstack.git + $ sudo ./devstack/tools/create-stack-user.sh -3. Switch to the ``stack`` user and clone DevStack and kuryr-kubernetes. +#. Switch to the ``stack`` user and clone DevStack and kuryr-kubernetes. -.. code-block:: console + .. code-block:: console - $ sudo su - stack - $ git clone https://opendev.org/openstack-dev/devstack.git - $ git clone https://opendev.org/openstack/kuryr-kubernetes.git + $ sudo su - stack + $ git clone https://opendev.org/openstack-dev/devstack.git + $ git clone https://opendev.org/openstack/kuryr-kubernetes.git -4. Configure DevStack to use Dragonflow. +#. Configure DevStack to use Dragonflow. -kuryr-kubernetes comes with a sample DevStack configuration file for Dragonflow -you can start with. You may change some values for the various variables in -that file, like password settings or what LBaaS service provider to use. -Feel free to edit it if you'd like, but it should work as-is. + kuryr-kubernetes comes with a sample DevStack configuration file for + Dragonflow you can start with. You may change some values for the various + variables in that file, like password settings or what LBaaS service + provider to use. Feel free to edit it if you'd like, but it should work + as-is. -.. code-block:: console + .. code-block:: console - $ cd devstack - $ cp ../kuryr-kubernetes/devstack/local.conf.df.sample local.conf + $ cd devstack + $ cp ../kuryr-kubernetes/devstack/local.conf.df.sample local.conf -Optionally, the ports pool funcionality can be enabled by following: -`How to enable ports pool with devstack`_. + Optionally, the ports pool functionality can be enabled by following: + `How to enable ports pool with devstack`_. -5. Run DevStack. +#. Run DevStack. -Expect it to take a while. It installs required packages, clones a bunch -of git repos, and installs everything from these git repos. + Expect it to take a while. It installs required packages, clones a bunch of + git repos, and installs everything from these git repos. -.. code-block:: console + .. code-block:: console - $ ./stack.sh + $ ./stack.sh + Once DevStack completes successfully, you should see output that looks + something like this: -Once DevStack completes successfully, you should see output that looks -something like this: + .. code-block:: console -.. code-block:: console + This is your host IP address: 192.168.5.10 + This is your host IPv6 address: ::1 + Keystone is serving at http://192.168.5.10/identity/ + The default users are: admin and demo + The password: pass - This is your host IP address: 192.168.5.10 - This is your host IPv6 address: ::1 - Keystone is serving at http://192.168.5.10/identity/ - The default users are: admin and demo - The password: pass +#. Extra configurations. + Create NAT rule that will cause "external" traffic from your instances to + get rewritten to your network controller's ip address and sent out on the + network: -6. Extra configurations. + .. code-block:: console -Create NAT rule that will cause "external" traffic from your instances to get -rewritten to your network controller's ip address and sent out on the network: - -.. code-block:: console - - $ sudo iptables -t nat -I POSTROUTING 1 -s 172.24.4.1/24 -j MASQUERADE + $ sudo iptables -t nat -I POSTROUTING 1 -s 172.24.4.1/24 -j MASQUERADE Inspect default Configuration @@ -140,20 +140,15 @@ use (step 4), in this case: The main differences with the default dragonflow local.conf sample are that: - - There is no need to enable the kuryr-kubernetes plugin as this will be - installed inside the VM (overcloud). - - - There is no need to enable the kuryr related services as they will also - be installed inside the VM: kuryr-kubernetes, kubelet, - kubernetes-api, kubernetes-controller-manager, kubernetes-scheduler and - kubelet. - - - Nova and Glance components need to be enabled to be able to create the VM - where we will install the overcloud. - - - Dragonflow Trunk service plugin need to be enable to ensure Trunk ports - support. - +- There is no need to enable the kuryr-kubernetes plugin as this will be + installed inside the VM (overcloud). +- There is no need to enable the kuryr related services as they will also be + installed inside the VM: kuryr-kubernetes, kubelet, kubernetes-api, + kubernetes-controller-manager, kubernetes-scheduler and kubelet. +- Nova and Glance components need to be enabled to be able to create the VM + where we will install the overcloud. +- Dragonflow Trunk service plugin need to be enable to ensure Trunk ports + support. Once the undercloud deployment has finished, the next steps are related to creating the overcloud VM by using a parent port of a Trunk so that containers @@ -168,17 +163,16 @@ Once the VM is up and running, we can start with the overcloud configuration. The steps to perform are the same as without Dragonflow integration, i.e., the same steps as for ML2/OVS: -1. Log in into the VM: +#. Log in into the VM: .. code-block:: console $ ssh -i id_rsa_demo centos@FLOATING_IP -2. Deploy devstack following steps 3 and 4 detailed at +#. Deploy devstack following steps 3 and 4 detailed at `How to try out nested-pods locally (VLAN + trunk)`_. - Testing Nested Network Connectivity +++++++++++++++++++++++++++++++++++ diff --git a/doc/source/installation/devstack/nested-macvlan.rst b/doc/source/installation/devstack/nested-macvlan.rst index efb2580bc..c1fcfab8e 100644 --- a/doc/source/installation/devstack/nested-macvlan.rst +++ b/doc/source/installation/devstack/nested-macvlan.rst @@ -5,46 +5,46 @@ How to try out nested-pods locally (MACVLAN) Following are the instructions for an all-in-one setup, using the nested MACVLAN driver rather than VLAN and trunk ports. -1. To install OpenStack services run devstack with +#. To install OpenStack services run devstack with ``devstack/local.conf.pod-in-vm.undercloud.sample``. -2. Launch a Nova VM with MACVLAN support +#. Launch a Nova VM with MACVLAN support -.. todo:: + .. todo:: - Add a list of neutron commands, required to launch a such a VM + Add a list of neutron commands, required to launch a such a VM -3. Log into the VM and set up Kubernetes along with Kuryr using devstack: +#. Log into the VM and set up Kubernetes along with Kuryr using devstack: - Since undercloud Neutron will be used by pods, Neutron services should be disabled in localrc. - Run devstack with ``devstack/local.conf.pod-in-vm.overcloud.sample``. Fill in the needed information, such as the subnet pool id to use or the router. -4. Once devstack is done and all services are up inside VM. Next steps are to +#. Once devstack is done and all services are up inside VM. Next steps are to configure the missing information at ``/etc/kuryr/kuryr.conf``: - - Configure worker VMs subnet: + - Configure worker VMs subnet: .. code-block:: ini [pod_vif_nested] worker_nodes_subnet = - - Configure "pod_vif_driver" as "nested-macvlan": + - Configure "pod_vif_driver" as "nested-macvlan": .. code-block:: ini [kubernetes] pod_vif_driver = nested-macvlan - - Configure binding section: + - Configure binding section: .. code-block:: ini [binding] link_iface = - - Restart kuryr-k8s-controller: + - Restart kuryr-k8s-controller: .. code-block:: console diff --git a/doc/source/installation/devstack/nested-vlan.rst b/doc/source/installation/devstack/nested-vlan.rst index a4d0965bc..18e9c73cc 100644 --- a/doc/source/installation/devstack/nested-vlan.rst +++ b/doc/source/installation/devstack/nested-vlan.rst @@ -7,7 +7,7 @@ also be running inside the same Nova VM in which Kuryr-controller and Kuryr-cni will be running. 4GB memory and 2 vCPUs, is the minimum resource requirement for the VM: -1. To install OpenStack services run devstack with +#. To install OpenStack services run devstack with ``devstack/local.conf.pod-in-vm.undercloud.sample``. Ensure that "trunk" service plugin is enabled in ``/etc/neutron/neutron.conf``: @@ -16,57 +16,57 @@ for the VM: [DEFAULT] service_plugins = neutron.services.l3_router.l3_router_plugin.L3RouterPlugin,neutron.services.trunk.plugin.TrunkPlugin -2. Launch a VM with `Neutron trunk port`_. The next steps can be followed: +#. Launch a VM with `Neutron trunk port`_. The next steps can be followed: `Boot VM with a Trunk Port`_. -3. Inside VM, install and setup Kubernetes along with Kuryr using devstack: - - Since undercloud Neutron will be used by pods, Neutron services should be - disabled in localrc. - - Run devstack with ``devstack/local.conf.pod-in-vm.overcloud.sample``. - but first fill in the needed information: +#. Inside VM, install and setup Kubernetes along with Kuryr using devstack: - - Point to the undercloud deployment by setting: + - Since undercloud Neutron will be used by pods, Neutron services should be + disabled in localrc. + - Run devstack with ``devstack/local.conf.pod-in-vm.overcloud.sample``. + But first fill in the needed information: + + - Point to the undercloud deployment by setting: .. code-block:: bash SERVICE_HOST=UNDERCLOUD_CONTROLLER_IP - - Fill in the subnetpool id of the undercloud deployment, as well as - the router where the new pod and service networks need to be - connected: + - Fill in the subnetpool id of the undercloud deployment, as well as the + router where the new pod and service networks need to be connected: .. code-block:: bash KURYR_NEUTRON_DEFAULT_SUBNETPOOL_ID=UNDERCLOUD_SUBNETPOOL_V4_ID KURYR_NEUTRON_DEFAULT_ROUTER=router1 - - Ensure the nested-vlan driver is going to be set by setting: + - Ensure the nested-vlan driver is going to be set by setting: .. code-block:: bash KURYR_POD_VIF_DRIVER=nested-vlan - - Optionally, the ports pool funcionality can be enabled by following: - `How to enable ports pool with devstack`_. + - Optionally, the ports pool funcionality can be enabled by following: + `How to enable ports pool with devstack`_. - - [OPTIONAL] If you want to enable the subport pools driver and the - VIF Pool Manager you need to include: + - [OPTIONAL] If you want to enable the subport pools driver and the VIF + Pool Manager you need to include: .. code-block:: bash KURYR_VIF_POOL_MANAGER=True -4. Once devstack is done and all services are up inside VM. Next steps are to +#. Once devstack is done and all services are up inside VM. Next steps are to configure the missing information at ``/etc/kuryr/kuryr.conf``: - - Configure worker VMs subnet: + - Configure worker VMs subnet: .. code-block:: ini [pod_vif_nested] worker_nodes_subnet = - - Configure binding section: + - Configure binding section: .. code-block:: ini @@ -74,13 +74,13 @@ for the VM: driver = kuryr.lib.binding.drivers.vlan link_iface = - - Restart kuryr-k8s-controller: + - Restart kuryr-k8s-controller: .. code-block:: console $ sudo systemctl restart devstack@kuryr-kubernetes.service - - Restart kuryr-daemon: + - Restart kuryr-daemon: .. code-block:: console diff --git a/doc/source/installation/devstack/odl_support.rst b/doc/source/installation/devstack/odl_support.rst index 0608547cf..0aceccd1b 100644 --- a/doc/source/installation/devstack/odl_support.rst +++ b/doc/source/installation/devstack/odl_support.rst @@ -26,19 +26,19 @@ and then cover a nested environment where containers are created inside VMs. Single Node Test Environment ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. Create a test system. +#. Create a test system. It's best to use a throwaway dev system for running DevStack. Your best bet is to use either CentOS 7 or the latest Ubuntu LTS (16.04, Xenial). -2. Create the ``stack`` user. +#. Create the ``stack`` user. .. code-block:: console $ git clone https://opendev.org/openstack-dev/devstack.git $ sudo ./devstack/tools/create-stack-user.sh -3. Switch to the ``stack`` user and clone DevStack and kuryr-kubernetes. +#. Switch to the ``stack`` user and clone DevStack and kuryr-kubernetes. .. code-block:: console @@ -46,32 +46,32 @@ to use either CentOS 7 or the latest Ubuntu LTS (16.04, Xenial). $ git clone https://opendev.org/openstack-dev/devstack.git $ git clone https://opendev.org/openstack/kuryr-kubernetes.git -4. Configure DevStack to use ODL. +#. Configure DevStack to use ODL. -kuryr-kubernetes comes with a sample DevStack configuration file for ODL you -can start with. For example, you may want to set some values for the various -PASSWORD variables in that file, or change the LBaaS service provider to use. -Feel free to edit it if you'd like, but it should work as-is. + kuryr-kubernetes comes with a sample DevStack configuration file for ODL you + can start with. For example, you may want to set some values for the various + PASSWORD variables in that file, or change the LBaaS service provider to + use. Feel free to edit it if you'd like, but it should work as-is. .. code-block:: console $ cd devstack $ cp ../kuryr-kubernetes/devstack/local.conf.odl.sample local.conf -Optionally, the ports pool funcionality can be enabled by following: -`How to enable ports pool with devstack`_. + Optionally, the ports pool functionality can be enabled by following: + `How to enable ports pool with devstack`_. -5. Run DevStack. +#. Run DevStack. -This is going to take a while. It installs a bunch of packages, clones a bunch -of git repos, and installs everything from these git repos. + This is going to take a while. It installs a bunch of packages, clones a + bunch of git repos, and installs everything from these git repos. .. code-block:: console $ ./stack.sh -Once DevStack completes successfully, you should see output that looks -something like this: + Once DevStack completes successfully, you should see output that looks + something like this: .. code-block:: console @@ -81,10 +81,10 @@ something like this: The default users are: admin and demo The password: pass -6. Extra configurations. +#. Extra configurations. -Devstack does not wire up the public network by default so we must do -some extra steps for floating IP usage as well as external connectivity: + Devstack does not wire up the public network by default so we must do some + extra steps for floating IP usage as well as external connectivity: .. code-block:: console @@ -92,9 +92,9 @@ some extra steps for floating IP usage as well as external connectivity: $ sudo ip route add 172.24.4.0/24 dev br-ex $ sudo ip addr add 172.24.4.1/24 dev br-ex -Then you can create forwarding and NAT rules that will cause "external" -traffic from your instances to get rewritten to your network controller's -ip address and sent out on the network: + Then you can create forwarding and NAT rules that will cause "external" + traffic from your instances to get rewritten to your network controller's ip + address and sent out on the network: .. code-block:: console @@ -144,18 +144,14 @@ local.conf to use (step 4), in this case: The main differences with the default odl local.conf sample are that: - - There is no need to enable the kuryr-kubernetes plugin as this will be - installed inside the VM (overcloud). - - - There is no need to enable the kuryr related services as they will also - be installed inside the VM: kuryr-kubernetes, kubelet, - kubernetes-api, kubernetes-controller-manager, kubernetes-scheduler and - kubelet. - - - Nova and Glance components need to be enabled to be able to create the VM - where we will install the overcloud. - - - ODL Trunk service plugin need to be enable to ensure Trunk ports support. +- There is no need to enable the kuryr-kubernetes plugin as this will be + installed inside the VM (overcloud). +- There is no need to enable the kuryr related services as they will also be + installed inside the VM: kuryr-kubernetes, kubelet, kubernetes-api, + kubernetes-controller-manager, kubernetes-scheduler and kubelet. +- Nova and Glance components need to be enabled to be able to create the VM + where we will install the overcloud. +- ODL Trunk service plugin need to be enable to ensure Trunk ports support. Once the undercloud deployment has finished, the next steps are related to create the overcloud VM by using a parent port of a Trunk so that containers @@ -167,16 +163,16 @@ Overcloud deployment ++++++++++++++++++++ Once the VM is up and running, we can start with the overcloud configuration. -The steps to perform are the same as without ODL integration, i.e., the -same steps as for ML2/OVS: +The steps to perform are the same as without ODL integration, i.e., the same +steps as for ML2/OVS: -1. Log in into the VM: +#. Log in into the VM: .. code-block:: console $ ssh -i id_rsa_demo centos@FLOATING_IP -2. Deploy devstack following steps 3 and 4 detailed at +#. Deploy devstack following steps 3 and 4 detailed at `How to try out nested-pods locally (VLAN + trunk)`_. diff --git a/doc/source/installation/devstack/ovn_support.rst b/doc/source/installation/devstack/ovn_support.rst index fa6cbe365..464f7cb0d 100644 --- a/doc/source/installation/devstack/ovn_support.rst +++ b/doc/source/installation/devstack/ovn_support.rst @@ -23,19 +23,19 @@ and then cover a nested environment where containers are created inside VMs. Single Node Test Environment ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. Create a test system. +#. Create a test system. -It's best to use a throwaway dev system for running DevStack. Your best bet is -to use either CentOS 7 or the latest Ubuntu LTS (16.04, Xenial). + It's best to use a throwaway dev system for running DevStack. Your best bet + is to use either CentOS 7 or the latest Ubuntu LTS (16.04, Xenial). -2. Create the ``stack`` user. +#. Create the ``stack`` user. .. code-block:: console $ git clone https://opendev.org/openstack-dev/devstack.git $ sudo ./devstack/tools/create-stack-user.sh -3. Switch to the ``stack`` user and clone DevStack and kuryr-kubernetes. +#. Switch to the ``stack`` user and clone DevStack and kuryr-kubernetes. .. code-block:: console @@ -43,48 +43,49 @@ to use either CentOS 7 or the latest Ubuntu LTS (16.04, Xenial). $ git clone https://opendev.org/openstack-dev/devstack.git $ git clone https://opendev.org/openstack/kuryr-kubernetes.git -4. Configure DevStack to use OVN. +#. Configure DevStack to use OVN. -kuryr-kubernetes comes with a sample DevStack configuration file for OVN you -can start with. For example, you may want to set some values for the various -PASSWORD variables in that file, or change the LBaaS service provider to use. -Feel free to edit it if you'd like, but it should work as-is. + kuryr-kubernetes comes with a sample DevStack configuration file for OVN you + can start with. For example, you may want to set some values for the various + PASSWORD variables in that file, or change the LBaaS service provider to + use. Feel free to edit it if you'd like, but it should work as-is. .. code-block:: console $ cd devstack $ cp ../kuryr-kubernetes/devstack/local.conf.ovn.sample local.conf -Note that due to OVN compiling OVS from source at -/usr/local/var/run/openvswitch we need to state at the local.conf that the path -is different from the default one (i.e., /var/run/openvswitch). + Note that due to OVN compiling OVS from source at + /usr/local/var/run/openvswitch we need to state at the local.conf that the + path is different from the default one (i.e., /var/run/openvswitch). -Optionally, the ports pool functionality can be enabled by following: -:doc:`./ports-pool` + Optionally, the ports pool functionality can be enabled by following: + :doc:`./ports-pool` -5. Run DevStack. +#. Run DevStack. -This is going to take a while. It installs a bunch of packages, clones a bunch -of git repos, and installs everything from these git repos. + This is going to take a while. It installs a bunch of packages, clones a + bunch of git repos, and installs everything from these git repos. .. code-block:: console $ ./stack.sh -Once DevStack completes successfully, you should see output that looks -something like this:: + Once DevStack completes successfully, you should see output that looks + something like this: - This is your host IP address: 192.168.5.10 - This is your host IPv6 address: ::1 - Keystone is serving at http://192.168.5.10/identity/ - The default users are: admin and demo - The password: pass + .. code-block:: + This is your host IP address: 192.168.5.10 + This is your host IPv6 address: ::1 + Keystone is serving at http://192.168.5.10/identity/ + The default users are: admin and demo + The password: pass -6. Extra configurations. +#. Extra configurations. -Devstack does not wire up the public network by default so we must do -some extra steps for floating IP usage as well as external connectivity: + Devstack does not wire up the public network by default so we must do some + extra steps for floating IP usage as well as external connectivity: .. code-block:: console @@ -92,9 +93,9 @@ some extra steps for floating IP usage as well as external connectivity: $ sudo ip route add 172.24.4.0/24 dev br-ex $ sudo ip addr add 172.24.4.1/24 dev br-ex -Then you can create forwarding and NAT rules that will cause "external" -traffic from your instances to get rewritten to your network controller's -ip address and sent out on the network: + Then you can create forwarding and NAT rules that will cause "external" + traffic from your instances to get rewritten to your network controller's ip + address and sent out on the network: .. code-block:: console @@ -141,21 +142,17 @@ local.conf to use (step 4), in this case: $ cd devstack $ cp ../kuryr-kubernetes/devstack/local.conf.pod-in-vm.undercloud.ovn.sample local.conf + The main differences with the default ovn local.conf sample are that: - - There is no need to enable the kuryr-kubernetes plugin as this will be - installed inside the VM (overcloud). - - - There is no need to enable the kuryr related services as they will also - be installed inside the VM: kuryr-kubernetes, kubelet, - kubernetes-api, kubernetes-controller-manager, kubernetes-scheduler and - kubelet. - - - Nova and Glance components need to be enabled to be able to create the VM - where we will install the overcloud. - - - OVN Trunk service plugin need to be enable to ensure Trunk ports support. - +- There is no need to enable the kuryr-kubernetes plugin as this will be + installed inside the VM (overcloud). +- There is no need to enable the kuryr related services as they will also be + installed inside the VM: kuryr-kubernetes, kubelet, kubernetes-api, + kubernetes-controller-manager, kubernetes-scheduler and kubelet. +- Nova and Glance components need to be enabled to be able to create the VM + where we will install the overcloud. +- OVN Trunk service plugin need to be enable to ensure Trunk ports support. Once the undercloud deployment has finished, the next steps are related to create the overcloud VM by using a parent port of a Trunk so that containers @@ -170,13 +167,13 @@ Once the VM is up and running, we can start with the overcloud configuration. The steps to perform are the same as without OVN integration, i.e., the same steps as for ML2/OVS: -1. Log in into the VM: +#. Log in into the VM: .. code-block:: console $ ssh -i id_rsa_demo centos@FLOATING_IP -2. Deploy devstack following steps 3 and 4 detailed at :doc:`./nested-vlan` +#. Deploy devstack following steps 3 and 4 detailed at :doc:`./nested-vlan` Testing Nested Network Connectivity diff --git a/doc/source/installation/devstack/ports-pool.rst b/doc/source/installation/devstack/ports-pool.rst index 47c51d4fa..f2cd53a15 100644 --- a/doc/source/installation/devstack/ports-pool.rst +++ b/doc/source/installation/devstack/ports-pool.rst @@ -5,13 +5,13 @@ How to enable ports pool with devstack To enable the utilization of the ports pool feature through devstack, the next options needs to be set at the local.conf file: -1. First, you need to enable the pools by setting: +#. First, you need to enable the pools by setting: .. code-block:: bash KURYR_USE_PORT_POOLS=True -2. Then, the proper pool driver needs to be set. This means that for the +#. Then, the proper pool driver needs to be set. This means that for the baremetal case you need to ensure the pod vif driver and the vif pool driver are set to the right baremetal drivers, for instance: @@ -27,7 +27,7 @@ options needs to be set at the local.conf file: KURYR_POD_VIF_DRIVER=nested-vlan KURYR_VIF_POOL_DRIVER=nested -3. Then, in case you want to set a limit to the maximum number of ports, or +#. Then, in case you want to set a limit to the maximum number of ports, or increase/reduce the default one for the minimum number, as well as to modify the way the pools are repopulated, both in time as well as regarding bulk operation sizes, the next option can be included and modified accordingly: diff --git a/doc/source/installation/multi_vif_with_npwg_spec.rst b/doc/source/installation/multi_vif_with_npwg_spec.rst index 56dca14dc..119df604a 100644 --- a/doc/source/installation/multi_vif_with_npwg_spec.rst +++ b/doc/source/installation/multi_vif_with_npwg_spec.rst @@ -6,89 +6,90 @@ To create pods with additional Interfaces follow the `Kubernetes Network Custom Resource Definition De-facto Standard Version 1`_, the next steps can be followed: -1. Create Neutron net/subnets which you want the additional interfaces attach +#. Create Neutron net/subnets which you want the additional interfaces attach to. -.. code-block:: bash + .. code-block:: console - $ openstack network create net-a - $ openstack subnet create subnet-a --subnet-range 192.0.2.0/24 --network net-a + $ openstack network create net-a + $ openstack subnet create subnet-a --subnet-range 192.0.2.0/24 --network net-a -2. Create CRD of 'NetworkAttachmentDefinition' as defined in NPWG spec. +#. Create CRD of 'NetworkAttachmentDefinition' as defined in NPWG spec. -.. code-block:: bash + .. code-block:: console - $ cat << EOF > nad.yaml - apiVersion: apiextensions.k8s.io/v1beta1 - kind: CustomResourceDefinition - metadata: - name: network-attachment-definitions.k8s.cni.cncf.io - spec: - group: k8s.cni.cncf.io - version: v1 - scope: Namespaced - names: - plural: network-attachment-definitions - singular: network-attachment-definition - kind: NetworkAttachmentDefinition - shortNames: - - net-attach-def - validation: - openAPIV3Schema: - properties: - spec: - properties: - config: - type: string - EOF - $ kubectl apply -f nad.yaml + $ cat << EOF > nad.yaml + apiVersion: apiextensions.k8s.io/v1beta1 + kind: CustomResourceDefinition + metadata: + name: network-attachment-definitions.k8s.cni.cncf.io + spec: + group: k8s.cni.cncf.io + version: v1 + scope: Namespaced + names: + plural: network-attachment-definitions + singular: network-attachment-definition + kind: NetworkAttachmentDefinition + shortNames: + - net-attach-def + validation: + openAPIV3Schema: + properties: + spec: + properties: + config: + type: string + EOF + $ kubectl apply -f nad.yaml -3. Create NetworkAttachmentDefinition object with the UUID of Neutron subnet -defined in step 1. +#. Create NetworkAttachmentDefinition object with the UUID of Neutron subnet + defined in step 1. -.. code-block:: bash + .. code-block:: console - $ cat << EOF > net-a.yaml - apiVersion: "k8s.cni.cncf.io/v1" - kind: NetworkAttachmentDefinition - metadata: - name: "net-a" - annotations: - openstack.org/kuryr-config: '{ - "subnetId": "uuid-of-neutron-subnet-a" - }' - EOF - $ kubectl apply -f net-a.yaml + $ cat << EOF > net-a.yaml + apiVersion: "k8s.cni.cncf.io/v1" + kind: NetworkAttachmentDefinition + metadata: + name: "net-a" + annotations: + openstack.org/kuryr-config: '{ + "subnetId": "uuid-of-neutron-subnet-a" + }' + EOF + $ kubectl apply -f net-a.yaml -4. Enable the multi-vif driver by setting 'multi_vif_drivers' in kuryr.conf. +#. Enable the multi-vif driver by setting 'multi_vif_drivers' in kuryr.conf. Then restart kuryr-controller. -.. code-block:: ini + .. code-block:: ini - [kubernetes] - multi_vif_drivers = npwg_multiple_interfaces + [kubernetes] + multi_vif_drivers = npwg_multiple_interfaces 5. Add additional interfaces to pods definition. e.g. -.. code-block:: bash + .. code-block:: console - $ cat << EOF > pod.yaml - apiVersion: v1 - kind: Pod - metadata: - name: nginx4 - annotations: - k8s.v1.cni.cncf.io/networks: net-a - spec: - containers: - - name: nginx - image: nginx:1.7.9 - ports: - - containerPort: 80 - EOF - $ kubectl apply -f pod.yaml + $ cat << EOF > pod.yaml + apiVersion: v1 + kind: Pod + metadata: + name: nginx4 + annotations: + k8s.v1.cni.cncf.io/networks: net-a + spec: + containers: + - name: nginx + image: nginx:1.7.9 + ports: + - containerPort: 80 + EOF + $ kubectl apply -f pod.yaml -You may put a list of network separated with comma to attach Pods to more networks. +You may put a list of network separated with comma to attach Pods to more +networks. .. _Kubernetes Network Custom Resource Definition De-facto Standard Version 1: https://docs.google.com/document/d/1Ny03h6IDVy_e_vmElOqR7UdTPAG_RNydhVE1Kx54kFQ/edit?usp=sharing diff --git a/doc/source/installation/network_namespace.rst b/doc/source/installation/network_namespace.rst index cf7e7fd64..4306dd253 100644 --- a/doc/source/installation/network_namespace.rst +++ b/doc/source/installation/network_namespace.rst @@ -5,7 +5,7 @@ Enable network per namespace functionality (handler + driver) To enable the subnet driver that creates a new network for each new namespace the next steps are needed: -1. Enable the namespace handler to reach to namespace events, in this case, +#. Enable the namespace handler to reach to namespace events, in this case, creation and deletion. To do that you need to add it to the list of the enabled handlers at kuryr.conf (details on how to edit this for containerized deployment can be found at :doc:`./devstack/containerized`): @@ -24,7 +24,7 @@ the next steps are needed: [kubernetes] enabled_handlers=vif,lb,lbaasspec,namespace,kuryrnet -2. Enable the namespace subnet driver by modifying the default +#. Enable the namespace subnet driver by modifying the default pod_subnet_driver option at kuryr.conf: .. code-block:: ini @@ -43,7 +43,7 @@ the next steps are needed: pod_security_groups_driver = namespace service_security_groups_driver = namespace -3. Select (and create if needed) the subnet pool from where the new subnets +#. Select (and create if needed) the subnet pool from where the new subnets will get their CIDR (e.g., the default on devstack deployment is shared-default-subnetpool-v4): @@ -52,7 +52,7 @@ the next steps are needed: [namespace_subnet] pod_subnet_pool = SUBNET_POOL_ID -4. Select (and create if needed) the router where the new subnet will be +#. Select (and create if needed) the router where the new subnet will be connected (e.g., the default on devstack deployments is router1): .. code-block:: ini @@ -64,7 +64,7 @@ the next steps are needed: requirements between pod, service and public subnets, as in the case for the default subnet driver. -5. Select (and create if needed) the security groups to be attached to the +#. Select (and create if needed) the security groups to be attached to the pods at the default namespace and to the others, enabling the cross access between them: @@ -108,14 +108,14 @@ to add the namespace handler and state the namespace subnet driver with: Testing the network per namespace functionality ----------------------------------------------- -1. Create two namespaces: +#. Create two namespaces: .. code-block:: console $ kubectl create namespace test1 $ kubectl create namespace test2 -2. Check resources has been created: +#. Check resources has been created: .. code-block:: console @@ -136,7 +136,7 @@ Testing the network per namespace functionality $ openstack subnet list | grep test1 | 8640d134-5ea2-437d-9e2a-89236f6c0198 | ns/test1-subnet | 7c7b68c5-d3c4-431c-9f69-fbc777b43ee5 | 10.0.1.128/26 | -3. Create a pod in the created namespaces: +#. Create a pod in the created namespaces: .. code-block:: console @@ -154,7 +154,7 @@ Testing the network per namespace functionality NAME READY STATUS RESTARTS AGE IP NODE demo-5135352253-dfghd 1/1 Running 0 7s 10.0.1.134 node1 -4. Create a service: +#. Create a service: .. code-block:: console @@ -165,7 +165,7 @@ Testing the network per namespace functionality NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE demo ClusterIP 10.0.0.141 80/TCP 18s -5. Test service connectivity from both namespaces: +#. Test service connectivity from both namespaces: .. code-block:: console @@ -177,7 +177,7 @@ Testing the network per namespace functionality test-2-pod$ curl 10.0.0.141 ## No response -6. And finally, to remove the namespace and all its resources, including +#. And finally, to remove the namespace and all its resources, including openstack networks, kuryrnet CRD, svc, pods, you just need to do: .. code-block:: console diff --git a/doc/source/installation/network_policy.rst b/doc/source/installation/network_policy.rst index c2708be04..bc5943247 100644 --- a/doc/source/installation/network_policy.rst +++ b/doc/source/installation/network_policy.rst @@ -100,7 +100,7 @@ to add the policy, pod_label and namespace handler and drivers with: Testing the network policy support functionality ------------------------------------------------ -1. Given a yaml file with a network policy, such as: +#. Given a yaml file with a network policy, such as: .. code-block:: yaml @@ -133,13 +133,13 @@ Testing the network policy support functionality - protocol: TCP port: 5978 -2. Apply the network policy: +#. Apply the network policy: .. code-block:: console $ kubectl apply -f network_policy.yml -3. Check that the resources has been created: +#. Check that the resources has been created: .. code-block:: console @@ -154,7 +154,7 @@ Testing the network policy support functionality $ openstack security group list | grep sg-test-network-policy | dabdf308-7eed-43ef-a058-af84d1954acb | sg-test-network-policy -4. Check that the rules are in place for the security group: +#. Check that the rules are in place for the security group: .. code-block:: console @@ -230,7 +230,7 @@ Testing the network policy support functionality | tcp | 5978:5978 | egress | +-------------+------------+-----------+ -5. Create a pod: +#. Create a pod: .. code-block:: console @@ -241,7 +241,7 @@ Testing the network policy support functionality NAME READY STATUS RESTARTS AGE IP demo-5558c7865d-fdkdv 1/1 Running 0 44s 10.0.0.68 -6. Get the pod port and check its security group rules: +#. Get the pod port and check its security group rules: .. code-block:: console @@ -260,13 +260,13 @@ Testing the network policy support functionality | tcp | 5978:5978 | egress | +-------------+------------+-----------+ -7. Try to curl the pod on port 8080 (hint: it won't work!): +#. Try to curl the pod on port 8080 (hint: it won't work!): .. code-block:: console $ curl 10.0.0.68:8080 -8. Update network policy to allow ingress 8080 port: +#. Update network policy to allow ingress 8080 port: .. code-block:: console @@ -343,19 +343,18 @@ Testing the network policy support functionality | tcp | 5978:5978 | egress | +-------------+------------+-----------+ -9. Try to curl the pod ip after patching the network policy: +#. Try to curl the pod ip after patching the network policy: .. code-block:: console $ curl 10.0.0.68:8080 demo-5558c7865d-fdkdv: HELLO! I AM ALIVE!!! -Note the curl only works from pods (neutron ports) on a namespace that has -the label `project: default` as stated on the policy namespaceSelector. + Note the curl only works from pods (neutron ports) on a namespace that has + the label `project: default` as stated on the policy namespaceSelector. -10. We can also create a single pod, without a label and check that there is - no connectivity to it, as it does not match the network policy - podSelector: +#. We can also create a single pod, without a label and check that there is no + connectivity to it, as it does not match the network policy podSelector: .. code-block:: console @@ -374,9 +373,9 @@ the label `project: default` as stated on the policy namespaceSelector. $ curl demo-pod-IP:8080 NO REPLY -11. If we add to the pod a label that match a network policy podSelector, in - this case 'project: default', the network policy will get applied on the - pod, and the traffic will be allowed: +#. If we add to the pod a label that match a network policy podSelector, in + this case 'project: default', the network policy will get applied on the + pod, and the traffic will be allowed: .. code-block:: console @@ -384,7 +383,7 @@ the label `project: default` as stated on the policy namespaceSelector. $ curl demo-pod-IP:8080 demo-pod-XXX: HELLO! I AM ALIVE!!! -12. Confirm the teardown of the resources once the network policy is removed: +#. Confirm the teardown of the resources once the network policy is removed: .. code-block:: console diff --git a/doc/source/installation/ocp_route.rst b/doc/source/installation/ocp_route.rst index 4c7c23dc5..63703645d 100644 --- a/doc/source/installation/ocp_route.rst +++ b/doc/source/installation/ocp_route.rst @@ -84,14 +84,14 @@ Router: Configure Kuryr to support L7 Router and OCP-Route resources ------------------------------------------------------------ -1. Configure the L7 Router by adding the LB UUID at kuryr.conf: +#. Configure the L7 Router by adding the LB UUID at kuryr.conf: .. code-block:: ini [ingress] l7_router_uuid = 99f580e6-d894-442a-bc5f-4d14b41e10d2 -2. Enable the ocp-route and k8s-endpoint handlers. For that you need to add +#. Enable the ocp-route and k8s-endpoint handlers. For that you need to add this handlers to the enabled handlers list at kuryr.conf (details on how to edit this for containerized deployment can be found at :doc:`./devstack/containerized`): @@ -127,7 +127,7 @@ with devstack, you just need to add the following at local.conf file: Testing OCP-Route functionality ------------------------------- -1. Create a service: +#. Create a service: .. code-block:: console @@ -135,7 +135,7 @@ Testing OCP-Route functionality $ oc scale dc/kuryr-demo --replicas=2 $ oc expose dc/kuryr-demo --port 80 --target-port 8080 -2. Create a Route object pointing to above service (kuryr-demo): +#. Create a Route object pointing to above service (kuryr-demo): .. code-block:: console @@ -152,7 +152,7 @@ Testing OCP-Route functionality > EOF $ oc create -f route.yaml -3. Curl L7 router's FIP using specified hostname: +#. Curl L7 router's FIP using specified hostname: .. code-block:: console diff --git a/doc/source/installation/ports-pool.rst b/doc/source/installation/ports-pool.rst index d1082e47b..f544e707b 100644 --- a/doc/source/installation/ports-pool.rst +++ b/doc/source/installation/ports-pool.rst @@ -93,13 +93,12 @@ pools just by restarting the kuryr-controller (or even before installing it). To do that you just need to ensure the ports are created with the right device_owner: - - For neutron pod driver: compute:kuryr (of the value at - kuryr.lib.constants.py) - - - For nested-vlan pod driver: trunk:subport or compute:kuryr (or the value - at kuryr.lib.constants.py). But in this case they also need to be - attached to an active neutron trunk port, i.e., they need to be subports - of an existing trunk +- For neutron pod driver: compute:kuryr (of the value at + kuryr.lib.constants.py) +- For nested-vlan pod driver: trunk:subport or compute:kuryr (or the value at + kuryr.lib.constants.py). But in this case they also need to be attached to an + active neutron trunk port, i.e., they need to be subports of an existing + trunk Subports pools management tool diff --git a/doc/source/installation/services.rst b/doc/source/installation/services.rst index 977609a80..897b57367 100644 --- a/doc/source/installation/services.rst +++ b/doc/source/installation/services.rst @@ -425,15 +425,17 @@ The services and pods subnets should be created. #. For the external services (type=LoadBalancer) case, two methods are supported: -* Pool - external IPs are allocated from pre-defined pool -* User - user specify the external IP address + + Pool - external IPs are allocated from pre-defined pool + + User - user specify the external IP address - In case 'Pool' method should be supported, execute the next steps + In case 'Pool' method should be supported, execute the next steps: - A. Create an external/provider network - B. Create subnet/pool range of external CIDR - C. Connect external subnet to kuryr-kubernetes router - D. Configure external network details in Kuryr.conf as follows: + #. Create an external/provider network + #. Create subnet/pool range of external CIDR + #. Connect external subnet to kuryr-kubernetes router + #. Configure external network details in Kuryr.conf as follows: + + .. code-block:: ini [neutron_defaults] external_svc_net= diff --git a/doc/source/installation/sriov.rst b/doc/source/installation/sriov.rst index fb8866a3f..53c4dfd08 100644 --- a/doc/source/installation/sriov.rst +++ b/doc/source/installation/sriov.rst @@ -8,166 +8,167 @@ Current approach of SR-IOV relies on `sriov-device-plugin`_. While creating pods with SR-IOV, sriov-device-plugin should be turned on on all nodes. To use a SR-IOV port on a baremetal installation the 3 following steps should be done: -1. Create OpenStack network and subnet for SR-IOV. -Following steps should be done with admin rights. +#. Create OpenStack network and subnet for SR-IOV. Following steps should be + done with admin rights. -.. code-block:: console + .. code-block:: console - neutron net-create vlan-sriov-net --shared --provider:physical_network physnet10_4 --provider:network_type vlan --provider:segmentation_id 3501 - neutron subnet-create vlan-sriov-net 203.0.114.0/24 --name vlan-sriov-subnet --gateway 203.0.114.1 + $ neutron net-create vlan-sriov-net --shared --provider:physical_network physnet10_4 --provider:network_type vlan --provider:segmentation_id 3501 + $ neutron subnet-create vlan-sriov-net 203.0.114.0/24 --name vlan-sriov-subnet --gateway 203.0.114.1 -Subnet id will be used later in NetworkAttachmentDefinition. + Subnet id will be used later in NetworkAttachmentDefinition. -2. Add sriov section into kuryr.conf. +#. Add sriov section into kuryr.conf. -.. code-block:: ini + .. code-block:: ini - [sriov] - physical_device_mappings = physnet1:ens4f0 - default_physnet_subnets = physnet1: + [sriov] + physical_device_mappings = physnet1:ens4f0 + default_physnet_subnets = physnet1: -This mapping is required for ability to find appropriate PF/VF functions at -binding phase. physnet1 is just an identifier for subnet . Such kind of transition is necessary to support many-to-many -relation. + This mapping is required for ability to find appropriate PF/VF functions at + binding phase. physnet1 is just an identifier for subnet . Such kind of transition is necessary to support + many-to-many relation. -3. Prepare NetworkAttachmentDefinition object. -Apply NetworkAttachmentDefinition with "sriov" driverType inside, -as described in `NPWG spec`_. +#. Prepare NetworkAttachmentDefinition object. Apply + NetworkAttachmentDefinition with "sriov" driverType inside, as described in + `NPWG spec`_. -.. code-block:: yaml + .. code-block:: yaml - apiVersion: "k8s.cni.cncf.io/v1" - kind: NetworkAttachmentDefinition - metadata: - name: "sriov-net1" - annotations: - openstack.org/kuryr-config: '{ - "subnetId": "UUID of vlan-sriov-net", - "driverType": "sriov" - }' + apiVersion: "k8s.cni.cncf.io/v1" + kind: NetworkAttachmentDefinition + metadata: + name: "sriov-net1" + annotations: + openstack.org/kuryr-config: '{ + "subnetId": "UUID of vlan-sriov-net", + "driverType": "sriov" + }' -Then add k8s.v1.cni.cncf.io/networks and request/limits for SR-IOV -into the pod's yaml. -.. code-block:: yaml + Then add k8s.v1.cni.cncf.io/networks and request/limits for SR-IOV into the + pod's yaml. - kind: Pod - metadata: - name: my-pod - namespace: my-namespace - annotations: - k8s.v1.cni.cncf.io/networks: sriov-net1,sriov-net2 - spec: - containers: - - name: containerName - image: containerImage - imagePullPolicy: IfNotPresent - command: ["tail", "-f", "/dev/null"] - resources: - requests: - intel.com/sriov: '2' - limits: - intel.com/sriov: '2' + .. code-block:: yaml -In the above example two SR-IOV devices will be attached to pod. First one is -described in sriov-net1 NetworkAttachmentDefinition, second one in sriov-net2. -They may have different subnetId. + kind: Pod + metadata: + name: my-pod + namespace: my-namespace + annotations: + k8s.v1.cni.cncf.io/networks: sriov-net1,sriov-net2 + spec: + containers: + - name: containerName + image: containerImage + imagePullPolicy: IfNotPresent + command: ["tail", "-f", "/dev/null"] + resources: + requests: + intel.com/sriov: '2' + limits: + intel.com/sriov: '2' -4. Specify resource names + In the above example two SR-IOV devices will be attached to pod. First one + is described in sriov-net1 NetworkAttachmentDefinition, second one in + sriov-net2. They may have different subnetId. -The resource name *intel.com/sriov*, which used in the above example is the -default resource name. This name was used in SR-IOV network device plugin in -version 1 (release-v1 branch). But since latest version the device plugin can -use any arbitrary name of the resources (see `SRIOV network device plugin for -Kubernetes`_). This name should match "^\[a-zA-Z0-9\_\]+$" regular expression. -To be able to work with arbitrary resource names physnet_resource_mappings and -device_plugin_resource_prefix in [sriov] section of kuryr-controller -configuration file should be filled. The default value for -device_plugin_resource_prefix is intel.com, the same as in SR-IOV network -device plugin, in case of SR-IOV network device plugin was started with value -of -resource-prefix option different from intel.com, than value should be set -to device_plugin_resource_prefix, otherwise kuryr-kubernetes will not work with -resource. +#. Specify resource names -Assume we have following SR-IOV network device plugin (defined by -config-file -option) + The resource name *intel.com/sriov*, which used in the above example is the + default resource name. This name was used in SR-IOV network device plugin in + version 1 (release-v1 branch). But since latest version the device plugin + can use any arbitrary name of the resources (see `SRIOV network device + plugin for Kubernetes`_). This name should match "^\[a-zA-Z0-9\_\]+$" + regular expression. To be able to work with arbitrary resource names + physnet_resource_mappings and device_plugin_resource_prefix in [sriov] + section of kuryr-controller configuration file should be filled. The + default value for device_plugin_resource_prefix is intel.com, the same as in + SR-IOV network device plugin, in case of SR-IOV network device plugin was + started with value of -resource-prefix option different from intel.com, than + value should be set to device_plugin_resource_prefix, otherwise + kuryr-kubernetes will not work with resource. -.. code-block:: json + Assume we have following SR-IOV network device plugin (defined by + -config-file option) - { - "resourceList": - [ - { - "resourceName": "numa0", - "rootDevices": ["0000:02:00.0"], - "sriovMode": true, - "deviceType": "netdevice" - } - ] - } + .. code-block:: json -We defined numa0 resource name, also assume we started sriovdp with --resource-prefix samsung.com value. The PCI address of ens4f0 interface is -"0000:02:00.0". If we assigned 8 VF to ens4f0 and launch SR-IOV network device -plugin, we can see following state of kubernetes + { + "resourceList": + [ + { + "resourceName": "numa0", + "rootDevices": ["0000:02:00.0"], + "sriovMode": true, + "deviceType": "netdevice" + } + ] + } -.. code-block:: console + We defined numa0 resource name, also assume we started sriovdp with + -resource-prefix samsung.com value. The PCI address of ens4f0 interface is + "0000:02:00.0". If we assigned 8 VF to ens4f0 and launch SR-IOV network + device plugin, we can see following state of kubernetes - $ kubectl get node node1 -o json | jq '.status.allocatable' - { - "cpu": "4", - "ephemeral-storage": "269986638772", - "hugepages-1Gi": "8Gi", - "hugepages-2Mi": "0Gi", - "samsung.com/numa0": "8", - "memory": "7880620Ki", - "pods": "1k" - } + .. code-block:: console -We have to add to the sriov section following mapping: + $ kubectl get node node1 -o json | jq '.status.allocatable' + { + "cpu": "4", + "ephemeral-storage": "269986638772", + "hugepages-1Gi": "8Gi", + "hugepages-2Mi": "0Gi", + "samsung.com/numa0": "8", + "memory": "7880620Ki", + "pods": "1k" + } -.. code-block:: ini + We have to add to the sriov section following mapping: - [sriov] - device_plugin_resource_prefix = samsung.com - physnet_resource_mappings = physnet1:numa0 + .. code-block:: ini -5. Enable Kubelet Pod Resources feature + [sriov] + device_plugin_resource_prefix = samsung.com + physnet_resource_mappings = physnet1:numa0 -To use SR-IOV functionality properly it is necessary to enable Kubelet Pod -Resources feature. Pod Resources is a service provided by Kubelet via gRPC -server that allows to request list of resources allocated for each pod and -container on the node. These resources are devices allocated by k8s device -plugins. Service was implemented mainly for monitoring purposes, but it also -suitable for SR-IOV binding driver allowing it to know which VF was allocated -for particular container. +#. Enable Kubelet Pod Resources feature -To enable Pod Resources service it is needed to add -``--feature-gates KubeletPodResources=true`` into ``/etc/sysconfig/kubelet``. -This file could look like: + To use SR-IOV functionality properly it is necessary to enable Kubelet Pod + Resources feature. Pod Resources is a service provided by Kubelet via gRPC + server that allows to request list of resources allocated for each pod and + container on the node. These resources are devices allocated by k8s device + plugins. Service was implemented mainly for monitoring purposes, but it also + suitable for SR-IOV binding driver allowing it to know which VF was + allocated for particular container. -.. code-block:: bash + To enable Pod Resources service it is needed to add ``--feature-gates + KubeletPodResources=true`` into ``/etc/sysconfig/kubelet``. This file could + look like: - KUBELET_EXTRA_ARGS="--feature-gates KubeletPodResources=true" + .. code-block:: bash -Note that it is important to set right value for parameter ``kubelet_root_dir`` -in ``kuryr.conf``. By default it is ``/var/lib/kubelet``. -In case of using containerized CNI it is necessary to mount -``'kubelet_root_dir'/pod-resources`` directory into CNI container. + KUBELET_EXTRA_ARGS="--feature-gates KubeletPodResources=true" -To use this feature add ``enable_pod_resource_service`` into kuryr.conf. + Note that it is important to set right value for parameter + ``kubelet_root_dir`` in ``kuryr.conf``. By default it is + ``/var/lib/kubelet``. In case of using containerized CNI it is necessary to + mount ``'kubelet_root_dir'/pod-resources`` directory into CNI container. -.. code-block:: ini + To use this feature add ``enable_pod_resource_service`` into kuryr.conf. - [sriov] - enable_pod_resource_service = True + .. code-block:: ini -6. Use privileged user + [sriov] + enable_pod_resource_service = True -To make neutron ports active kuryr-k8s makes requests to neutron API to update -ports with binding:profile information. Due to this it is necessary to make -actions with privileged user with admin rights. +#. Use privileged user + + To make neutron ports active kuryr-k8s makes requests to neutron API to + update ports with binding:profile information. Due to this it is necessary + to make actions with privileged user with admin rights. .. _NPWG spec: https://docs.openstack.org/kuryr-kubernetes/latest/specs/rocky/npwg_spec_support.html diff --git a/doc/source/installation/testing_sriov_functional.rst b/doc/source/installation/testing_sriov_functional.rst index dda96b369..25405b1a3 100644 --- a/doc/source/installation/testing_sriov_functional.rst +++ b/doc/source/installation/testing_sriov_functional.rst @@ -52,195 +52,196 @@ that is expected to be used for SR-IOV ports: | updated_at | 2018-11-21T10:57:34Z | +-------------------+--------------------------------------------------+ -1. Create deployment definition with one SR-IOV +#. Create deployment definition with one SR-IOV interface (apart from default one). Deployment definition file might look like: -.. code-block:: yaml + .. code-block:: yaml - apiVersion: extensions/v1beta1 - kind: Deployment - metadata: - name: nginx-sriov - spec: - replicas: 1 - template: - metadata: - name: nginx-sriov - labels: - app: nginx-sriov - annotations: - k8s.v1.cni.cncf.io/networks: net-sriov - spec: - containers: - - name: nginx-sriov - image: nginx - resources: - requests: - intel.com/sriov: '1' - cpu: "1" - memory: "512Mi" - limits: - intel.com/sriov: '1' - cpu: "1" - memory: "512Mi" + apiVersion: extensions/v1beta1 + kind: Deployment + metadata: + name: nginx-sriov + spec: + replicas: 1 + template: + metadata: + name: nginx-sriov + labels: + app: nginx-sriov + annotations: + k8s.v1.cni.cncf.io/networks: net-sriov + spec: + containers: + 1. name: nginx-sriov + image: nginx + resources: + requests: + intel.com/sriov: '1' + cpu: "1" + memory: "512Mi" + limits: + intel.com/sriov: '1' + cpu: "1" + memory: "512Mi" -Here ``net-sriov`` is the name of ``NetworkAttachmentDefinition`` -created before. + Here ``net-sriov`` is the name of ``NetworkAttachmentDefinition`` created + before. -2. Create deployment with the following command: +#. Create deployment with the following command: -.. code-block:: console + .. code-block:: console - $ kubectl create -f + $ kubectl create -f -3. Wait for the pod to get to Running phase. +#. Wait for the pod to get to Running phase. -.. code-block:: console + .. code-block:: console - $ kubectl get pods - NAME READY STATUS RESTARTS AGE - nginx-sriov-558db554d7-rvpxs 1/1 Running 0 1m + $ kubectl get pods + NAME READY STATUS RESTARTS AGE + nginx-sriov-558db554d7-rvpxs 1/1 Running 0 1m -4. If your image contains ``iputils`` (for example, busybox image), you can +#. If your image contains ``iputils`` (for example, busybox image), you can attach to the pod and check that the correct interface has been attached to the Pod. -.. code-block:: console + .. code-block:: console - $ kubectl get pod - $ kubectl exec -it nginx-sriov-558db554d7-rvpxs -- /bin/bash - $ ip a + $ kubectl get pod + $ kubectl exec -it nginx-sriov-558db554d7-rvpxs -- /bin/bash + $ ip a -You should see default and eth1 interfaces. eth1 is the SR-IOV VF interface. + You should see default and eth1 interfaces. eth1 is the SR-IOV VF interface. -.. code-block:: console + .. code-block:: - 1: lo: mtu 65536 qdisc noqueue state UNKNOWN qlen 1000 - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 - inet 127.0.0.1/8 scope host lo - valid_lft forever preferred_lft forever - inet6 ::1/128 scope host - valid_lft forever preferred_lft forever - 3: eth0@if43: mtu 1500 qdisc noqueue state UP qlen 1000 - link/ether fa:16:3e:1a:c0:43 brd ff:ff:ff:ff:ff:ff link-netnsid 0 - inet 192.168.0.9/24 scope global eth0 - valid_lft forever preferred_lft forever - inet6 fe80::f816:3eff:fe1a:c043/64 scope link - valid_lft forever preferred_lft forever - 13: eth1: mtu 1500 qdisc pfifo_fast state UP qlen 1000 - link/ether fa:16:3e:b3:2e:70 brd ff:ff:ff:ff:ff:ff - inet 192.168.2.6/24 scope global eth1 - valid_lft forever preferred_lft forever - inet6 fe80::f816:3eff:fea8:55af/64 scope link - valid_lft forever preferred_lft forever + 1: lo: mtu 65536 qdisc noqueue state UNKNOWN qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + inet 127.0.0.1/8 scope host lo + valid_lft forever preferred_lft forever + inet6 ::1/128 scope host + valid_lft forever preferred_lft forever + 3: eth0@if43: mtu 1500 qdisc noqueue state UP qlen 1000 + link/ether fa:16:3e:1a:c0:43 brd ff:ff:ff:ff:ff:ff link-netnsid 0 + inet 192.168.0.9/24 scope global eth0 + valid_lft forever preferred_lft forever + inet6 fe80::f816:3eff:fe1a:c043/64 scope link + valid_lft forever preferred_lft forever + 13: eth1: mtu 1500 qdisc pfifo_fast state UP qlen 1000 + link/ether fa:16:3e:b3:2e:70 brd ff:ff:ff:ff:ff:ff + inet 192.168.2.6/24 scope global eth1 + valid_lft forever preferred_lft forever + inet6 fe80::f816:3eff:fea8:55af/64 scope link + valid_lft forever preferred_lft forever -4.1. Alternatively you can login to k8s worker and do the same from the host -system. Use the following command to find out ID of running SR-IOV container: + Alternatively you can login to k8s worker and do the same from the host + system. Use the following command to find out ID of running SR-IOV + container: -.. code-block:: console + .. code-block:: console - $ docker ps + $ docker ps -Suppose that ID of created container is ``eb4e10f38763``. Use the following -command to get PID of that container: + Suppose that ID of created container is ``eb4e10f38763``. + Use the following command to get PID of that container: -.. code-block:: console + .. code-block:: console - $ docker inspect --format {{.State.Pid}} eb4e10f38763 + $ docker inspect --format {{.State.Pid}} eb4e10f38763 -Suppose that output of previous command is bellow: + Suppose that output of previous command is bellow: -.. code-block:: console + .. code-block:: console - $ 32609 + $ 32609 -Use the following command to get interfaces of container: + Use the following command to get interfaces of container: -.. code-block:: console + .. code-block:: console - $ nsenter -n -t 32609 ip a + $ nsenter -n -t 32609 ip a -You should see default and eth1 interfaces. eth1 is the SR-IOV VF interface. + You should see default and eth1 interfaces. eth1 is the SR-IOV VF interface. -.. code-block:: console + .. code-block:: console - 1: lo: mtu 65536 qdisc noqueue state UNKNOWN qlen 1000 - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 - inet 127.0.0.1/8 scope host lo - valid_lft forever preferred_lft forever - inet6 ::1/128 scope host - valid_lft forever preferred_lft forever - 3: eth0@if43: mtu 1500 qdisc noqueue state UP qlen 1000 - link/ether fa:16:3e:1a:c0:43 brd ff:ff:ff:ff:ff:ff link-netnsid 0 - inet 192.168.0.9/24 scope global eth0 - valid_lft forever preferred_lft forever - inet6 fe80::f816:3eff:fe1a:c043/64 scope link - valid_lft forever preferred_lft forever - 13: eth1: mtu 1500 qdisc pfifo_fast state UP qlen 1000 - link/ether fa:16:3e:b3:2e:70 brd ff:ff:ff:ff:ff:ff - inet 192.168.2.6/24 scope global eth1 - valid_lft forever preferred_lft forever - inet6 fe80::f816:3eff:fea8:55af/64 scope link - valid_lft forever preferred_lft forever + 1: lo: mtu 65536 qdisc noqueue state UNKNOWN qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + inet 127.0.0.1/8 scope host lo + valid_lft forever preferred_lft forever + inet6 ::1/128 scope host + valid_lft forever preferred_lft forever + 3: eth0@if43: mtu 1500 qdisc noqueue state UP qlen 1000 + link/ether fa:16:3e:1a:c0:43 brd ff:ff:ff:ff:ff:ff link-netnsid 0 + inet 192.168.0.9/24 scope global eth0 + valid_lft forever preferred_lft forever + inet6 fe80::f816:3eff:fe1a:c043/64 scope link + valid_lft forever preferred_lft forever + 13: eth1: mtu 1500 qdisc pfifo_fast state UP qlen 1000 + link/ether fa:16:3e:b3:2e:70 brd ff:ff:ff:ff:ff:ff + inet 192.168.2.6/24 scope global eth1 + valid_lft forever preferred_lft forever + inet6 fe80::f816:3eff:fea8:55af/64 scope link + valid_lft forever preferred_lft forever -In our example sriov interface has address 192.168.2.6 + In our example sriov interface has address 192.168.2.6 -5. Use neutron CLI to check the port with exact address has been created on +#. Use neutron CLI to check the port with exact address has been created on neutron: -.. code-block:: console + .. code-block:: console - $ openstack port list | grep 192.168.2.6 + $ openstack port list | grep 192.168.2.6 -Suppose that previous command returns a list with one openstack port that -has ID ``545ec21d-6bfc-4179-88c6-9dacaf435ea7``. You can see its information -with the following command: + Suppose that previous command returns a list with one openstack port that + has ID ``545ec21d-6bfc-4179-88c6-9dacaf435ea7``. You can see its information + with the following command: -.. code-block:: console + .. code-block:: console - $ openstack port show 545ec21d-6bfc-4179-88c6-9dacaf435ea7 - +-----------------------+----------------------------------------------------------------------------+ - | Field | Value | - +-----------------------+----------------------------------------------------------------------------+ - | admin_state_up | UP | - | allowed_address_pairs | | - | binding_host_id | novactl | - | binding_profile | | - | binding_vif_details | port_filter='True' | - | binding_vif_type | hw_veb | - | binding_vnic_type | direct | - | created_at | 2018-11-26T09:13:07Z | - | description | | - | device_id | 7ab02cf9-f15b-11e8-bdf4-525400152cf3 | - | device_owner | compute:kuryr:sriov | - | dns_assignment | None | - | dns_name | None | - | extra_dhcp_opts | | - | fixed_ips | ip_address='192.168.2.6', subnet_id='88d0b025-2710-4f02-a348-2829853b45da' | - | id | 545ec21d-6bfc-4179-88c6-9dacaf435ea7 | - | ip_address | None | - | mac_address | fa:16:3e:b3:2e:70 | - | name | default/nginx-sriov-558db554d7-rvpxs | - | network_id | 2f8b9103-e9ec-47fa-9617-0fb9deacfc00 | - | option_name | None | - | option_value | None | - | port_security_enabled | False | - | project_id | 92a4d7734b17486ba24e635bc7fad595 | - | qos_policy_id | None | - | revision_number | 5 | - | security_groups | 1e7bb965-2ad5-4a09-a5ac-41aa466af25b | - | status | DOWN | - | subnet_id | None | - | updated_at | 2018-11-26T09:13:07Z | - +-----------------------+----------------------------------------------------------------------------+ + $ openstack port show 545ec21d-6bfc-4179-88c6-9dacaf435ea7 + +-----------------------+----------------------------------------------------------------------------+ + | Field | Value | + +-----------------------+----------------------------------------------------------------------------+ + | admin_state_up | UP | + | allowed_address_pairs | | + | binding_host_id | novactl | + | binding_profile | | + | binding_vif_details | port_filter='True' | + | binding_vif_type | hw_veb | + | binding_vnic_type | direct | + | created_at | 2018-11-26T09:13:07Z | + | description | | + | device_id | 7ab02cf9-f15b-11e8-bdf4-525400152cf3 | + | device_owner | compute:kuryr:sriov | + | dns_assignment | None | + | dns_name | None | + | extra_dhcp_opts | | + | fixed_ips | ip_address='192.168.2.6', subnet_id='88d0b025-2710-4f02-a348-2829853b45da' | + | id | 545ec21d-6bfc-4179-88c6-9dacaf435ea7 | + | ip_address | None | + | mac_address | fa:16:3e:b3:2e:70 | + | name | default/nginx-sriov-558db554d7-rvpxs | + | network_id | 2f8b9103-e9ec-47fa-9617-0fb9deacfc00 | + | option_name | None | + | option_value | None | + | port_security_enabled | False | + | project_id | 92a4d7734b17486ba24e635bc7fad595 | + | qos_policy_id | None | + | revision_number | 5 | + | security_groups | 1e7bb965-2ad5-4a09-a5ac-41aa466af25b | + | status | DOWN | + | subnet_id | None | + | updated_at | 2018-11-26T09:13:07Z | + +-----------------------+----------------------------------------------------------------------------+ -The port would have the name of the pod, ``compute::kuryr::sriov`` for device -owner and 'direct' vnic_type. Verify that IP and MAC addresses of the port -match the ones on the container. Currently the neutron-sriov-nic-agent does -not properly detect SR-IOV ports assigned to containers. This means that direct -ports in neutron would always remain in *DOWN* state. This doesn't affect the -feature in any way other than cosmetically. + The port would have the name of the pod, ``compute::kuryr::sriov`` for + device owner and 'direct' vnic_type. Verify that IP and MAC addresses of the + port match the ones on the container. Currently the neutron-sriov-nic-agent + does not properly detect SR-IOV ports assigned to containers. This means + that direct ports in neutron would always remain in *DOWN* state. This + doesn't affect the feature in any way other than cosmetically. .. _sriov-device-plugin: https://docs.google.com/document/d/1Ewe9Of84GkP0b2Q2PC0y9RVZNkN2WeVEagX9m99Nrzc diff --git a/doc/source/installation/trunk_ports.rst b/doc/source/installation/trunk_ports.rst index dabab24b0..f69803991 100644 --- a/doc/source/installation/trunk_ports.rst +++ b/doc/source/installation/trunk_ports.rst @@ -5,7 +5,7 @@ Boot VM with a Trunk Port To create a VM that makes use of the Neutron Trunk port support, the next steps can be followed: -1. Use the demo tenant and create a key to be used to log in into the overcloud +#. Use the demo tenant and create a key to be used to log in into the overcloud VM: .. code-block:: console @@ -14,14 +14,14 @@ steps can be followed: $ openstack keypair create demo > id_rsa_demo $ chmod 600 id_rsa_demo -2. Ensure the demo default security group allows ping and ssh access: +#. Ensure the demo default security group allows ping and ssh access: .. code-block:: console $ openstack security group rule create --protocol icmp default $ openstack security group rule create --protocol tcp --dst-port 22 default -3. Download and import an image that allows vlans, as cirros does not support +#. Download and import an image that allows vlans, as cirros does not support it: .. code-block:: console @@ -29,7 +29,7 @@ steps can be followed: $ wget http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 $ openstack image create --container-format bare --disk-format qcow2 --file CentOS-7-x86_64-GenericCloud.qcow2 centos7 -4. Create a port for the overcloud VM and create the trunk with that port as +#. Create a port for the overcloud VM and create the trunk with that port as the parent port (untagged traffic): .. code-block:: console @@ -37,7 +37,7 @@ steps can be followed: $ openstack port create --network private --security-group default port0 $ openstack network trunk create --parent-port port0 trunk0 -5. Create the overcloud VM and assign a floating ip to it to be able to log in +#. Create the overcloud VM and assign a floating ip to it to be able to log in into it: .. code-block:: console @@ -45,9 +45,9 @@ steps can be followed: $ openstack server create --image centos7 --flavor ds4G --nic port-id=port0 --key-name demo overcloud_vm $ openstack floating ip create --port port0 public -Note subports can be added to the trunk port, and be used inside the VM with -the specific vlan, 102 in the example, by doing: + Note subports can be added to the trunk port, and be used inside the VM with + the specific vlan, 102 in the example, by doing: -.. code-block:: console + .. code-block:: console - $ openstack network trunk set --subport port=subport0,segmentation-type=vlan,segmentation-id=102 trunk0 + $ openstack network trunk set --subport port=subport0,segmentation-type=vlan,segmentation-id=102 trunk0