Documentation for nested-dpdk case
This commit contains spec file for nested-dpdk case in kuryr-kubernetes. It describes how to setup and use it. Change-Id: I4cb83d49b56b16382b00edacf8cfbdd814f0d504 Implements: blueprint nested-dpdk-support Signed-off-by: Danil Golov <d.golov@samsung.com>
This commit is contained in:
parent
6441a6e3dd
commit
20f6a3cbb6
|
@ -35,6 +35,7 @@ ML2 drivers.
|
||||||
basic
|
basic
|
||||||
nested-vlan
|
nested-vlan
|
||||||
nested-macvlan
|
nested-macvlan
|
||||||
|
nested-dpdk
|
||||||
odl_support
|
odl_support
|
||||||
ovn_support
|
ovn_support
|
||||||
dragonflow_support
|
dragonflow_support
|
||||||
|
|
|
@ -0,0 +1,222 @@
|
||||||
|
=========================================
|
||||||
|
How to try out nested-pods locally (DPDK)
|
||||||
|
=========================================
|
||||||
|
|
||||||
|
Following are the instructions for an all-in-one setup, using the nested DPDK
|
||||||
|
driver. We assume that we already have the 'undercloud' configured with at
|
||||||
|
least one VM as nova instance which is also a kubernetes minion. We assume
|
||||||
|
that VM has an access to the Internet to install necessary packages.
|
||||||
|
|
||||||
|
Configure the VM:
|
||||||
|
|
||||||
|
#. Install kernel version supporting uio_pci_generic module:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
sudo apt install linux-image-`uname -r` linux-headers-`uname -r`
|
||||||
|
sudo update-grub
|
||||||
|
sudo reboot
|
||||||
|
|
||||||
|
#. Install DPDK. On Ubuntu:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
sudo apt update
|
||||||
|
sudo apt install dpdk
|
||||||
|
|
||||||
|
#. Enable hugepages:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
sudo sysctl -w vm.nr_hugepages=768
|
||||||
|
|
||||||
|
#. Load DPDK userspace driver:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
sudo modprobe uio_pci_generic
|
||||||
|
|
||||||
|
#. Clone devstack repository:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
cd ~
|
||||||
|
git clone https://git.openstack.org/openstack-dev/devstack
|
||||||
|
|
||||||
|
#. Edit local.conf:
|
||||||
|
|
||||||
|
.. code-block:: ini
|
||||||
|
|
||||||
|
[[local|localrc]]
|
||||||
|
|
||||||
|
RECLONE="no"
|
||||||
|
|
||||||
|
enable_plugin kuryr-kubernetes \
|
||||||
|
https://git.openstack.org/openstack/kuryr-kubernetes
|
||||||
|
|
||||||
|
OFFLINE="no"
|
||||||
|
LOGFILE=devstack.log
|
||||||
|
LOG_COLOR=False
|
||||||
|
ADMIN_PASSWORD=<undercloud_password>
|
||||||
|
DATABASE_PASSWORD=<undercloud_password>
|
||||||
|
RABBIT_PASSWORD=<undercloud_password>
|
||||||
|
SERVICE_PASSWORD=<undercloud_password>
|
||||||
|
SERVICE_TOKEN=<undercloud_password>
|
||||||
|
IDENTITY_API_VERSION=3
|
||||||
|
ENABLED_SERVICES=""
|
||||||
|
|
||||||
|
HOST_IP=<vm-ip-address>
|
||||||
|
|
||||||
|
SERVICE_HOST=<undercloud-host-ip-address>
|
||||||
|
MULTI_HOST=1
|
||||||
|
KEYSTONE_SERVICE_HOST=$SERVICE_HOST
|
||||||
|
MYSQL_HOST=$SERVICE_HOST
|
||||||
|
RABBIT_HOST=$SERVICE_HOST
|
||||||
|
|
||||||
|
KURYR_CONFIGURE_NEUTRON_DEFAULTS=False
|
||||||
|
KURYR_CONFIGURE_BAREMETAL_KUBELET_IFACE=False
|
||||||
|
|
||||||
|
enable_service docker
|
||||||
|
enable_service etcd3
|
||||||
|
enable_service kubernetes-api
|
||||||
|
enable_service kubernetes-controller-manager
|
||||||
|
enable_service kubernetes-scheduler
|
||||||
|
enable_service kubelet
|
||||||
|
enable_service kuryr-kubernetes
|
||||||
|
enable_service kuryr-daemon
|
||||||
|
|
||||||
|
[[post-config|$KURYR_CONF]]
|
||||||
|
[nested_dpdk]
|
||||||
|
dpdk_driver = uio_pci_generic
|
||||||
|
|
||||||
|
#. Stack:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
cd ~/devstack
|
||||||
|
./stack.sh
|
||||||
|
|
||||||
|
#. Install CNI plugins:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
wget https://github.com/containernetworking/plugins/releases/download/v0.6.0/cni-plugins-amd64-v0.6.0.tgz
|
||||||
|
tar xf cni-plugins-amd64-v0.6.0.tgz -C ~/cni/bin/
|
||||||
|
|
||||||
|
#. Install Multus CNI using this guide: https://github.com/intel/multus-cni#build
|
||||||
|
|
||||||
|
- *Note: Kuryr natively supports multiple VIFs now. In step 13 solution*
|
||||||
|
*without Multus is described*
|
||||||
|
|
||||||
|
#. Create Multus CNI configuration file ~/cni/conf/multus-cni.conf:
|
||||||
|
|
||||||
|
.. code-block:: json
|
||||||
|
|
||||||
|
{
|
||||||
|
"name":"multus-demo-network",
|
||||||
|
"type":"multus",
|
||||||
|
"delegates":[
|
||||||
|
{
|
||||||
|
"type":"kuryr-cni",
|
||||||
|
"kuryr_conf":"/etc/kuryr/kuryr.conf",
|
||||||
|
"debug":true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type":"macvlan",
|
||||||
|
"master":"ens3",
|
||||||
|
"masterplugin":true,
|
||||||
|
"ipam":{
|
||||||
|
"type":"host-local",
|
||||||
|
"subnet":"10.0.0.0/24"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
#. Create a directory to store pci devices used by container:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
mkdir /var/pci_address
|
||||||
|
|
||||||
|
#. If you do not use Multus CNI as a tool to have multiple interfaces in
|
||||||
|
container but use some multi vif driver, then change Kuryr configuration file
|
||||||
|
/etc/kuryr/kuryr.conf:
|
||||||
|
|
||||||
|
.. code-block:: ini
|
||||||
|
|
||||||
|
[kubernetes]
|
||||||
|
pod_vif_driver = nested-vlan
|
||||||
|
multi_vif_drivers = npwg_multiple_interfaces
|
||||||
|
[vif_pool]
|
||||||
|
vif_pool_mapping = nested-vlan:nested,nested-dpdk:noop
|
||||||
|
|
||||||
|
#. Also prepare and apply network attachment definition, for example:
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
apiVersion: "k8s.cni.cncf.io/v1"
|
||||||
|
kind: NetworkAttachmentDefinition
|
||||||
|
metadata:
|
||||||
|
name: "net-nested-dpdk"
|
||||||
|
annotations:
|
||||||
|
openstack.org/kuryr-config: '{
|
||||||
|
"subnetId": "<NEUTRON SUBNET ID>",
|
||||||
|
"driverType": "nested-dpdk"
|
||||||
|
}'
|
||||||
|
|
||||||
|
#. Reload systemd services:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
|
||||||
|
#. Restart systemd services:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
sudo systemctl restart devstack@kubelet.service devstack@kuryr-kubernetes.service devstack@kuryr-daemon.service
|
||||||
|
|
||||||
|
#. Create pod specifying additional interface in annotations:
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
apiVersion: extensions/v1beta1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: nginx-nested-dpdk
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
name: nginx-nested-dpdk
|
||||||
|
labels:
|
||||||
|
app: nginx-nested-dpdk
|
||||||
|
annotations:
|
||||||
|
k8s.v1.cni.cncf.io/networks: net-nested-dpdk
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: nginx-nested-dpdk
|
||||||
|
image: nginx
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "1"
|
||||||
|
memory: "512Mi"
|
||||||
|
limits:
|
||||||
|
cpu: "1"
|
||||||
|
memory: "512Mi"
|
||||||
|
volumeMounts:
|
||||||
|
- name: dev
|
||||||
|
mountPath: /dev
|
||||||
|
- name: pci_address
|
||||||
|
mountPath: /var/pci_address
|
||||||
|
volumes:
|
||||||
|
- name: dev
|
||||||
|
hostPath:
|
||||||
|
path: /dev
|
||||||
|
type: Directory
|
||||||
|
- name: pci_address
|
||||||
|
hostPath:
|
||||||
|
path: /var/pci_address
|
||||||
|
type: Directory
|
||||||
|
|
Loading…
Reference in New Issue