Remove way of running without kuryr-daemon

Deploying without kuryr-daemon is deprecated since Rocky and we
announced that it will be removed in the Rocky release notes. This
commit removes all the code that allows that, updates the documentation,
DevStack plugin and gates definitions.

Implements: blueprint remove-non-daemon

Change-Id: I65598d4a6ecb5c3dfde04dc5fefd7b02fc72a0cb
This commit is contained in:
Michał Dulko 2019-01-16 17:49:52 +01:00
parent 375e61a566
commit 3e3ed9dbb3
19 changed files with 83 additions and 405 deletions

View File

@ -76,6 +76,7 @@
kubernetes-scheduler: true kubernetes-scheduler: true
kubelet: true kubelet: true
kuryr-kubernetes: true kuryr-kubernetes: true
kuryr-daemon: true
zuul_copy_output: zuul_copy_output:
'{{ devstack_log_dir }}/kubernetes': 'logs' '{{ devstack_log_dir }}/kubernetes': 'logs'
irrelevant-files: irrelevant-files:

View File

@ -13,10 +13,10 @@
# limitations under the License. # limitations under the License.
- job: - job:
name: kuryr-kubernetes-tempest-daemon-openshift-octavia-multi-vif name: kuryr-kubernetes-tempest-openshift-octavia-multi-vif
parent: kuryr-kubernetes-tempest-daemon-openshift-octavia parent: kuryr-kubernetes-tempest-openshift-octavia
description: | description: |
Kuryr-Kubernetes tempest job using octavia, CNI daemon, Openshift and NPWG multi-vif driver Kuryr-Kubernetes tempest job using octavia, Openshift and NPWG multi-vif driver
vars: vars:
devstack_localrc: devstack_localrc:
KURYR_MULTI_VIF_DRIVER: npwg_multiple_interfaces KURYR_MULTI_VIF_DRIVER: npwg_multiple_interfaces

View File

@ -13,7 +13,7 @@
# limitations under the License. # limitations under the License.
- job: - job:
name: kuryr-kubernetes-tempest-multinode-daemon-octavia-containerized name: kuryr-kubernetes-tempest-multinode-octavia-containerized
parent: kuryr-kubernetes-tempest-octavia parent: kuryr-kubernetes-tempest-octavia
description: | description: |
Kuryr-Kubernetes tempest multinode job using octavia Kuryr-Kubernetes tempest multinode job using octavia
@ -55,8 +55,8 @@
voting: false voting: false
- job: - job:
name: kuryr-kubernetes-tempest-multinode-daemon-octavia-ha name: kuryr-kubernetes-tempest-multinode-octavia-ha
parent: kuryr-kubernetes-tempest-multinode-daemon-octavia-containerized parent: kuryr-kubernetes-tempest-multinode-octavia-containerized
description: | description: |
Kuryr-Kubernetes tempest multinode job using octavia and running Kuryr-Kubernetes tempest multinode job using octavia and running
containerized in HA containerized in HA

View File

@ -44,7 +44,6 @@
o-cw: true o-cw: true
o-hk: true o-hk: true
o-hm: true o-hm: true
kuryr-daemon: false
- job: - job:
name: kuryr-kubernetes-tempest-octavia-centos-7 name: kuryr-kubernetes-tempest-octavia-centos-7
@ -53,38 +52,29 @@
voting: false voting: false
- job: - job:
name: kuryr-kubernetes-tempest-daemon-octavia name: kuryr-kubernetes-tempest-octavia-py36
parent: kuryr-kubernetes-tempest-octavia parent: kuryr-kubernetes-tempest-octavia
description: | description: |
Kuryr-Kubernetes tempest job using octavia and CNI daemon Tempest with Octavia with DevStack running on Python 3.6
vars:
devstack_services:
kuryr-daemon: true
- job:
name: kuryr-kubernetes-tempest-daemon-octavia-py36
parent: kuryr-kubernetes-tempest-daemon-octavia
description: |
Tempest with Octavia, CNI daemon with DevStack running on Python 3.6
vars: vars:
devstack_localrc: devstack_localrc:
USE_PYTHON3: true USE_PYTHON3: true
- job: - job:
name: kuryr-kubernetes-tempest-daemon-containerized-octavia name: kuryr-kubernetes-tempest-containerized-octavia
parent: kuryr-kubernetes-tempest-daemon-octavia parent: kuryr-kubernetes-tempest-octavia
description: | description: |
Kuryr-Kubernetes tempest job using octavia, kuryr containerized and CNI daemon Kuryr-Kubernetes tempest job using octavia, kuryr containerized
vars: vars:
devstack_localrc: devstack_localrc:
KURYR_K8S_CONTAINERIZED_DEPLOYMENT: true KURYR_K8S_CONTAINERIZED_DEPLOYMENT: true
voting: false voting: false
- job: - job:
name: kuryr-kubernetes-tempest-daemon-containerized-octavia-l2 name: kuryr-kubernetes-tempest-containerized-octavia-l2
parent: kuryr-kubernetes-tempest-daemon-containerized-octavia parent: kuryr-kubernetes-tempest-containerized-octavia
description: | description: |
Kuryr-Kubernetes tempest job using octavia in l2 mode, kuryr containerized and CNI daemon Kuryr-Kubernetes tempest job using octavia in l2 mode, kuryr containerized
vars: vars:
devstack_localrc: devstack_localrc:
KURYR_K8S_OCTAVIA_MEMBER_MODE: L2 KURYR_K8S_OCTAVIA_MEMBER_MODE: L2
@ -97,6 +87,9 @@
vars: vars:
devstack_localrc: devstack_localrc:
DOCKER_CGROUP_DRIVER: "systemd" DOCKER_CGROUP_DRIVER: "systemd"
KURYR_SUBNET_DRIVER: namespace
KURYR_SG_DRIVER: namespace
KURYR_ENABLED_HANDLERS: vif,lb,lbaasspec,namespace
devstack_services: devstack_services:
kubernetes-api: false kubernetes-api: false
kubernetes-controller-manager: false kubernetes-controller-manager: false
@ -106,26 +99,13 @@
openshift-node: true openshift-node: true
openshift-dnsmasq: true openshift-dnsmasq: true
openshift-dns: true openshift-dns: true
- job:
name: kuryr-kubernetes-tempest-daemon-openshift-octavia
parent: kuryr-kubernetes-tempest-openshift-octavia
description: |
Kuryr-Kubernetes tempest job using octavia, CNI daemon and OpenShift
vars:
devstack_services:
kuryr-daemon: true
devstack_localrc:
KURYR_SUBNET_DRIVER: namespace
KURYR_SG_DRIVER: namespace
KURYR_ENABLED_HANDLERS: vif,lb,lbaasspec,namespace
voting: false voting: false
- job: - job:
name: kuryr-kubernetes-tempest-daemon-containerized-octavia-pools-namespace name: kuryr-kubernetes-tempest-containerized-octavia-pools-namespace
description: | description: |
Tempest with Octavia, CNI daemon, containers, port pools and namespace subnet driver Tempest with Octavia, CNI daemon, containers, port pools and namespace subnet driver
parent: kuryr-kubernetes-tempest-daemon-containerized-octavia parent: kuryr-kubernetes-tempest-containerized-octavia
vars: vars:
devstack_localrc: devstack_localrc:
KURYR_SUBNET_DRIVER: namespace KURYR_SUBNET_DRIVER: namespace
@ -136,10 +116,10 @@
KURYR_VIF_POOL_DRIVER: neutron KURYR_VIF_POOL_DRIVER: neutron
- job: - job:
name: kuryr-kubernetes-tempest-daemon-containerized-octavia-network-policy name: kuryr-kubernetes-tempest-containerized-octavia-network-policy
description: | description: |
Tempest with Octavia, CNI daemon, containers and network policy driver Tempest with Octavia, CNI daemon, containers and network policy driver
parent: kuryr-kubernetes-tempest-daemon-containerized-octavia parent: kuryr-kubernetes-tempest-containerized-octavia
vars: vars:
tempest_test_regex: '^(kuryr_tempest_plugin.tests.scenario.test_network_policy.TestNetworkPolicyScenario)' tempest_test_regex: '^(kuryr_tempest_plugin.tests.scenario.test_network_policy.TestNetworkPolicyScenario)'
devstack_localrc: devstack_localrc:
@ -149,11 +129,10 @@
voting: false voting: false
- job: - job:
name: kuryr-kubernetes-tempest-daemon-containerized-octavia-py36 name: kuryr-kubernetes-tempest-containerized-octavia-py36
description: | description: |
Tempest with Octavia, CNI daemon, containers with Kuryr running on Tempest with Octavia and Kuryr running on Python3.6 containers
Python3.6 containers parent: kuryr-kubernetes-tempest-containerized-octavia
parent: kuryr-kubernetes-tempest-daemon-containerized-octavia
vars: vars:
devstack_localrc: devstack_localrc:
KURYR_CONTAINERS_USE_PY3: True KURYR_CONTAINERS_USE_PY3: True
@ -161,17 +140,17 @@
voting: true voting: true
- job: - job:
name: kuryr-kubernetes-tempest-daemon-containerized-openshift-octavia name: kuryr-kubernetes-tempest-containerized-openshift-octavia
description: Tempest with Octavia, CNI daemon enabled, containers and OpenShift description: Tempest with Octavia, containers and OpenShift
parent: kuryr-kubernetes-tempest-daemon-openshift-octavia parent: kuryr-kubernetes-tempest-openshift-octavia
vars: vars:
devstack_localrc: devstack_localrc:
KURYR_K8S_CONTAINERIZED_DEPLOYMENT: true KURYR_K8S_CONTAINERIZED_DEPLOYMENT: true
- job: - job:
name: kuryr-kubernetes-tempest-daemon-containerized-openshift-octavia-serial name: kuryr-kubernetes-tempest-containerized-openshift-octavia-serial
description: Tempest with Octavia running in serial, CNI daemon enabled, containers and OpenShift description: Tempest with Octavia running in serial, containers and OpenShift
parent: kuryr-kubernetes-tempest-daemon-containerized-openshift-octavia parent: kuryr-kubernetes-tempest-containerized-openshift-octavia
vars: vars:
devstack_localrc: devstack_localrc:
KURYR_K8S_SERIAL_TESTS: True KURYR_K8S_SERIAL_TESTS: True
@ -179,8 +158,8 @@
- job: - job:
name: kuryr-kubernetes-tempest-daemon-openshift-octavia-ingress name: kuryr-kubernetes-tempest-openshift-octavia-ingress
parent: kuryr-kubernetes-tempest-daemon-openshift-octavia parent: kuryr-kubernetes-tempest-openshift-octavia
description: | description: |
Kuryr-Kubernetes tempest job using octavia, ingress controller and OpenShift Kuryr-Kubernetes tempest job using octavia, ingress controller and OpenShift
vars: vars:
@ -190,8 +169,8 @@
voting: false voting: false
- job: - job:
name: kuryr-kubernetes-tempest-daemon-containerized-octavia-crio name: kuryr-kubernetes-tempest-containerized-octavia-crio
parent: kuryr-kubernetes-tempest-daemon-containerized-octavia parent: kuryr-kubernetes-tempest-containerized-octavia
nodeset: openstack-single-node-bionic nodeset: openstack-single-node-bionic
vars: vars:
devstack_localrc: devstack_localrc:

View File

@ -17,33 +17,31 @@
check: check:
jobs: jobs:
- kuryr-kubernetes-tempest-octavia - kuryr-kubernetes-tempest-octavia
- kuryr-kubernetes-tempest-daemon-octavia - kuryr-kubernetes-tempest-openshift-octavia
- kuryr-kubernetes-tempest-daemon-openshift-octavia - kuryr-kubernetes-tempest-containerized-octavia
- kuryr-kubernetes-tempest-daemon-containerized-octavia - kuryr-kubernetes-tempest-containerized-ovn
- kuryr-kubernetes-tempest-daemon-containerized-ovn - kuryr-kubernetes-tempest-octavia-py36
- kuryr-kubernetes-tempest-daemon-octavia-py36 - kuryr-kubernetes-tempest-containerized-octavia-py36
- kuryr-kubernetes-tempest-daemon-containerized-octavia-py36 - kuryr-kubernetes-tempest-multinode-octavia-containerized
- kuryr-kubernetes-tempest-multinode-daemon-octavia-containerized
- kuryr-kubernetes-tempest-octavia-centos-7 - kuryr-kubernetes-tempest-octavia-centos-7
gate: gate:
jobs: jobs:
- kuryr-kubernetes-tempest-octavia - kuryr-kubernetes-tempest-octavia
- kuryr-kubernetes-tempest-daemon-octavia - kuryr-kubernetes-tempest-octavia-py36
- kuryr-kubernetes-tempest-daemon-octavia-py36 - kuryr-kubernetes-tempest-containerized-octavia-py36
- kuryr-kubernetes-tempest-daemon-containerized-octavia-py36
experimental: experimental:
jobs: jobs:
- kuryr-kubernetes-tempest-dragonflow - kuryr-kubernetes-tempest-dragonflow
- kuryr-kubernetes-tempest-daemon-containerized-openshift-octavia - kuryr-kubernetes-tempest-containerized-openshift-octavia
- kuryr-kubernetes-tempest-daemon-containerized-octavia-l2 - kuryr-kubernetes-tempest-containerized-octavia-l2
- kuryr-kubernetes-tempest-daemon-containerized-octavia-pools-namespace - kuryr-kubernetes-tempest-containerized-octavia-pools-namespace
- kuryr-kubernetes-tempest-daemon-containerized-octavia-network-policy - kuryr-kubernetes-tempest-containerized-octavia-network-policy
- kuryr-kubernetes-tempest-daemon-containerized-openshift-octavia-serial - kuryr-kubernetes-tempest-containerized-openshift-octavia-serial
- kuryr-kubernetes-tempest-daemon-ovn - kuryr-kubernetes-tempest-ovn
- kuryr-kubernetes-tempest-daemon-openshift-octavia-ingress - kuryr-kubernetes-tempest-openshift-octavia-ingress
- kuryr-kubernetes-tempest-daemon-openshift-octavia-multi-vif - kuryr-kubernetes-tempest-openshift-octavia-multi-vif
- kuryr-kubernetes-tempest-multinode-daemon-octavia-ha - kuryr-kubernetes-tempest-multinode-octavia-ha
- kuryr-kubernetes-tempest-daemon-containerized-octavia-crio - kuryr-kubernetes-tempest-containerized-octavia-crio
- project: - project:
templates: templates:

View File

@ -41,23 +41,13 @@
q-dhcp: false q-dhcp: false
q-meta: false q-meta: false
q-trunk: true q-trunk: true
kuryr-daemon: true
voting: false voting: false
- job: - job:
name: kuryr-kubernetes-tempest-daemon-ovn name: kuryr-kubernetes-tempest-containerized-ovn
parent: kuryr-kubernetes-tempest-ovn parent: kuryr-kubernetes-tempest-ovn
description: | description: |
Kuryr-Kubernetes tempest job using OVN, CNI daemon Kuryr-Kubernetes tempest job using OVN and Containerized
vars:
devstack_services:
kuryr-daemon: true
- job:
name: kuryr-kubernetes-tempest-daemon-containerized-ovn
parent: kuryr-kubernetes-tempest-daemon-ovn
description: |
Kuryr-Kubernetes tempest job using OVN, CNI daemon and Containerized
vars: vars:
devstack_localrc: devstack_localrc:
KURYR_K8S_CONTAINERIZED_DEPLOYMENT: true KURYR_K8S_CONTAINERIZED_DEPLOYMENT: true

View File

@ -61,9 +61,4 @@ EOF
cleanup cleanup
deploy deploy
# Start CNI daemon if required exec kuryr-daemon --config-file /etc/kuryr/kuryr.conf
if [ "$CNI_DAEMON" == "True" ]; then
exec kuryr-daemon --config-file /etc/kuryr/kuryr.conf
else
exec sleep infinity
fi

View File

@ -583,9 +583,8 @@ EOF
function generate_cni_daemon_set() { function generate_cni_daemon_set() {
output_dir=$1 output_dir=$1
cni_health_server_port=$2 cni_health_server_port=$2
cni_daemon=${3:-False} cni_bin_dir=${3:-/opt/cni/bin}
cni_bin_dir=${4:-/opt/cni/bin} cni_conf_dir=${4:-/etc/cni/net.d}
cni_conf_dir=${5:-/etc/cni/net.d}
mkdir -p "$output_dir" mkdir -p "$output_dir"
rm -f ${output_dir}/cni_ds.yml rm -f ${output_dir}/cni_ds.yml
cat >> "${output_dir}/cni_ds.yml" << EOF cat >> "${output_dir}/cni_ds.yml" << EOF
@ -624,8 +623,6 @@ spec:
valueFrom: valueFrom:
fieldRef: fieldRef:
fieldPath: metadata.name fieldPath: metadata.name
- name: CNI_DAEMON
value: "${cni_daemon}"
securityContext: securityContext:
privileged: true privileged: true
volumeMounts: volumeMounts:
@ -645,8 +642,7 @@ EOF
mountPath: /var/run mountPath: /var/run
EOF EOF
fi fi
if [ "$cni_daemon" == "True" ]; then cat >> "${output_dir}/cni_ds.yml" << EOF
cat >> "${output_dir}/cni_ds.yml" << EOF
readinessProbe: readinessProbe:
httpGet: httpGet:
path: /ready path: /ready
@ -659,9 +655,6 @@ EOF
path: /alive path: /alive
port: ${cni_health_server_port} port: ${cni_health_server_port}
initialDelaySeconds: 60 initialDelaySeconds: 60
EOF
fi
cat >> "${output_dir}/cni_ds.yml" << EOF
volumes: volumes:
- name: bin - name: bin
hostPath: hostPath:

View File

@ -99,8 +99,6 @@ function configure_kuryr {
iniset "$KURYR_CONFIG" cni_health_server cg_path \ iniset "$KURYR_CONFIG" cni_health_server cg_path \
"/system.slice/system-devstack.slice/devstack@kuryr-daemon.service" "/system.slice/system-devstack.slice/devstack@kuryr-daemon.service"
fi fi
else
iniset "$KURYR_CONFIG" cni_daemon daemon_enabled False
fi fi
create_kuryr_cache_dir create_kuryr_cache_dir
@ -131,8 +129,6 @@ function configure_kuryr {
} }
function generate_containerized_kuryr_resources { function generate_containerized_kuryr_resources {
local cni_daemon
cni_daemon=$1
if [[ $KURYR_CONTROLLER_REPLICAS -eq 1 ]]; then if [[ $KURYR_CONTROLLER_REPLICAS -eq 1 ]]; then
KURYR_CONTROLLER_HA="False" KURYR_CONTROLLER_HA="False"
else else
@ -892,9 +888,6 @@ function update_tempest_conf_file {
if [[ "$KURYR_ENABLED_HANDLERS" =~ .*policy.* ]]; then if [[ "$KURYR_ENABLED_HANDLERS" =~ .*policy.* ]]; then
iniset $TEMPEST_CONFIG kuryr_kubernetes network_policy_enabled True iniset $TEMPEST_CONFIG kuryr_kubernetes network_policy_enabled True
fi fi
if ! is_service_enabled kuryr-daemon; then
iniset $TEMPEST_CONFIG kuryr_kubernetes kuryr_daemon_enabled False
fi
# NOTE(yboaron): Services with protocol UDP are supported in Kuryr # NOTE(yboaron): Services with protocol UDP are supported in Kuryr
# starting from Stein release # starting from Stein release
iniset $TEMPEST_CONFIG kuryr_kubernetes test_udp_services True iniset $TEMPEST_CONFIG kuryr_kubernetes test_udp_services True
@ -1038,11 +1031,7 @@ if [[ "$1" == "stack" && "$2" == "extra" ]]; then
KURYR_FORCE_IMAGE_BUILD=$(trueorfalse False KURYR_FORCE_IMAGE_BUILD) KURYR_FORCE_IMAGE_BUILD=$(trueorfalse False KURYR_FORCE_IMAGE_BUILD)
if is_service_enabled kuryr-kubernetes || [[ ${KURYR_FORCE_IMAGE_BUILD} == "True" ]]; then if is_service_enabled kuryr-kubernetes || [[ ${KURYR_FORCE_IMAGE_BUILD} == "True" ]]; then
if [ "$KURYR_K8S_CONTAINERIZED_DEPLOYMENT" == "True" ]; then if [ "$KURYR_K8S_CONTAINERIZED_DEPLOYMENT" == "True" ]; then
if is_service_enabled kuryr-daemon; then build_kuryr_containers
build_kuryr_containers True
else
build_kuryr_containers False
fi
fi fi
fi fi
@ -1050,11 +1039,7 @@ if [[ "$1" == "stack" && "$2" == "extra" ]]; then
/usr/local/bin/kubectl apply -f ${KURYR_HOME}/kubernetes_crds/kuryrnet.yaml /usr/local/bin/kubectl apply -f ${KURYR_HOME}/kubernetes_crds/kuryrnet.yaml
/usr/local/bin/kubectl apply -f ${KURYR_HOME}/kubernetes_crds/kuryrnetpolicy.yaml /usr/local/bin/kubectl apply -f ${KURYR_HOME}/kubernetes_crds/kuryrnetpolicy.yaml
if [ "$KURYR_K8S_CONTAINERIZED_DEPLOYMENT" == "True" ]; then if [ "$KURYR_K8S_CONTAINERIZED_DEPLOYMENT" == "True" ]; then
if is_service_enabled kuryr-daemon; then generate_containerized_kuryr_resources
generate_containerized_kuryr_resources True
else
generate_containerized_kuryr_resources False
fi
fi fi
if [ "$KURYR_MULTI_VIF_DRIVER" == "npwg_multiple_interfaces" ]; then if [ "$KURYR_MULTI_VIF_DRIVER" == "npwg_multiple_interfaces" ]; then
/usr/local/bin/kubectl apply -f ${KURYR_HOME}/kubernetes_crds/network_attachment_definition_crd.yaml /usr/local/bin/kubectl apply -f ${KURYR_HOME}/kubernetes_crds/network_attachment_definition_crd.yaml

View File

@ -194,19 +194,27 @@ CNI driver to complete pod handling.
The NeutronPodVifDriver is the default driver that creates neutron port upon The NeutronPodVifDriver is the default driver that creates neutron port upon
Pod addition and deletes port upon Pod removal. Pod addition and deletes port upon Pod removal.
CNI Driver
----------
CNI driver is just a thin client that passes CNI ADD and DEL requests to
kuryr-daemon instance via its HTTP API. It's simple Python executable that is
supposed to be called by kublet's CNI.
.. _cni-daemon: .. _cni-daemon:
CNI Daemon CNI Daemon
---------- ----------
CNI Daemon is a service that should run on every Kubernetes node. Starting from CNI Daemon is a service that should run on every Kubernetes node. Starting from
Rocky release it should be seen as a default supported deployment option. Rocky release it should be seen as a default supported deployment option. And
It is responsible for watching pod events on the node it's running on, running without it is impossible starting from Stein release. It is responsible
answering calls from CNI Driver and attaching VIFs when they are ready. In the for watching pod events on the node it's running on, answering calls from CNI
future it will also keep information about pooled ports in memory. This helps Driver and attaching VIFs when they are ready. In the future it will also keep
to limit the number of processes spawned when creating multiple Pods, as a information about pooled ports in memory. This helps to limit the number of
single Watcher is enough for each node and CNI Driver will only wait on local processes spawned when creating multiple Pods, as a single Watcher is enough
network socket for response from the Daemon. for each node and CNI Driver will only wait on local network socket for
response from the Daemon.
Currently CNI Daemon consists of two processes i.e. Watcher and Server. Currently CNI Daemon consists of two processes i.e. Watcher and Server.
Processes communicate between each other using Python's Processes communicate between each other using Python's
@ -229,7 +237,7 @@ expected to be JSON).
For reference see updated pod creation flow diagram: For reference see updated pod creation flow diagram:
.. image:: ../../images/pod_creation_flow_daemon.png .. image:: ../../images/pod_creation_flow.png
:alt: Controller-CNI-daemon interaction :alt: Controller-CNI-daemon interaction
:align: center :align: center
:width: 100% :width: 100%
@ -255,43 +263,6 @@ deserialized using o.vo's ``obj_from_primitive()`` method.
When running in daemonized mode, CNI Driver will call CNI Daemon over those APIs When running in daemonized mode, CNI Driver will call CNI Daemon over those APIs
to perform its tasks and wait on socket for result. to perform its tasks and wait on socket for result.
CNI Driver (deprecated)
-----------------------
.. warning::
Running with CNI Driver in this mode is deprecated since Rocky release.
Currently the preferred way of deploying kuryr-kubernetes is with
kuryr-daemon that takes over most of the CNI Driver tasks. In that case CNI
driver becomes a thin client that passes CNI ADD and DEL requests to
kuryr-daemon instance via its HTTP API.
Kuryr kubernetes integration takes advantage of the kubernetes `CNI plugin
<http://kubernetes.io/docs/admin/network-plugins/#cni>`_ and introduces
Kuryr-K8s CNI Driver. Based on design decision, kuryr-kubernetes
CNI Driver should get all information required to plug and bind Pod via
kubernetes control plane and should not depend on Neutron. CNI plugin/driver
is invoked in a blocking manner by kubelet (Kubernetes node agent), therefore
it is expected to return when either success or error state determined.
Kuryr-K8s CNI Driver has 2 sources for Pod binding information: kubelet/node
environment and Kubernetes API. The Kuryr-K8s Controller Service and CNI share the
contract that defines Pod annotation that Controller Server adds and CNI
driver reads. The contract is `os_vif VIF
<https://github.com/openstack/os-vif/blob/master/os_vif/objects/vif.py>`_
With VIF object loaded from the Pod object annotation, the CNI driver performs
Pod plugging. Kuryr-K8s CNI driver uses ov_vif library to perform Pod plug and
unplug operations. The CNI driver should complete its job and return control to
Kubelet when all the network plugging is completed.
In the cases when Neutron initially creates port in 'Down' state, CNI driver
will plug the Pod, but will have to watch the Pod annotations for vif state
change to 'Active' before returning the control to the caller.
.. image:: ../../images/pod_creation_flow.png
:alt: Controller-CNI interaction
:align: center
:width: 100%
Kubernetes Documentation Kubernetes Documentation
------------------------ ------------------------

View File

@ -48,9 +48,6 @@ Now edit ``devstack/local.conf`` to set up some initial options:
omitted. omitted.
* If you already have Docker installed on the machine, you can comment out line * If you already have Docker installed on the machine, you can comment out line
starting with ``enable_plugin devstack-plugin-container``. starting with ``enable_plugin devstack-plugin-container``.
* If you want to disable kuryr-daemon add ``disable_service kuryr-daemon``
line. Please note that running without kuryr-daemon was deprecated in Rocky
release.
Once ``local.conf`` is configured, you can start the installation: :: Once ``local.conf`` is configured, you can start the installation: ::

View File

@ -26,7 +26,6 @@ from os_vif.objects import base
from oslo_log import log as logging from oslo_log import log as logging
from oslo_serialization import jsonutils from oslo_serialization import jsonutils
from kuryr_kubernetes.cni import utils
from kuryr_kubernetes import config from kuryr_kubernetes import config
from kuryr_kubernetes import constants as k_const from kuryr_kubernetes import constants as k_const
from kuryr_kubernetes import exceptions as k_exc from kuryr_kubernetes import exceptions as k_exc
@ -129,25 +128,6 @@ class CNIRunner(object):
return result return result
class CNIStandaloneRunner(CNIRunner):
def __init__(self, plugin):
self._plugin = plugin
def _add(self, params):
vif = self._plugin.add(params)
return self._vif_data(vif, params)
def _delete(self, params):
self._plugin.delete(params)
def prepare_env(self, env, stdin):
return utils.CNIParameters(env, stdin)
def get_container_id(self, params):
return params.CNI_CONTAINERID
class CNIDaemonizedRunner(CNIRunner): class CNIDaemonizedRunner(CNIRunner):
def _add(self, params): def _add(self, params):

View File

@ -21,11 +21,9 @@ import sys
import os_vif import os_vif
from oslo_config import cfg from oslo_config import cfg
from oslo_log import log as logging from oslo_log import log as logging
from oslo_log import versionutils
from oslo_serialization import jsonutils from oslo_serialization import jsonutils
from kuryr_kubernetes.cni import api as cni_api from kuryr_kubernetes.cni import api as cni_api
from kuryr_kubernetes.cni.plugins import k8s_cni
from kuryr_kubernetes.cni import utils from kuryr_kubernetes.cni import utils
from kuryr_kubernetes import config from kuryr_kubernetes import config
from kuryr_kubernetes import constants as k_const from kuryr_kubernetes import constants as k_const
@ -56,13 +54,7 @@ def run():
k_objects.register_locally_defined_vifs() k_objects.register_locally_defined_vifs()
os_vif.initialize() os_vif.initialize()
if CONF.cni_daemon.daemon_enabled: runner = cni_api.CNIDaemonizedRunner()
runner = cni_api.CNIDaemonizedRunner()
else:
versionutils.deprecation_warning(
'Deploying kuryr-kubernetes without kuryr-daemon service', 'R')
runner = cni_api.CNIStandaloneRunner(k8s_cni.K8sCNIPlugin())
LOG.info("Using '%s' ", runner.__class__.__name__)
def _timeout(signum, frame): def _timeout(signum, frame):
runner._write_dict(sys.stdout, { runner._write_dict(sys.stdout, {

View File

@ -1,49 +0,0 @@
# Copyright (c) 2016 Mirantis, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from kuryr_kubernetes import clients
from kuryr_kubernetes.cni import handlers as h_cni
from kuryr_kubernetes.cni.plugins import base as base_cni
from kuryr_kubernetes import constants as k_const
from kuryr_kubernetes import watcher as k_watcher
class K8sCNIPlugin(base_cni.CNIPlugin):
def add(self, params):
self._setup(params)
self._pipeline.register(h_cni.AddHandler(params, self._done))
self._watcher.start()
return self._vif
def delete(self, params):
self._setup(params)
self._pipeline.register(h_cni.DelHandler(params, self._done))
self._watcher.start()
def _done(self, vif):
self._vif = vif
self._watcher.stop()
def _setup(self, params):
clients.setup_kubernetes_client()
self._pipeline = h_cni.CNIPipeline()
self._watcher = k_watcher.Watcher(self._pipeline)
self._watcher.add(
"%(base)s/namespaces/%(namespace)s/pods"
"?fieldSelector=metadata.name=%(pod)s" % {
'base': k_const.K8S_API_BASE,
'namespace': params.args.K8S_POD_NAMESPACE,
'pod': params.args.K8S_POD_NAME})

View File

@ -31,13 +31,6 @@ kuryr_k8s_opts = [
] ]
daemon_opts = [ daemon_opts = [
cfg.BoolOpt('daemon_enabled',
help=_('Enable CNI Daemon configuration.'),
default=True,
deprecated_for_removal=True,
deprecated_reason="Deployment without kuryr-daemon is now "
"deprecated.",
deprecated_since="Rocky"),
cfg.StrOpt('bind_address', cfg.StrOpt('bind_address',
help=_('Bind address for CNI daemon HTTP server. It is ' help=_('Bind address for CNI daemon HTTP server. It is '
'recommened to allow only local connections.'), 'recommened to allow only local connections.'),

View File

@ -1,67 +0,0 @@
# Copyright (c) 2017 NEC Corporation.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import mock
from kuryr_kubernetes.cni.plugins import k8s_cni
from kuryr_kubernetes import constants
from kuryr_kubernetes.tests import base as test_base
class TestK8sCNIPlugin(test_base.TestCase):
@mock.patch('kuryr_kubernetes.watcher.Watcher')
@mock.patch('kuryr_kubernetes.cni.handlers.CNIPipeline')
@mock.patch('kuryr_kubernetes.cni.handlers.DelHandler')
@mock.patch('kuryr_kubernetes.cni.handlers.AddHandler')
def _test_method(self, method, m_add_handler, m_del_handler, m_cni_pipe,
m_watcher_class):
self.passed_handler = None
def _save_handler(params, handler):
self.passed_handler = handler
def _call_handler(*args):
self.passed_handler(mock.sentinel.vif)
m_add_handler.side_effect = _save_handler
m_del_handler.side_effect = _save_handler
m_watcher = mock.MagicMock(
add=mock.MagicMock(),
start=mock.MagicMock(side_effect=_call_handler))
m_watcher_class.return_value = m_watcher
m_params = mock.MagicMock()
m_params.args.K8S_POD_NAMESPACE = 'k8s_pod_namespace'
m_params.args.K8S_POD_NAME = 'k8s_pod'
cni_plugin = k8s_cni.K8sCNIPlugin()
result = getattr(cni_plugin, method)(m_params)
self.assertEqual(mock.sentinel.vif, cni_plugin._vif)
m_watcher.add.assert_called_with(
"%(base)s/namespaces/%(namespace)s/pods"
"?fieldSelector=metadata.name=%(pod)s" % {
'base': constants.K8S_API_BASE,
'namespace': m_params.args.K8S_POD_NAMESPACE,
'pod': m_params.args.K8S_POD_NAME})
return result
def test_add(self):
result = self._test_method('add')
self.assertEqual(result, mock.sentinel.vif)
def test_delete(self):
self._test_method('delete')

View File

@ -22,7 +22,6 @@ from oslo_config import cfg
from oslo_serialization import jsonutils from oslo_serialization import jsonutils
from kuryr_kubernetes.cni import api from kuryr_kubernetes.cni import api
from kuryr_kubernetes.cni.plugins import k8s_cni
from kuryr_kubernetes.tests import base as test_base from kuryr_kubernetes.tests import base as test_base
from kuryr_kubernetes.tests import fake from kuryr_kubernetes.tests import fake
@ -51,62 +50,6 @@ class TestCNIRunnerMixin(object):
self.assertEqual(api.CNIRunner.VERSION, result['cniVersion']) self.assertEqual(api.CNIRunner.VERSION, result['cniVersion'])
class TestCNIStandaloneRunner(test_base.TestCase, TestCNIRunnerMixin):
def setUp(self):
super(TestCNIStandaloneRunner, self).setUp()
self.runner = api.CNIStandaloneRunner(k8s_cni.K8sCNIPlugin())
@mock.patch('kuryr_kubernetes.cni.plugins.k8s_cni.K8sCNIPlugin.add')
def test_run_add(self, m_k8s_add):
vif = fake._fake_vif()
m_k8s_add.return_value = vif
m_fin = StringIO()
m_fout = StringIO()
container_id = 'a4181c680a39'
env = {
'CNI_COMMAND': 'ADD',
'CNI_CONTAINERID': container_id,
'CNI_ARGS': 'foo=bar',
}
self.runner.run(env, m_fin, m_fout)
self.assertTrue(m_k8s_add.called)
self.assertEqual('foo=bar', m_k8s_add.call_args[0][0].CNI_ARGS)
result = jsonutils.loads(m_fout.getvalue())
self.assertDictEqual(
{"cniVersion": '0.3.1',
"dns": {"nameservers": ["192.168.0.1"]},
"ips": [
{
"version": "4",
"gateway": "192.168.0.1",
"address": "192.168.0.2/24",
"interface": 0,
}],
"interfaces": [
{
"name": vif.vif_name,
"mac": vif.address,
"sandbox": container_id,
}],
"routes": []},
result)
@mock.patch('kuryr_kubernetes.cni.plugins.k8s_cni.K8sCNIPlugin.delete')
def test_run_del(self, m_k8s_delete):
vif = fake._fake_vif()
m_k8s_delete.return_value = vif
m_fin = StringIO()
m_fout = StringIO()
env = {
'CNI_COMMAND': 'DEL',
'CNI_CONTAINERID': 'a4181c680a39',
'CNI_ARGS': 'foo=bar',
}
self.runner.run(env, m_fin, m_fout)
self.assertTrue(m_k8s_delete.called)
self.assertEqual('foo=bar', m_k8s_delete.call_args[0][0].CNI_ARGS)
@mock.patch('requests.post') @mock.patch('requests.post')
class TestCNIDaemonizedRunner(test_base.TestCase, TestCNIRunnerMixin): class TestCNIDaemonizedRunner(test_base.TestCase, TestCNIRunnerMixin):
def setUp(self): def setUp(self):

View File

@ -15,8 +15,6 @@
import mock import mock
from oslo_config import cfg
from kuryr_kubernetes.cni import main from kuryr_kubernetes.cni import main
from kuryr_kubernetes.tests import base as test_base from kuryr_kubernetes.tests import base as test_base
@ -37,32 +35,6 @@ class TestCNIMain(test_base.TestCase):
m_cni_dr.return_value = mock.MagicMock() m_cni_dr.return_value = mock.MagicMock()
m_cni_daemon = m_cni_dr.return_value m_cni_daemon = m_cni_dr.return_value
cfg.CONF.set_override('daemon_enabled', True, group='cni_daemon')
main.run()
m_config_init.assert_called()
m_setup_logging.assert_called()
m_cni_daemon.run.assert_called()
m_sysexit.assert_called()
@mock.patch('kuryr_kubernetes.cni.main.jsonutils.load')
@mock.patch('sys.exit')
@mock.patch('sys.stdin')
@mock.patch('kuryr_kubernetes.cni.utils.CNIConfig')
@mock.patch('kuryr_kubernetes.cni.api')
@mock.patch('kuryr_kubernetes.config.init')
@mock.patch('kuryr_kubernetes.config.setup_logging')
@mock.patch('kuryr_kubernetes.cni.api.CNIStandaloneRunner')
def test_standalone_run(self, m_cni_sr, m_setup_logging, m_config_init,
m_api, m_conf, m_sys, m_sysexit, m_json):
m_conf.debug = mock.Mock()
m_conf.debug.return_value = True
m_cni_sr.return_value = mock.MagicMock()
m_cni_daemon = m_cni_sr.return_value
cfg.CONF.set_override('daemon_enabled', False, group='cni_daemon')
main.run() main.run()
m_config_init.assert_called() m_config_init.assert_called()

View File

@ -0,0 +1,5 @@
---
upgrade:
- |
As announced, possiblity of running Kuryr-Kubernetes without kuryr-daemon
service is now removed from the project and considered not supported.