CNI Daemon documentation

This commit extends developer docs to add information about CNI Daemon
and APIs between CNI Driver and CNI Daemon.

Implements: blueprint cni-split-exec-daemon
Co-Authored-By: Janonymous <janonymous.codevulture@gmail.com>
Change-Id: I0188c133d656e32b75bf7d9b6c012da33ffa2571

doc/source/devref/kuryr_kubernetes_design.rst

@@ -192,6 +192,66 @@ change to ‘Active’ before returning the control to the caller.
     :align: center
     :width: 100%
 
+.. _cni-daemon:
+
+CNI Daemon
+----------
+
+CNI Daemon is an optional service that should run on every Kubernetes node. It
+is responsible for watching pod events on the node it's running on, answering
+calls from CNI Driver and attaching VIFs when they are ready. In the future
+it will also keep information about pooled ports in memory. This helps to limit
+the number of processes spawned when creating multiple Pods, as a single
+Watcher is enough for each node and CNI Driver will only wait on local network
+socket for response from the Daemon.
+
+Currently CNI Daemon consists of two processes i.e. Watcher and Server.
+Processes communicate between each other using Python's
+``multiprocessing.Manager`` and a shared dictionary object. Watcher is
+responsible for extracting VIF annotations from Pod events and putting them
+into the shared dictionary. Server is a regular WSGI server that will answer
+CNI Driver calls. When a CNI request comes, Server is waiting for VIF object to
+appear in the shared dictionary. As annotations are read from
+kubernetes API and added to the registry by Watcher thread, Server will
+eventually get VIF it needs to connect for a given pod. Then it waits for the
+VIF to become active before returning to the CNI Driver.
+
+Communication
+~~~~~~~~~~~~~
+
+CNI Daemon Server is starting an HTTP server on a local network socket
+(``127.0.0.1:50036`` by default). Currently server is listening for 2 API
+calls. Both calls load the ``CNIParameters`` from the body of the call (it is
+expected to be JSON).
+
+For reference see updated pod creation flow diagram:
+
+.. image:: ../../images/pod_creation_flow_daemon.png
+    :alt: Controller-CNI-daemon interaction
+    :align: center
+    :width: 100%
+
+/addNetwork
++++++++++++
+**Function**: Is equivalent of running ``K8sCNIPlugin.add``.
+
+**Return code:** 201 Created
+
+**Return body:** Returns VIF data in JSON form. This is serialized
+oslo.versionedobject from ``os_vif`` library. On the other side it can be
+deserialized using o.vo's ``obj_from_primitive()`` method.
+
+/delNetwork
++++++++++++
+**Function**: Is equivalent of running ``K8sCNIPlugin.delete``.
+
+**Return code:** 204 No content
+
+**Return body:** None.
+
+When running in daemonized mode, CNI Driver will call CNI Daemon over those APIs
+to perform its tasks and wait on socket for result.
+
 Kubernetes Documentation
 ------------------------
 The `Kubernetes reference documentation <https://kubernetes.io/docs/reference/>`_

doc/source/installation/devstack/basic.rst

@@ -48,6 +48,8 @@ Now edit ``devstack/local.conf`` to set up some initial options:
   omitted.
 * If you already have Docker installed on the machine, you can comment out line
   starting with ``enable_plugin devstack-plugin-container``.
+* If you want to enable kuryr-daemon uncomment ``enable_service kuryr-daemon``
+  line.
 
 Once ``local.conf`` is configured, you can start the installation: ::
 

doc/source/installation/manual.rst

@@ -157,3 +157,52 @@ to work correctly::
 
     deactivate
     sudo pip install 'oslo.privsep>=1.20.0' 'os-vif>=1.5.0'
+
+Configure Kuryr CNI Daemon (optional)
+-------------------------------------
+
+Kuryr CNI Daemon is an optional service designed to increased scalability of
+the Kuryr operations done on Kubernetes nodes. More information can be found on
+:ref:`cni-daemon` page.
+
+If you want to use Kuryr CNI Daemon, it needs to be installed on every
+Kubernetes node, so following steps need to be repeated.
+
+Edit ``kuryr.conf``::
+
+    [cni_daemon]
+    daemon_enabled=True
+
+.. note::
+  You can tweak configuration of some timeouts to match your environment. It's
+  crucial for scalability of the whole deployment. In general the timeout to
+  serve CNI request from kubelet to Kuryr is 180 seconds. After that time
+  kubelet will retry the request. Additionally there are two configuration
+  options::
+
+    [cni_daemon]
+    vif_annotation_timeout=60
+    pyroute2_timeout=10
+
+  ``vif_annotation_timeout`` is time the Kuryr CNI Daemon will wait for Kuryr
+  Controller to create a port in Neutron and add information about it to Pod's
+  metadata. If either Neutron or Kuryr Controller doesn't keep up with high
+  number of requests, it's advised to increase this timeout. Please note that
+  increasing it over 180 seconds will not have any effect as the request will
+  time out anyway and will be retried (which is safe).
+
+  ``pyroute2_timeout`` is internal timeout of pyroute2 library, that is
+  responsible for doing modifications to Linux Kernel networking stack (e.g.
+  moving interfaces to Pod's namespaces, adding routes and ports or assigning
+  addresses to interfaces). When serving a lot of ADD/DEL CNI requests on a
+  regular basis it's advised to increase that timeout. Please note that the
+  value denotes *maximum* time to wait for kernel to complete the operations.
+  If operation succeeds earlier, request isn't delayed.
+
+Run kuryr-daemon::
+
+    $ kuryr-daemon --config-file /etc/kuryr/kuryr.conf -d
+
+Alternatively you may run it in screen::
+
+    $ screen -dm kuryr-daemon --config-file /etc/kuryr/kuryr.conf -d