From 0b0af3d3e63b13dd93afed785cecbb8920658ff2 Mon Sep 17 00:00:00 2001 From: Taku Fukushima Date: Tue, 15 Mar 2016 20:42:24 +0900 Subject: [PATCH] Add devref of Raven This patch adds the devref of Raven, a K8s API watcher component. The devref contains the abstract of Raven, the design of it, the useful information for developers. Change-Id: I8f0e801f81f101a51d62f4b4cc854176e67e75dc Signed-off-by: Taku Fukushima Related: blueprint kuryr-k8s-integration --- doc/source/devref/k8s_api_watcher_design.rst | 959 +++++++++++++++++++ 1 file changed, 959 insertions(+) create mode 100644 doc/source/devref/k8s_api_watcher_design.rst diff --git a/doc/source/devref/k8s_api_watcher_design.rst b/doc/source/devref/k8s_api_watcher_design.rst new file mode 100644 index 00000000..9719a376 --- /dev/null +++ b/doc/source/devref/k8s_api_watcher_design.rst @@ -0,0 +1,959 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + + Convention for heading levels in Neutron devref: + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + (Avoid deeper levels because they do not render well.) + +============================= +Kubernetes API Watcher Design +============================= + +This documentation describes the `Kuberenetes API`_ watcher daemon component, +**Raven**, of Kuryr. + +What is Raven +------------- + +Raven is a daemon watches the internal Kubernetes (K8s) state through its API +server and receives the changes with the event notifications. Raven then +translate the state changes of K8s into requests against Neutron API and +constructs the virtual network topology on Neutron. + +Raven must act as the centralized component for the translations due to the +constraints come from the concurrent deployments of the pods on worker nodes. +Unless it's centralized, each plugin on each work node would make requests +against Neutron API and it would lead the conflicts of the requests due to the +race condition because of the lack of the lock or the serialization mechanisms +for the requests against Neutron API. + +Raven doesn't take care of the bindings between the virtual ports and the +physical interfaces on worker nodes. It is the responsibility of Kuryr CNI_ +plugin for K8s and it shall recognize which Neutron port should be bound to the +physical interface associated with the pod to be deployed. So Raven focuses +only on building the virtual network topology translated from the events of the +internal state changes of K8s through its API server. + +Goal +~~~~ + +Through Raven the changes to K8s API server are translated into the appropriate +requests against Neutron API and we can make sure their logical networking states +are synchronized and consistent when the pods are deployed. + +Translation Mapping +------------------- + +The detailed specification of the translation mappings is described in another +document, :doc:`../specs/kuryr_k8s_integration`. In this document we touch what +to be translated briefly. + +The main focus of Raven is the following resources. + +* Namespace +* Pod +* Service (Optional) + +Namespaces are translated into the networking basis, Neutron networks and +subnets for the cluster and the service with the explicitly predefined values +in the configuration file, or implicitly specified by the environment +variables, e.g., ``FLANNEL_NET=172.16.0.0/16`` as specified +`in the deployment phase`_, or . Raven also creates Neutron routers for +connecting the cluster subnets and the service subnets. + +When each namespace is created, a cluster network that contains a cluster +subnet, a service network that contains a service subnet, and a router that +connects the cluster subnet and the service subnet are created through Neutron +API. The apiserver ensures namespaces are created before pods are creted. + +Pods contain the information required by creating Neutron ports. If pods are +associated with the specific namespace, the ports are created and associated +with the subnets for the namespace. In this case Raven also creates a LBaaS +member for each port translated from the pod coordinating with the service +translation. + +Although it's optional, Raven can emulate kube-proxy_. This is for the network +controller that leverages isolated datapath from ``docker0`` bridge such as +Open vSwitch datapath. Services contain the information for the emulation. Raven +maps kube-proxy to Neutron load balancers with VIPs. For "externalIPs" type K8s +service, Raven associates a floating IP with a load balancer for enabling the +pubilc accesses. + +================= ============= +Kubernetes Neutron +================= ============= +Namespace Network +(Cluster subnet) (Subnet) +Pod Port + LBaaS Member +Service LBaaS Pool + LBaaS VIP + (FloatingIP) +================= ============= + + +.. _k8s-api-behaviour: + +K8s API behaviour +----------------- + +We look at the responses from the pod endpoints as an exmple. + +The following behaviour is based on the 1.2.0 release, which is the latest one +as of March 17th, 2016. + +:: + + $ ./kubectl.sh version + Client Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.0", GitCommit:"5cb86ee022267586db386f62781338b0483733b3", GitTreeState:"clean"} + Server Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.0", GitCommit:"5cb86ee022267586db386f62781338b0483733b3", GitTreeState:"clean"} + +Regular requests +~~~~~~~~~~~~~~~~ + +If there's no pod, the K8s API server returns the following JSON response that +has the empty list for the ``"items"`` property. + +:: + + $ curl -X GET -i http://127.0.0.1:8080/api/v1/pods + HTTP/1.1 200 OK + Content-Type: application/json + Date: Tue, 15 Mar 2016 07:15:46 GMT + Content-Length: 145 + + { + "kind": "PodList", + "apiVersion": "v1", + "metadata": { + "selfLink": "/api/v1/pods", + "resourceVersion": "227806" + }, + "items": [] + } + +We deploy a pod as follow. + +:: + + $ ./kubectl.sh run --image=nginx nginx-app --port=80 + replicationcontroller "nginx-app" created + +Then the response from the API server contains the pod information in +``"items"`` property of the JSON response. + +:: + + $ curl -X GET -i http://127.0.0.1:8080/api/v1/pods + HTTP/1.1 200 OK + Content-Type: application/json + Date: Tue, 15 Mar 2016 08:18:25 GMT + Transfer-Encoding: chunked + + { + "kind": "PodList", + "apiVersion": "v1", + "metadata": { + "selfLink": "/api/v1/pods", + "resourceVersion": "228211" + }, + "items": [ + { + "metadata": { + "name": "nginx-app-o0kvl", + "generateName": "nginx-app-", + "namespace": "default", + "selfLink": "/api/v1/namespaces/default/pods/nginx-app-o0kvl", + "uid": "090cc0c8-ea84-11e5-8c79-42010af00003", + "resourceVersion": "228094", + "creationTimestamp": "2016-03-15T08:00:51Z", + "labels": { + "run": "nginx-app" + }, + "annotations": { + "kubernetes.io/created-by": "{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicationController\",\"namespace\":\"default\",\"name\":\"nginx-app\",\"uid\":\"090bfb57-ea84-11e5-8c79-42010af00003\",\"apiVersion\":\"v1\",\"resourceVersion\":\"228081\"}}\n" + } + }, + "spec": { + "volumes": [ + { + "name": "default-token-wpfjn", + "secret": { + "secretName": "default-token-wpfjn" + } + } + ], + "containers": [ + { + "name": "nginx-app", + "image": "nginx", + "ports": [ + { + "containerPort": 80, + "protocol": "TCP" + } + ], + "resources": {}, + "volumeMounts": [ + { + "name": "default-token-wpfjn", + "readOnly": true, + "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount" + } + ], + "terminationMessagePath": "/dev/termination-log", + "imagePullPolicy": "Always" + } + ], + "restartPolicy": "Always", + "terminationGracePeriodSeconds": 30, + "dnsPolicy": "ClusterFirst", + "serviceAccountName": "default", + "serviceAccount": "default", + "nodeName": "10.240.0.4", + "securityContext": {} + }, + "status": { + "phase": "Running", + "conditions": [ + { + "type": "Ready", + "status": "True", + "lastProbeTime": null, + "lastTransitionTime": "2016-03-15T08:00:52Z" + } + ], + "hostIP": "10.240.0.4", + "podIP": "172.16.49.2", + "startTime": "2016-03-15T08:00:51Z", + "containerStatuses": [ + { + "name": "nginx-app", + "state": { + "running": { + "startedAt": "2016-03-15T08:00:52Z" + } + }, + "lastState": {}, + "ready": true, + "restartCount": 0, + "image": "nginx", + "imageID": "docker://sha256:af4b3d7d5401624ed3a747dc20f88e2b5e92e0ee9954aab8f1b5724d7edeca5e", + "containerID": "docker://b97168314ad58404dbce7cb94291db7a976d2cb824b39e5864bf4bdaf27af255" + } + ] + } + } + ] + } + +We get the current snapshot of the requested resources with the regular +requests against the K8s API server. + +Requests with ``watch=true`` query string +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +K8s provides the "watch" capability for the endpoints with ``/watch/`` prefix +for the specific resource name, i.e., ``/api/v1/watch/pods``, or ``watch=true`` +query string. + +If there's no pod, we get only the response header and the connection is kept +open. + +:: + + $ curl -X GET -i http://127.0.0.1:8080/api/v1/pods?watch=true + HTTP/1.1 200 OK + Transfer-Encoding: chunked + Date: Tue, 15 Mar 2016 08:00:09 GMT + Content-Type: text/plain; charset=utf-8 + Transfer-Encoding: chunked + + +We create a pod as we did for the case without the ``watch=true`` query string. + +:: + + $ ./kubectl.sh run --image=nginx nginx-app --port=80 + replicationcontroller "nginx-app" created + +Then we observe the JSON data corresponds to the event is given by each line. +The event type is given in ``"type"`` property of the JSON data, i.e., +``"ADDED"``, ``"MODIFIED"`` and ``"DELETED"``. + +:: + + $ curl -X GET -i http://127.0.0.1:8080/api/v1/pods?watch=true + HTTP/1.1 200 OK + Transfer-Encoding: chunked + Date: Tue, 15 Mar 2016 08:00:09 GMT + Content-Type: text/plain; charset=utf-8 + Transfer-Encoding: chunked + + {"type":"ADDED","object":{"kind":"Pod","apiVersion":"v1","metadata":{"name":"nginx-app-o0kvl","generateName":"nginx-app-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/nginx-app-o0kvl","uid":"090cc0c8-ea84-11e5-8c79-42010af00003","resourceVersion":"228082","creationTimestamp":"2016-03-15T08:00:51Z","labels":{"run":"nginx-app"},"annotations":{"kubernetes.io/created-by":"{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicationController\",\"namespace\":\"default\",\"name\":\"nginx-app\",\"uid\":\"090bfb57-ea84-11e5-8c79-42010af00003\",\"apiVersion\":\"v1\",\"resourceVersion\":\"228081\"}}\n"}},"spec":{"volumes":[{"name":"default-token-wpfjn","secret":{"secretName":"default-token-wpfjn"}}],"containers":[{"name":"nginx-app","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-wpfjn","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{}},"status":{"phase":"Pending"}}} + {"type":"MODIFIED","object":{"kind":"Pod","apiVersion":"v1","metadata":{"name":"nginx-app-o0kvl","generateName":"nginx-app-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/nginx-app-o0kvl","uid":"090cc0c8-ea84-11e5-8c79-42010af00003","resourceVersion":"228084","creationTimestamp":"2016-03-15T08:00:51Z","labels":{"run":"nginx-app"},"annotations":{"kubernetes.io/created-by":"{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicationController\",\"namespace\":\"default\",\"name\":\"nginx-app\",\"uid\":\"090bfb57-ea84-11e5-8c79-42010af00003\",\"apiVersion\":\"v1\",\"resourceVersion\":\"228081\"}}\n"}},"spec":{"volumes":[{"name":"default-token-wpfjn","secret":{"secretName":"default-token-wpfjn"}}],"containers":[{"name":"nginx-app","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-wpfjn","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"10.240.0.4","securityContext":{}},"status":{"phase":"Pending"}}} + {"type":"MODIFIED","object":{"kind":"Pod","apiVersion":"v1","metadata":{"name":"nginx-app-o0kvl","generateName":"nginx-app-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/nginx-app-o0kvl","uid":"090cc0c8-ea84-11e5-8c79-42010af00003","resourceVersion":"228088","creationTimestamp":"2016-03-15T08:00:51Z","labels":{"run":"nginx-app"},"annotations":{"kubernetes.io/created-by":"{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicationController\",\"namespace\":\"default\",\"name\":\"nginx-app\",\"uid\":\"090bfb57-ea84-11e5-8c79-42010af00003\",\"apiVersion\":\"v1\",\"resourceVersion\":\"228081\"}}\n"}},"spec":{"volumes":[{"name":"default-token-wpfjn","secret":{"secretName":"default-token-wpfjn"}}],"containers":[{"name":"nginx-app","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-wpfjn","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"10.240.0.4","securityContext":{}},"status":{"phase":"Pending","conditions":[{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2016-03-15T08:00:51Z","reason":"ContainersNotReady","message":"containers with unready status: [nginx-app]"}],"hostIP":"10.240.0.4","startTime":"2016-03-15T08:00:51Z","containerStatuses":[{"name":"nginx-app","state":{"waiting":{"reason":"ContainerCreating","message":"Image: nginx is ready, container is creating"}},"lastState":{},"ready":false,"restartCount":0,"image":"nginx","imageID":""}]}}} + {"type":"MODIFIED","object":{"kind":"Pod","apiVersion":"v1","metadata":{"name":"nginx-app-o0kvl","generateName":"nginx-app-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/nginx-app-o0kvl","uid":"090cc0c8-ea84-11e5-8c79-42010af00003","resourceVersion":"228094","creationTimestamp":"2016-03-15T08:00:51Z","labels":{"run":"nginx-app"},"annotations":{"kubernetes.io/created-by":"{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicationController\",\"namespace\":\"default\",\"name\":\"nginx-app\",\"uid\":\"090bfb57-ea84-11e5-8c79-42010af00003\",\"apiVersion\":\"v1\",\"resourceVersion\":\"228081\"}}\n"}},"spec":{"volumes":[{"name":"default-token-wpfjn","secret":{"secretName":"default-token-wpfjn"}}],"containers":[{"name":"nginx-app","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-wpfjn","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"10.240.0.4","securityContext":{}},"status":{"phase":"Running","conditions":[{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2016-03-15T08:00:52Z"}],"hostIP":"10.240.0.4","podIP":"172.16.49.2","startTime":"2016-03-15T08:00:51Z","containerStatuses":[{"name":"nginx-app","state":{"running":{"startedAt":"2016-03-15T08:00:52Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"nginx","imageID":"docker://sha256:af4b3d7d5401624ed3a747dc20f88e2b5e92e0ee9954aab8f1b5724d7edeca5e","containerID":"docker://b97168314ad58404dbce7cb94291db7a976d2cb824b39e5864bf4bdaf27af255"}]}}} + +Raven Technical Design Overview +------------------------------- + +Problem Statement +~~~~~~~~~~~~~~~~~ + +To conform to the I/O bound requirement described in :ref:`k8s-api-behaviour`, +the multiplexed concurrent network I/O is demanded. eventlet_ is used in +various OpenStack projects for this purpose as well as other libraries such as +Twisted_, Tornado_ and gevent_. However, it has problems as described in +"`What's wrong with eventlet?`_" on the OpenStack wiki page. + +asyncio and Python 3 by default +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +asyncio_ was introduced as a standard asynchronous I/O library in Python 3.4. +Its event loop and coroutines provide the mechanism to multiplex network I/O +in the asynchronous fashion. Compared with eventlet, we can explicitly mark the +I/O operations asynchronous with ``yield from`` or ``await`` introduced in +Python 3.5. + +Trollius_ is a port of asyncio to Python 2.x. However `Trollius documentation`_ +is describing a list of problems and even promoting the migration to Python 3 +with asyncio. + +Kuryr is still a quite young project in OpenStack Neutron big tent. In addition +to that, since it's a container related project it should be able to be run +inside a container. So do Raven. Therefore we take a path to support for only +Python 3 and drop Python 2. + +With asyncio we can achieve concurrent networking I/O operations required by +watchers watch multiple endpoints and translate their responses into requests +against Neutorn and K8s API server. + +Watchers +~~~~~~~~ + +A watcher can be represented as a pair of an API endpoint and a function used +for the translation essentially. That is, the pair of what is translated and +how it is. The API endpoint URI is associated with the stream of the event +notifications and the translation function maps each event coming from the +apiserver into another form such as the request against Neutron API server. + +Watchers can be considered as concerns and reactions. They should be decouped +from the actual task dispatcher and their consumers. A single or multiple +watchers can be mixed into the single class that leverages them, i.e., Raven, +or even mutliple classes leverage them can have the same concern and the same +reaction. The watchers can be able to be mixed into the single entity of the +watcher user but they should work independently. For instance, ``AliceWatcher`` +does its work and knows nothing about other watchers such as ``BobWatcher``. +They don't work together depending on one or each. + +A minimum watcher can be defined as follow. + +.. code-block:: python + + from kuryr.raven import watchers + + class SomeWatcher(watchers.K8sApiWatcher): + WATCH_ENDPOINT = '/' + + def translate(self, deserialized_json): + pass + +The watcher is defined in the declarative way and ideally doesn't care when it +is called and by whom. However, it needs to recongnize the context such as the +event type and behave appropriately according to the situation. + +Raven +~~~~~ + +Raven acts as a daemon and it should be able to be started or stopped by +operators. It delegates the actual watch tasks to the watchers and dispatch +them with the single JSON response corresponds to each endpoint on which the +watcher has its concern. + +Hence, Raven holds one or multiple watchers, opens connections for each +endpoint, makes HTTP requests, gets HTTP responses and parses every event +notification and dispatches the translate methods of the watchers routed based +on their corresponding endpoints. + +To register the watchers to Raven or any class, ``register_watchers`` decorator +is used. It simply inserts the watchers into the dictionary in the class, +``WATCH_ENDPOINTS_AND_CALLBACKS`` and it's up to the class how use the +registerd watchers. The classes passed to ``register_watchers`` are defined in +the configuration file and you can specify only what you need. + +In the case of Raven, it starts the event loop, open connections for each +registered watcher and keeps feeding the notified events to the translate +methods of the watchers. + +Raven is a service implements ``oslo_service.service.Service``. When ``start`` +method is called, it starts the event loop and delegatations of the watch tasks. +If ``SIGINT`` or ``SIGTERM`` signal is sent to Raven, it cancells all watch +tasks, closes connections and stops immediately. Otherwise Raven lets watchers +keep watching the API endpionts until the API server sends EOF strings. When +``stop`` is called, it cancells all watch tasks, closes connections and stops +as well. + +Ideally, the translate method can be a pure function that doesn't depend on the +user of the watcher. However, the translation gets involved in requests against +Neutron and possibly the K8s API server. And it depends on the Neutron client +that shall be shared among the watchers. Hence, Raven calls the translate +methods of the watchers binding itself to ``self``. That is, Raven can +propagate its contexts to the watchers and in this way watchers can share the +same contexts. However, it's responsibilty of the writer of the watchers to +track which variables are defined in Raven and what they are. + +Appendix A: JSON response from the apiserver for each resource +-------------------------------------------------------------- + +Namespace +~~~~~~~~~ + +:: + + /api/v1/namespaces?watch=true + +ADDED ++++++ + +:: + + { + "type": "ADDED", + "object": { + "kind": "Namespace", + "apiVersion": "v1", + "metadata": { + "name": "test", + "selfLink": "/api/v1/namespaces/test", + "uid": "f094ea6b-06c2-11e6-8128-42010af00003", + "resourceVersion": "497821", + "creationTimestamp": "2016-04-20T06:41:41Z" + }, + "spec": { + "finalizers": [ + "kubernetes" + ] + }, + "status": { + "phase": "Active" + } + } + } + +MODIFIED +++++++++ + +:: + + { + "type": "MODIFIED", + "object": { + "kind": "Namespace", + "apiVersion": "v1", + "metadata": { + "name": "test", + "selfLink": "/api/v1/namespaces/test", + "uid": "f094ea6b-06c2-11e6-8128-42010af00003", + "resourceVersion": "519095", + "creationTimestamp": "2016-04-20T06:41:41Z", + "deletionTimestamp": "2016-04-21T08:47:53Z" + }, + "spec": { + "finalizers": [ + "kubernetes" + ] + }, + "status": { + "phase": "Terminating" + } + } + } + +DELETED ++++++++ + +:: + + { + "type": "DELETED", + "object": { + "kind": "Namespace", + "apiVersion": "v1", + "metadata": { + "name": "test", + "selfLink": "/api/v1/namespaces/test", + "uid": "f094ea6b-06c2-11e6-8128-42010af00003", + "resourceVersion": "519099", + "creationTimestamp": "2016-04-20T06:41:41Z", + "deletionTimestamp": "2016-04-21T08:47:53Z" + }, + "spec": {}, + "status": { + "phase": "Terminating" + } + } + } + + +Pod +~~~ + +:: + + /api/v1/pods?watch=true + +ADDED ++++++ + +:: + { + "type": "ADDED", + "object": { + "kind": "Pod", + "apiVersion": "v1", + "metadata": { + "name": "my-nginx-y67ky", + "generateName": "my-nginx-", + "namespace": "default", + "selfLink": "/api/v1/namespaces/default/pods/my-nginx-y67ky", + "uid": "d42b0bb2-dc4e-11e5-8c79-42010af00003", + "resourceVersion": "63355", + "creationTimestamp": "2016-02-26T06:04:42Z", + "labels": { + "run": "my-nginx" + }, + "annotations": { + "kubernetes.io/created-by": { + "kind": "SerializedReference", + "apiVersion": "v1", + "reference": { + "kind": "ReplicationController", + "namespace": "default", + "name": "my-nginx", + "uid": "d42a4ee1-dc4e-11e5-8c79-42010af00003", + "apiVersion": "v1", + "resourceVersion": "63348" + } + } + } + }, + "spec": { + "volumes": [ + { + "name": "default-token-wpfjn", + "secret": { + "secretName": "default-token-wpfjn" + } + } + ], + "containers": [ + { + "name": "my-nginx", + "image": "nginx", + "ports": [ + { + "containerPort": 80, + "protocol": "TCP" + } + ], + "resources": {}, + "volumeMounts": [ + { + "name": "default-token-wpfjn", + "readOnly": true, + "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount" + } + ], + "terminationMessagePath": "/dev/termination-log", + "imagePullPolicy": "Always" + } + ], + "restartPolicy": "Always", + "terminationGracePeriodSeconds": 30, + "dnsPolicy": "ClusterFirst", + "serviceAccountName": "default", + "serviceAccount": "default", + "nodeName": "10.240.0.4", + "securityContext": {} + }, + "status": { + "phase": "Pending", + "conditions": [ + { + "type": "Ready", + "status": "False", + "lastProbeTime": null, + "lastTransitionTime": "2016-02-26T06:04:43Z", + "reason": "ContainersNotReady", + "message": "containers with unready status: [my-nginx]" + } + ], + "hostIP": "10.240.0.4", + "startTime": "2016-02-26T06:04:43Z", + "containerStatuses": [ + { + "name": "my-nginx", + "state": { + "waiting": { + "reason": "ContainerCreating", + "message": "Image: nginx is ready, container is creating" + } + }, + "lastState": {}, + "ready": false, + "restartCount": 0, + "image": "nginx", + "imageID": "" + } + ] + } + } + } + +MODIFIED +~~~~~~~~ + +:: + + { + "type": "MODIFIED", + "object": { + "kind": "Pod", + "apiVersion": "v1", + "metadata": { + "name": "my-nginx-y67ky", + "generateName": "my-nginx-", + "namespace": "default", + "selfLink": "/api/v1/namespaces/default/pods/my-nginx-y67ky", + "uid": "d42b0bb2-dc4e-11e5-8c79-42010af00003", + "resourceVersion": "63425", + "creationTimestamp": "2016-02-26T06:04:42Z", + "deletionTimestamp": "2016-02-26T06:06:16Z", + "deletionGracePeriodSeconds": 30, + "labels": { + "run": "my-nginx" + }, + "annotations": { + "kubernetes.io/created-by": { + "kind": "SerializedReference", + "apiVersion": "v1", + "reference": { + "kind": "ReplicationController", + "namespace": "default", + "name": "my-nginx", + "uid": "d42a4ee1-dc4e-11e5-8c79-42010af00003", + "apiVersion": "v1", + "resourceVersion": "63348" + } + } + } + }, + "spec": { + "volumes": [ + { + "name": "default-token-wpfjn", + "secret": { + "secretName": "default-token-wpfjn" + } + } + ], + "containers": [ + { + "name": "my-nginx", + "image": "nginx", + "ports": [ + { + "containerPort": 80, + "protocol": "TCP" + } + ], + "resources": {}, + "volumeMounts": [ + { + "name": "default-token-wpfjn", + "readOnly": true, + "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount" + } + ], + "terminationMessagePath": "/dev/termination-log", + "imagePullPolicy": "Always" + } + ], + "restartPolicy": "Always", + "terminationGracePeriodSeconds": 30, + "dnsPolicy": "ClusterFirst", + "serviceAccountName": "default", + "serviceAccount": "default", + "nodeName": "10.240.0.4", + "securityContext": {} + }, + "status": { + "phase": "Pending", + "conditions": [ + { + "type": "Ready", + "status": "False", + "lastProbeTime": null, + "lastTransitionTime": "2016-02-26T06:04:43Z", + "reason": "ContainersNotReady", + "message": "containers with unready status: [my-nginx]" + } + ], + "hostIP": "10.240.0.4", + "startTime": "2016-02-26T06:04:43Z", + "containerStatuses": [ + { + "name": "my-nginx", + "state": { + "waiting": { + "reason": "ContainerCreating", + "message": "Image: nginx is ready, container is creating" + } + }, + "lastState": {}, + "ready": false, + "restartCount": 0, + "image": "nginx", + "imageID": "" + } + ] + } + } + } + +DELETED ++++++++ + +:: + + { + "type": "DELETED", + "object": { + "kind": "Pod", + "apiVersion": "v1", + "metadata": { + "name": "my-nginx-y67ky", + "generateName": "my-nginx-", + "namespace": "default", + "selfLink": "/api/v1/namespaces/default/pods/my-nginx-y67ky", + "uid": "d42b0bb2-dc4e-11e5-8c79-42010af00003", + "resourceVersion": "63431", + "creationTimestamp": "2016-02-26T06:04:42Z", + "deletionTimestamp": "2016-02-26T06:05:46Z", + "deletionGracePeriodSeconds": 0, + "labels": { + "run": "my-nginx" + }, + "annotations": { + "kubernetes.io/created-by": { + "kind": "SerializedReference", + "apiVersion": "v1", + "reference": { + "kind": "ReplicationController", + "namespace": "default", + "name": "my-nginx", + "uid": "d42a4ee1-dc4e-11e5-8c79-42010af00003", + "apiVersion": "v1", + "resourceVersion": "63348" + } + } + } + }, + "spec": { + "volumes": [ + { + "name": "default-token-wpfjn", + "secret": { + "secretName": "default-token-wpfjn" + } + } + ], + "containers": [ + { + "name": "my-nginx", + "image": "nginx", + "ports": [ + { + "containerPort": 80, + "protocol": "TCP" + } + ], + "resources": {}, + "volumeMounts": [ + { + "name": "default-token-wpfjn", + "readOnly": true, + "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount" + } + ], + "terminationMessagePath": "/dev/termination-log", + "imagePullPolicy": "Always" + } + ], + "restartPolicy": "Always", + "terminationGracePeriodSeconds": 30, + "dnsPolicy": "ClusterFirst", + "serviceAccountName": "default", + "serviceAccount": "default", + "nodeName": "10.240.0.4", + "securityContext": {} + }, + "status": { + "phase": "Pending", + "conditions": [ + { + "type": "Ready", + "status": "False", + "lastProbeTime": null, + "lastTransitionTime": "2016-02-26T06:04:43Z", + "reason": "ContainersNotReady", + "message": "containers with unready status: [my-nginx]" + } + ], + "hostIP": "10.240.0.4", + "startTime": "2016-02-26T06:04:43Z", + "containerStatuses": [ + { + "name": "my-nginx", + "state": { + "waiting": { + "reason": "ContainerCreating", + "message": "Image: nginx is ready, container is creating" + } + }, + "lastState": {}, + "ready": false, + "restartCount": 0, + "image": "nginx", + "imageID": "" + } + ] + } + } + } + +Service +~~~~~~~ + +:: + + /api/v1/services?watch=true + +ADDED ++++++ + +:: + + { + "type": "ADDED", + "object": { + "kind": "Service", + "apiVersion": "v1", + "metadata": { + "name": "redis-master", + "namespace": "default", + "selfLink": "/api/v1/namespaces/default/services/redis-master", + "uid": "7aecfdac-d54c-11e5-8cc5-42010af00002", + "resourceVersion": "2074", + "creationTimestamp": "2016-02-17T08:00:16Z", + "labels": { + "app": "redis", + "role": "master", + "tier": "backend" + } + }, + "spec": { + "ports": [ + { + "protocol": "TCP", + "port": 6379, + "targetPort": 6379 + } + ], + "selector": { + "app": "redis", + "role": "master", + "tier": "backend" + }, + "clusterIP": "10.0.0.102", + "type": "ClusterIP", + "sessionAffinity": "None" + }, + "status": { + "loadBalancer": {} + } + } + } + +MODIFIED +++++++++ + +The event could not be observed. + +DELETED ++++++++ + +:: + + { + "type": "DELETED", + "object": { + "kind": "Service", + "apiVersion": "v1", + "metadata": { + "name": "redis-master", + "namespace": "default", + "selfLink": "/api/v1/namespaces/default/services/redis-master", + "uid": "7aecfdac-d54c-11e5-8cc5-42010af00002", + "resourceVersion": "2806", + "creationTimestamp": "2016-02-17T08:00:16Z", + "labels": { + "app": "redis", + "role": "master", + "tier": "backend" + } + }, + "spec": { + "ports": [ + { + "protocol": "TCP", + "port": 6379, + "targetPort": 6379 + } + ], + "selector": { + "app": "redis", + "role": "master", + "tier": "backend" + }, + "clusterIP": "10.0.0.102", + "type": "ClusterIP", + "sessionAffinity": "None" + }, + "status": { + "loadBalancer": {} + } + } + } + + +.. _`Kubernetes API`: http://kubernetes.io/docs/api/ +.. _CNI: https://github.com/appc/cni +.. _`in the deployment phase`: https://github.com/kubernetes/kubernetes/search?utf8=%E2%9C%93&q=FLANNEL_NET +.. _kube-proxy: http://kubernetes.io/docs/user-guide/services/#virtual-ips-and-service-proxies +.. _eventlet: http://eventlet.net/ +.. _Twisted: https://twistedmatrix.com/trac/ +.. _Tornado: http://tornadoweb.org/ +.. _gevent: http://www.gevent.org/ +.. _`What's wrong with eventlet?`: https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio#What.27s_wrong_with_eventlet.3F +.. _asyncio: https://www.python.org/dev/peps/pep-3156/ +.. _Trollius: http://trollius.readthedocs.org/ +.. _`Trollius documentation`: http://trollius.readthedocs.org/deprecated.html