diff --git a/doc/source/devref/libnetwork_remote_driver_design.rst b/doc/source/devref/libnetwork_remote_driver_design.rst index 82043ec1..cb1c835d 100644 --- a/doc/source/devref/libnetwork_remote_driver_design.rst +++ b/doc/source/devref/libnetwork_remote_driver_design.rst @@ -1,12 +1,12 @@ -=============================== -Libnetwork Remote Driver Design -================================ +======================================= +Libnetwork Remote Network Driver Design +======================================= What is Kuryr -------------- -Kuryr implements a `libnetwork remote driver`_ and maps its calls to OpenStack +Kuryr implements a `libnetwork remote network driver`_ and maps its calls to OpenStack `Neutron`_. It works as a translator between libnetwork's `Container Network Model`_ (CNM) and `Neutron's networking model`_. @@ -26,7 +26,7 @@ the host, e.g., Linux bridge, Open vSwitch datapath and so on. Kuryr Workflow - Host Networking --------------------------------- Kuryr resides in each host that runs Docker containers and serves `APIs`_ -required for the libnetwork remote driver. +required for the libnetwork remote network driver. It is planned to use the `Adding tags to resources`_ new Neutron feature by Kuryr, to map between Neutron resource Id's and Docker Id's (UUID's) @@ -37,7 +37,7 @@ to map between Neutron resource Id's and Docker Id's (UUID's) 2. libnetwork registers Kuryr as a remote driver -3. A user makes requests against libnetwork with the driver specifier for Kuryr +3. A user makes requests against libnetwork with the network driver specifier for Kuryr - i.e., ``--driver=kuryr`` or ``-d kuryr`` for the Docker CLI @@ -56,121 +56,98 @@ to map between Neutron resource Id's and Docker Id's (UUID's) - the key/value datastore is abstracted by `libkv`_ -Libnetwork User Workflow (with Kuryr as remove driver) - Host Networking -------------------------------------------------------------------------- +Libnetwork User Workflow (with Kuryr as remote network driver) - Host Networking +--------------------------------------------------------------------------------- 1. A user creates a network ``foo`` :: $ sudo docker network create --driver=kuryr foo - 51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1 + 286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364 This makes a HTTP POST call on ``/NetworkDriver.CreateNetwork`` with the following JSON data. :: { - "NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1", - "Options": { - ... - } + "NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364", + "IPv4Data": [{ + "Pool": "172.18.0.0/16", + "Gateway": "172.18.0.1/16", + "AddressSpace": "" + }], + "IPv6Data": [], + "Options": { "com.docker.network.generic": {}} } - The Kuryr remote driver will then generate a Neutron API request to create an underlying Neutron network. - When the Neutron network has been created, the Kuryr remote driver will generate an empty success response + The Kuryr remote network driver will then generate a Neutron API request to create an underlying Neutron network. + When the Neutron network has been created, the Kuryr remote network driver will generate an empty success response to the docker daemon. Kuryr tags the Neutron network with the NetworkID from docker. -2. A user creates a service ``bar`` against network ``foo`` +2. A user launches a container against network ``foo`` :: - $ sudo docker service publish bar.foo - 98953db3f8e6628caf4a7cad3c866cb090654e3dee3e37206ad8c0a81355f1b7 + $ sudo docker run --net=foo -itd --name=container1 busybox + 78c0458ba00f836f609113dd369b5769527f55bb62b5680d03aa1329eb416703 This makes a HTTP POST call on ``/NetworkDriver.CreateEndpoint`` with the - following JSON data. + following JSON format. :: - { - "NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1", - "EndpointID": "98953db3f8e6628caf4a7cad3c866cb090654e3dee3e37206ad8c0a81355f1b7", - "Interfaces": [ - ... - ], - "Options": { - ... - } - } + { + "NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364", + "Interface": { + "AddressIPv6": "", + "MacAddress": "", + "Address": "172.18.0.2/16" + }, + "Options": { + "com.docker.network.endpoint.exposedports": [], + "com.docker.network.portmap": [] + }, + "EndpointID": "edb23d36d77336d780fe25cdb5cf0411e5edd91b0777982b4b28ad125e28a4dd" + } - The Kuryr remote driver then generate a Neutron API request to create a Neutron - subnet and a port with the matching fields for all interfaces in the request. + The Kuryr remote network driver then generate a Neutron API request to create a Neutron + subnet and a port with the matching fields for interface in the request. Kuryr needs to create the subnet dynamically as it has no information on - the interfaces list IP's. + the interface IP. - the following steps are taken: + Following steps are taken: 1) On the endpoint creation Kuryr examine if there's a subnet which CIDR corresponds to Address or AddressIPv6 requested. 2) If there's a subnet, Kuryr tries to reuse it without creating a new subnet. otherwise it create a new one with the given CIDR - 3) If a CIDR is not passed, Kuryr creates a default IPv4 or IPv6 subnets from a - specific subnet pool. - more information can be found in Kuryr `IPAM blueprint`_ - 4) Kuryr creates a port assigning the IP address to it and associating the port with - the subnet based on it's already allocated in 2. + 3) Kuryr creates a port assigning the IP address to it and associating the port with + the subnet based on what it has already allocated in 2. + 4) Kuryr tags the Neutron subnet and port with EndpointID. On the subnet creation described in (2) and (3) above, Kuryr tries to grab the allocation pool greedily by not specifying ``allocation_pool``. Without ``allocation_pool``, Neutron allocates all IP addresses in the range of the subnet CIDR as described in `Neutron's API reference`_. - When the Neutron port has been created, the Kuryr remote driver will generate a response to the - docker daemon indicating the port's IPv4, IPv6, and Mac addresses as follows. + When the Neutron port has been created, the Kuryr remote driver will generate an empty response to the + docker daemon indicating the SUCCESS. {} + (https://github.com/docker/libnetwork/blob/master/docs/remote.md#create-endpoint) :: { - "Interfaces": [{ - "ID": , - "Address": , - "AddressIPv6": , - "MacAddress": - }, ...] + "Interface": {"MacAddress": "08:22:e0:a8:7d:db"} } - Kuryr tags the Neutron subnet and port with Docker Interface id. -3. A user shows information of the service + On receiving success response, libnetwork makes a HTTP POST call on ``/NetworkDriver.Join`` with + the following JOSN data. :: - $ sudo docker service info test.bar - Service Id: db0524fa27184de3dfe274908e77e05155e12e20269c782984468b251fe507d7 - Name: bar - Network: foo - -4. A user attaches a container to the service - :: - - $ CID=$(sudo docker run -itd busybox) - $ sudo docker service attach $CID bar.foo - - or if a network interface needs to be attached to the container before its - launch, - :: - - $ sudo docker run --publish-service=bar.foo -itd busybox - 12bbda391ed0728787b2c2131f091f6d5744806b538b9314c15e789e5a1ba047 - - This makes a HTTP POST call on ``/NetworkDriver.Join`` with the following - JOSN data. - :: - - { - "NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1", - "EndpointID": "98953db3f8e6628caf4a7cad3c866cb090654e3dee3e37206ad8c0a81355f1b7", - "SandboxKey": "/var/run/docker/netns/12bbda391ed0", - "Options": { - ... - } - } + { + "NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364", + "SandboxKey": "/var/run/docker/netns/052b9aa6e9cd", + "Options": null, + "EndpointID": "edb23d36d77336d780fe25cdb5cf0411e5edd91b0777982b4b28ad125e28a4dd" + } Kuryr connects the container to the corresponding neutron network by doing the following steps: @@ -180,42 +157,99 @@ Libnetwork User Workflow (with Kuryr as remove driver) - Host Networking 3) Perform a neutron-port-type-dependent VIF-binding to the corresponding Neutron port using the VIF binding layer and depending on the specific port type. - After the VIF-binding is completed, the Neutron remote driver generate a response to the Docker + After the VIF-binding is completed, the Kuryr remote network driver generates a response to the Docker daemon as specified in the libnetwork documentation for a join request. (https://github.com/docker/libnetwork/blob/master/docs/remote.md#join) -5. A user detaches the container from the service +3. A user requests information about the network :: - $ sudo docker service detach $CID bar.foo + $ sudo docker network inspect foo + { + "name": "foo", + "id": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364", + "scope": "local", + "driver": "kuryr", + "ipam": { + "driver": "default", + "config": [ + {} + ] + }, + "containers": { + "78c0458ba00f836f609113dd369b5769527f55bb62b5680d03aa1329eb416703": { + "endpoint": "edb23d36d77336d780fe25cdb5cf0411e5edd91b0777982b4b28ad125e28a4dd", + "mac_address": "02:42:c0:a8:7b:cb", + "ipv4_address": "172.18.0.2/24", + "ipv6_address": "" + } + } + } + + +4. A user connects one more container to the network + :: + + $ sudo docker network connect foo container2 + d7fcc280916a8b771d2375688b700b036519d92ba2989622627e641bdde6e646 + + $ sudo docker network inspect foo + { + "name": "foo", + "id": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364", + "scope": "local", + "driver": "kuryr", + "ipam": { + "driver": "default", + "config": [ + {} + ] + }, + "containers": { + "78c0458ba00f836f609113dd369b5769527f55bb62b5680d03aa1329eb416703": { + "endpoint": "edb23d36d77336d780fe25cdb5cf0411e5edd91b0777982b4b28ad125e28a4dd", + "mac_address": "02:42:c0:a8:7b:cb", + "ipv4_address": "172.18.0.2/24", + "ipv6_address": "" + }, + "d7fcc280916a8b771d2375688b700b036519d92ba2989622627e641bdde6e646": { + "endpoint": "a55976bafaad19f2d455c4516fd3450d3c52d9996a98beb4696dc435a63417fc", + "mac_address": "02:42:c0:a8:7b:cc", + "ipv4_address": "172.18.0.3/24", + "ipv6_address": "" + } + } + } + + +5. A user disconnects a container from the network + :: + + $ CID=d7fcc280916a8b771d2375688b700b036519d92ba2989622627e641bdde6e646 + $ sudo docker network disconnet foo $CID This makes a HTTP POST call on ``/NetworkDriver.Leave`` with the following JSON data. :: { - "NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1", - "EndpointID": "98953db3f8e6628caf4a7cad3c866cb090654e3dee3e37206ad8c0a81355f1b7" + "NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364", + "EndpointID": "a55976bafaad19f2d455c4516fd3450d3c52d9996a98beb4696dc435a63417fc" } - Kuryr remote driver will remove the VIF binding between the container and the Neutron port, + Kuryr remote network driver will remove the VIF binding between the container and the Neutron port, and generate an empty response to the Docker daemon. -6. A user unpublishes the service - :: - - $ sudo docker unpublish bar.foo - - This makes a HTTP POST call on ``/NetworkDriver.DeleteEndpoint`` with the + Then libnetwork makes a HTTP POST call on ``/NetworkDriver.DeleteEndpoint`` with the following JSON data. :: { - "NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1", - "EndpointID": "98953db3f8e6628caf4a7cad3c866cb090654e3dee3e37206ad8c0a81355f1b7" + "NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364", + "EndpointID": "a55976bafaad19f2d455c4516fd3450d3c52d9996a98beb4696dc435a63417fc" } - Kuryr remote driver generate a Neutron API request to delete the associated Neutron port, + Kuryr remote network driver generates a Neutron API request to delete the associated Neutron port, in case the relevant port subnet is empty, Kuryr also deletes the subnet object using Neutron API and generate an empty response to the Docker daemon: {} @@ -229,18 +263,13 @@ Libnetwork User Workflow (with Kuryr as remove driver) - Host Networking :: { - "NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1" + "NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364" } - Kuryr remote driver generate a Neutron API request to delete the corresponding Neutron network. - When the Neutron network has been deleted, the Kuryr remote driver generate an empty response + Kuryr remote network driver generates a Neutron API request to delete the corresponding Neutron network. + When the Neutron network has been deleted, the Kuryr remote network driver generate an empty response to the docker daemon: {} -The workflows described in 2., 4., 5. and 6. can be done in the following -single command.:: - - $ sudo docker run --publish-service=cont.implicit.foo -itd busybox - Mapping between the CNM and the Neutron's Networking Model ------------------------------------------------------------ @@ -254,7 +283,7 @@ libnetwork Neutron ===================== ====================== Network Network Sandbox Subnet, Port and netns -Endpoint Subnet, Port +Endpoint Port ===================== ====================== libnetwork's Sandbox and Endpoint can be mapped into Neutron's Subnet and Port, @@ -263,7 +292,8 @@ visible and editable resource entity attachable to containers from users' perspective. Sandbox manages information exposed by Endpoint behind the scene automatically. -.. _libnetwork remote driver: https://github.com/docker/libnetwork/blob/master/docs/remote.md +.. _libnetwork remote network driver: https://github.com/docker/libnetwork/blob/master/docs/remote.md +.. _libnetwork IPAM driver: https://github.com/docker/libnetwork/blob/master/docs/ipam.md .. _Neutron: https://wiki.openstack.org/wiki/Neutron .. _Container Network Model: https://github.com/docker/libnetwork/blob/master/docs/design.md#the-container-network-model .. _Neutron's networking model: https://wiki.openstack.org/wiki/Neutron/APIv2-specification