From d1dee1e9a36a432825edebcf004094902cda2340 Mon Sep 17 00:00:00 2001 From: Thomas Hartland Date: Wed, 11 Mar 2020 11:39:22 +0100 Subject: [PATCH] Add node groups documentation Add documentation with examples for the life cycle of creating/resizing/deleting a node group. Change-Id: Ie34ebef6b46fa6223119696671cd1ac23d4ff855 --- doc/source/user/index.rst | 6 ++ doc/source/user/node-groups.rst | 147 ++++++++++++++++++++++++++++++++ 2 files changed, 153 insertions(+) create mode 100644 doc/source/user/node-groups.rst diff --git a/doc/source/user/index.rst b/doc/source/user/index.rst index 4924e5240e..b307ede220 100644 --- a/doc/source/user/index.rst +++ b/doc/source/user/index.rst @@ -36,6 +36,7 @@ created and managed by Magnum to support the COE's. #. `Kubernetes External Load Balancer`_ #. `Rolling Upgrade`_ #. `Keystone Authentication and Authorization for Kubernetes`_ +#. `Node Groups`_ Overview ======== @@ -3411,3 +3412,8 @@ Keystone Authentication and Authorization for Kubernetes ======================================================== .. include:: k8s-keystone-authN-authZ.rst + +Node Groups +=========== + +.. include:: node-groups.rst diff --git a/doc/source/user/node-groups.rst b/doc/source/user/node-groups.rst new file mode 100644 index 0000000000..4fcd37362e --- /dev/null +++ b/doc/source/user/node-groups.rst @@ -0,0 +1,147 @@ +Node groups can be used to create heterogeneous clusters. + +This functionality is only supported for Kubernetes clusters. + +When a cluster is created it already has two node groups, +``default-master`` and ``default-worker``. + +:: + + $ openstack coe cluster list + +--------------------------------------+------+-----------+------------+--------------+-----------------+---------------+ + | uuid | name | keypair | node_count | master_count | status | health_status | + +--------------------------------------+------+-----------+------------+--------------+-----------------+---------------+ + | ef7011bb-d404-4198-a145-e8808204cde3 | kube | default | 1 | 1 | CREATE_COMPLETE | HEALTHY | + +--------------------------------------+------+-----------+------------+--------------+-----------------+---------------+ + + $ openstack coe nodegroup list kube + +--------------------------------------+----------------+-----------+----------------------------------------+------------+-----------------+--------+ + | uuid | name | flavor_id | image_id | node_count | status | role | + +--------------------------------------+----------------+-----------+----------------------------------------+------------+-----------------+--------+ + | adc3ecfa-d11e-4da7-8c44-4092ea9dddd9 | default-master | m1.small | Fedora-AtomicHost-29-20190820.0.x86_64 | 1 | CREATE_COMPLETE | master | + | 186e131f-8103-4285-a900-eb0dcf18a670 | default-worker | m1.small | Fedora-AtomicHost-29-20190820.0.x86_64 | 1 | CREATE_COMPLETE | worker | + +--------------------------------------+----------------+-----------+----------------------------------------+------------+-----------------+--------+ + +The ``default-worker`` node group cannot be removed or reconfigured, so +the initial cluster configuration should take this into account. + +Create a new node group +----------------------- + +To add a new node group, use ``openstack coe nodegroup create``. The +only required parameters are the cluster ID and the name for the new +node group, but several extra options are available. + +Roles ++++++ + +Roles can be used to show the purpose of a node group, and multiple node +groups can be given the same role if they share a common purpose. + +:: + + $ openstack coe nodegroup create kube test-ng --node-count 1 --role test + +When listing node groups, the role may be used as a filter: + +:: + + $ openstack coe nodegroup list kube --role test + +--------------------------------------+---------+-----------+----------------------------------------+------------+--------------------+------+ + | uuid | name | flavor_id | image_id | node_count | status | role | + +--------------------------------------+---------+-----------+----------------------------------------+------------+--------------------+------+ + | b4ab1fcb-f23a-4d1f-b583-d699a2f1e2d7 | test-ng | m1.small | Fedora-AtomicHost-29-20190820.0.x86_64 | 1 | CREATE_IN_PROGRESS | test | + +--------------------------------------+---------+-----------+----------------------------------------+------------+--------------------+------+ + +The node group role will default to “worker” if unset, and the only +reserved role is “master”. + +Role information is available within Kubernetes as labels on the nodes. + +:: + + $ kubectl get nodes -L magnum.openstack.org/role + NAME STATUS AGE VERSION ROLE + kube-r6cyw4bjb4lr-master-0 Ready 5d5h v1.16.0 master + kube-r6cyw4bjb4lr-node-0 Ready 5d5h v1.16.0 worker + kube-test-ng-lg7bkvjgus4y-node-0 Ready 61s v1.16.0 test + +This information can be used for scheduling, using a `node +selector `__. + +.. code:: yaml + + nodeSelector: + magnum.openstack.org/role: test + +The label ``magnum.openstack.org/nodegroup`` is also available for +selecting a specific node group. + +Flavor +++++++ + +The node group flavor will default to the minion flavor given when +creating the cluster, but can be changed for each new node group. + +:: + + $ openstack coe nodegroup create ef7011bb-d404-4198-a145-e8808204cde3 large-ng --flavor m2.large + +This can be used if you require nodes of different sizes in the same +cluster, or to switch from one flavor to another by creating a new node +group and deleting the old one. + +Availability zone ++++++++++++++++++ + +To create clusters which span more than one availability zone, multiple +node groups must be used. The availability zone is passed as a label to +the node group. + +:: + + $ openstack coe nodegroup create kube zone-a --labels availability_zone=zone-a --labels ... + $ openstack coe nodegroup create kube zone-b --labels availability_zone=zone-b --labels ... + $ openstack coe nodegroup create kube zone-c --labels availability_zone=zone-c --labels ... + +Where ``--labels ...`` are the rest of the labels that the cluster was +created with, which can be obtained from the cluster with this script: + +:: + + $ openstack coe cluster show -f json | + jq --raw-output '.labels | to_entries | + map("--labels \(.key)=\"\(.value)\"") | join(" ")' + +Zone information is available within the cluster as the label +``topology.kubernetes.io/zone`` on each node, or as the now deprecated +label ``failure-domain.beta.kubernetes.io/zone``. + +From Kubernetes 1.16 and onwards it is possible to `balance the number +of pods in a deployment across availability +zones `__ +(or any other label). + +Resize +------ + +Resizing a node group is done with the same API as resizing a cluster, +but the ``--nodegroup`` parameter must be used. + +:: + + $ openstack coe cluster resize kube --nodegroup default-worker 2 + Request to resize cluster ef7011bb-d404-4198-a145-e8808204cde3 has been accepted. + +As usual the ``--nodes-to-remove`` parameter may be used to remove +specific nodes when decreasing the size of a node group. + +Delete +------ + +Any node group except the default master and worker node groups can be +deleted, by specifying the cluster and nodegroup name or ID. + +:: + + $ openstack coe nodegroup delete ef7011bb-d404-4198-a145-e8808204cde3 test-ng