Add update_labels spec

The update_labels spec outlines the workflow and associated changes
by which kubernetes node labels can be updated using Airship.

Change-Id: Idac6237aeba92d7a852031863c360a82b0fff9a5
Bryan Strassner 5 years ago
parent 6104cac943
commit 1636f898eb

@ -0,0 +1,244 @@
This work is licensed under a Creative Commons Attribution 3.0 Unported
.. index::
single: Kubernetes node labels
single: workflow
single: Promenade
single: Shipyard
single: Drydock
Airship workflow to update Kubernetes node labels
Proposal to enhance Airship to support updating `Kubernetes node labels`_ as a
triggered workflow using Shipyard as an entrypoint, Deckhand as a document
repository, Drydock as the decision maker about application of node labels, and
Promenade as the interactive layer to Kubernetes_.
Problem description
Over the lifecycle of a deployed site, there is a need to maintain the labels
applied to Kubernetes nodes. Prior to this change the only Airship-supplied
mechanism for this was during a node's deployment. Effectively, the way to
change or remove labels from a deployed node is through a manual process.
Airship aims to eliminate or minimize manual action on a deploy site.
Without the ability to declaratively update the labels for a Kubernetes node,
the engineers responsible for a site lose finer-grained ability to adjust where
deployed software runs -- i.e. node affinity/anti-affinity. While the
software's Helm or Armada chart could be adjusted and the site updated, the
granularity of marking a single node with a label is still missed.
Impacted components
The following Airship components would be impacted by this solution:
#. Drydock - endpoint(s) to evaluate and trigger adding or removing labels on
a node
#. Promenade - endpoint(s) to add/remove labels on a node.
#. Shipyard - new workflow: update_labels
Proposed change
.. note::
External to Airship, the process requires updating the site definition
documents describing `Baremetal Nodes`_ to properly reflect the desired
labels for a node. The workflow proposed below does not allow for addition
or elimination of node labels without going through an update of the site
definition documents.
To achieve the goal of fine-grained declarative Kubernetes label management,
a new Shipyard action will be introduced: ``update_labels``. The update_labels
action will accept a list of targeted nodes as an action parameter. E.g.::
POST /v1.0/actions
"name" : "action name",
"parameters" : {
"target_nodes": [ "node1", "node2"]
The most recent committed configuration documents will be used to drive the
labels that result on the target nodes.
- If there is no committed version of the configuration documents, the action
will be rejected.
- If there are no revisions of the configuration documents marked as
participating in a site action, the action will be rejected.
A new workflow will be invoked for ``update_labels``, being passed the
configuration documents revision and the target nodes as input, using existing
parameter mechanisms.
.. note::
At the time of writing this blueprint, there are no other actions exposed by
Shipyard that are focused on a set of target nodes instead of the whole site.
This introduces a new category of ``targeted`` actions, as opposed to the
existing ``site`` actions. Targeted actions should not set the site action
labels (e.g. successful-site-action) on Deckhand revisions as part of the
The workflow will perform a standard validation of the configuration documents
by the involved components (Deckhand, Drydock, Promenade).
Within the Shipyard codebase, a new Drydock operator will be defined to invoke
and monitor the invocation of Drydock to trigger label updates. Using the
task interface of Drydock, a node filter containing the target nodes will be
used to limit the scope of the request to only those nodes, along with the
design reference. E.g.::
POST /v1.0/tasks
"action": "relabel_nodes",
"design_ref": "<deckhand_uri>",
"node_filter": {
"filter_set_type": "union",
"filter_set": [
"filter_type": "union",
"node_names": ["node1", "node2"],
"node_tags": [],
"node_labels": {},
"rack_names": [],
"rack_labels": {},
.. note::
Since a node filter is part of this interface, it would technically allow for
other ways to assign labels across nodes. However at this time, Shipyard will
only leverage the node_names field.
After invoking Drydock (see below), the workflow step will use the top level
Drydock task result, and disposition the step as failure if any nodes are
unsuccessful. This may result in a partial update. No rollbacks will be
Drydock's task API will provide a new action ``relabel_nodes``. This action will
perform necessary analysis of the design to determine the full set of labels
that should be applied to each node. Some labels are generated dynamically
during node deployment; these will need to be generated and included in the
set of node labels.
Since multiple nodes can be targeted, and success or failure may occur on each,
Drydock will track these as subtasks and roll up success/failure per node to
the top level task.
For each node, Drydock will invoke Promenade to apply the set of labels to the
Kubernetes node, using a ``PUT`` against the (new) ``node-labels/{node_name}``
endpoint. The payload of this request is a list of labels for that node. E.g.::
PUT /v1.0/node-labels/node1
Promenade will perform a difference of the existing node labels against the
requested node labels. Promenade will then in order:
#) apply new labels and change existing labels with new values
#) remove labels that are not in the request body
API Clients and CLIs
The Drydock, Promenade, and Shipyard API Clients and CLI components will need
to be updated to match the new functionality defined above.
Documentation impact
Each of the identified components have associated API (and CLI) documentation
that will be updated to match the new API endpoints and associated payload
formats as noted above.
Security impact
None - No new security impacts are introduced with this design. Existing
mechanisms will be applied to the changes introduced.
Performance impact
None - This workflow has no specific performance implications for the
components involved.
High level process
Shipyard Workflow Drydock Promenade
+---------------+ +-------------+
| Submit Action +------> | |
| update_labels | | |
| | |Drydock Task:| +------------------+
+---------------+ | relabel_node+-----> |Evaluate baremetal|
| | |definition; |
|Monitor Task +-----> |generate k8s node |
| | Poll |labels |
| | <-----+ |
| | |Promenade: | +-------------------+
| | | PUT node-labels +-------> |Diff existing node |
| | | (list of labels)| Wait | labels. |
| | | | <-------+ Add new labels |
| | +------------------+ | Remove orphaned |
| | | labels |
| | | |
| | +-------------------+
|End workflow |
| |
There are no specific milestones identified for this blueprint. is work that has started for
.. _Kubernetes:
.. _Kubernetes node labels:
.. _Baremetal Nodes: