airship
/
specs
Code Issues Proposed changes

[WIP] Add AirshipCTL Spec 

This spec details the command-line interface, behavior, and design
of the Airship 2.0 target-state `airshipctl` tool.

Change-Id: I43b690c7c43ddc4f87c0d205f2209cd169a1c0e6
Co-Authored-By: Rodolfo Pacheco <rp2723@att.com>
This commit is contained in:
Matt McEuen 2019-05-17 09:19:40 -05:00 committed by Rodolfo Pacheco
parent deea90ad1c
commit 3c21e12b21
3 changed files with 499 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
images/airship_2_0_airshipcli_and_clusterctl.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 265 KiB

BIN
images/airshipctl.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 106 KiB

499
specs/approved/airshipctl.rst Normal file
View File

@ -0,0 +1,499 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
single: airshipctl
single: Command Line
====================================================
Airship CTL - Airship 2.0 Command Line Interface
====================================================
*Airshipctl* will be a command-line interface for driving deployments and upgrades
of Airship-based infrastrucure, and it will be the primary user interface for
the Airship 2.0 platform. This spec details the interface, behavior, and
high-level design of the *airshipctl* tool.
Links
=====
The work to author and implement this spec will be tracked under this Jira
Problem description
===================
The main goal of Airship 2.0 is to:
- Allow for smaller deployments by introducing an Ephemeral Airship control plane.
- Further the adoption of entrenched projects.
- Continue vanishing complexity.
A big part of achieving the goals for Airship 2.0 is the inception of a new
component called ``*airshipctl*``.
This spec defines what this component needs to support.
Impacted components
===================
The existing Airship components are not directly impacted by *airshipctl*, however
some of the functionality introduced via
the aitshipctl will replace, abstract or encapsulate the purpose of existing tools.
#. Pegleg: Will become a plugin driven by the #airshipctl# document features.
#. Deckhand: Functionality such as Document Revision will now be achieved via
an Airshipctl document function.
#. Shipyard: The front door of Airship will not be the #airshipctl#
Proposed change
===============
Airshipctl Architecture
-----------------------
The heart of the Airship 2.0 platform is the *airshipctl* command line interface.
It places an emphasis on a thick client that is effectively able to speak
to k8s in remote sites and natively understands argo workflows
to drive cluster life cycle management. This is in contrast to Airship 1.0 which
leveraged a Shipyard API in the remote site, which was a long-lived service.
The goals for this pivot are to reduce the underlying infrastructure required to
support updates to an existing site and reduce the number of Airship specific YAML
documents required to produce and manage a site, vastly simplifying the overall design.
This utility is a net-new go module that produces two binaries: *airshipctl*
and airshipui. Both of these utilities operate on a kubernetes cluster
security context, and understand how to interpret and generate a
skeleton Airship document set. The *airshipctl* utility is the main entrypoint
for bootstrapping a cluster, collecting and pushing documents,
and managing workflows.
All functionality provided by the *airshipctl* will support a framework for
supported plugins. Every feature provided by *airshipctl*
will need to be implemented by providing explicit plugins.
Framework
---------
Define and create the framework for the go plugin that will produce *airshipctl*.
This includes defining the plugin framework to ensure it is extensible, logging,
vendoring, and basic CI/CD so the project can gain momentum.
The goal is that all known initial subcommands, e.g. bootstrap, document, and
so on will be plugins themselves, ensuring it supports extensibility day one.
Specifically the framework needs to define:
- Well-defined mechanism to consume and use plugins.
- Guidlines on expectations from plugins.
- Transactional information: Mechanism to identify a request initiated by *airshipctl* that allows for correlation between CTL and UI, and/or logging.
Configuration Context
---------------------
Both the CLI and the UI will need to be able to leverage a standard k8s
security context for interacting with remote sites. It should also understand
Airship site contexts so that it can be pointed at a local document set and
work with that during subsequent commands without needing
to be fed the same document set with every command similar to interacting
with the same Kubernetes cluster throughout a kubectl session.
The initial proposed list of commands that will be implemented is defined below:
Config
------
*Airshipctl* will be able to interact with multiple clusters, and it will follow kubectl's principles on
how to configure access to multiple clusters by using configuration files.
A configuration file describes yaml, location, clusters, users, and contexts.
Modify *airshipctl* configuration files using subcommands like
``*airshipctl* config set current-context my-context``.
In general this command allows us to modify the configuration of
airchipctl in a similar fashion as kubectl context management.
*Airshipctl* will have an airshipconfig file that can be manually managed or can be updated via the config options.
The loading order follows these rules:
The *--airshipconfig* flag will allow a user to specify a non default config file.
If the *--airshipconfig* flag is set, then only that file is loaded.
The flag may only be set once and no merging takes place.
If $AIRSHIPCONFIG environment variable is set, then it is used as
a list of paths (normal path delimiting rules for your system).
These paths are merged. When a value is modified, it is modified in the file
that defines the stanza. When a value is created, it is created in the first
file that exists. If no files in the chain exist, then it creates the last
file in the list.
Otherwise, ``${HOME}/.airship/config`` is used and no merging takes place.
Like kubectl it should support managing clusters, contexts and users.
**CLUSTER**
- ``*airshipctl* *config set-cluster*``
+----------------------+----------------------------------------------------------------------------------------+
| NAME | USAGE |
+======================+========================================================================================+
| set-cluster | Provide cluster name or label, as well as --server and --certificate-authority, or |
| | and --insecure-skip-tls-verify |
+----------------------+----------------------------------------------------------------------------------------+
**CONTEXT**
- ``*airshipctl* *config current-context*``
- ``*airshipctl* *config delete-context*``
- ``*airshipctl* *config get-contexts*``
- ``*airshipctl* *config set-context*``
- ``*airshipctl* *config use-context*``
+----------------------+----------------------------------------------------------------------------------------+
| NAME | USAGE |
+======================+========================================================================================+
| current-context | |
+----------------------+----------------------------------------------------------------------------------------+
| delete-context | |
+----------------------+----------------------------------------------------------------------------------------+
| get-contexts | |
+----------------------+----------------------------------------------------------------------------------------+
| set-context | |
+----------------------+----------------------------------------------------------------------------------------+
| use-context | |
+----------------------+----------------------------------------------------------------------------------------+
**USER**
- ``*airshipctl* *config set-credentials*``
+----------------------+----------------------------------------------------------------------------------------+
| NAME | USAGE |
+==========+====================================================================================================+
| set-credentials | Provide user --username and --password or --clent-certificate and --client-key |
+----------------------+----------------------------------------------------------------------------------------+
**YAML REPOSITORY**
- ``*airshipctl* *config set-repository*``
- ``*airshipctl* *config set-clone-path*``
- ``*airshipctl* *config set-extra-repository*``
- ``*airshipctl* *config set-repo-username*``
- ``*airshipctl* *config set-repo-key*``
+----------------------+----------------------------------------------------------------------------------------+
| NAME | USAGE |
+======================+========================================================================================+
| set-repository | Path or URL to the primary repository (containing site_definition.yaml) repo. |
+----------------------+----------------------------------------------------------------------------------------+
| set-clone-path | The path where the repo will be cloned. |
+----------------------+----------------------------------------------------------------------------------------+
| set-extra-repository | Path or URL of additional repositories. These should be named per the site-definition |
| | file, e.g. -e global=/opt/global -e secrets=/opt/secrets. |
+----------------------+----------------------------------------------------------------------------------------+
| set-repo-username | The SSH username to use when cloning remote authenticated repositories specified in |
| | site-definition file. |
+----------------------+----------------------------------------------------------------------------------------+
| set-repo-key | The SSH public key to use when cloning remote authenticated repositories. |
+----------------------+----------------------------------------------------------------------------------------+
This will allow us to specify a single local or remote repositories for the location of the yaml files.
Boostrap
---------
The bootstrap plugin will drive the lifecycle required to:
- Stand up a bootsrap cluster if one doesn't exist.
- Initialize a target remote cluster, by configuring at least the
initial server for that cluster.
- The plugin will leverage the intentions for the site provided
by the yaml declarations, including BareMetalHost CRs.
- ``*airshipctl* **boostrap**``
.. note::
*I think we dont need sitename as an argument, it can be implicit
by the setting of the context, unless it refers to the label or
structure of the documents only*
The lifecycle driven by the bootstrap command is depicted by
the flow in the image below:
.. image:: ../../images/airship_2_0_airshipcli_and_clusterctl.png
:width: 600
The goal is for Airship bootstrap to function with Metal3-io, Azure, Openstack,
and the generic-ssh providers to support baremetal, cloud, Openstack vms,
and bring your own infrastructure use cases day one.
Document
--------
Documents are the life-blood of Airship. Airship documents, which are a
collection of metal3-io, machine, cluster-api, armada, and argo documents,
need to be collected, pushed to sites and validated.
This plugin would be responsible for handling all of that client-side.
Including document revisions, document as secrets, etc.
The goal here is also that the document plugin--for flexibility--should
support pushing a subset of document sets.
For instance, much like we have update_site and deploy_site today, which
basically indicate whether baremetal documents are submitted or not, the
same should be true of Airship 2.0 but with more flexibility.
- ``*airshipctl* **document**``
.. note::
*I think we dont need sitename as an argument, it can be implicit by
the setting of the context, unless it refers to the label or structure
of the documents only*
**OPTIONS**
+----------+---------------------------------------------------------------------------------+
| NAME | USAGE |
+==========+=================================================================================+
| revision | Provides options to manage revision of manifests |
+----------+---------------------------------------------------------------------------------+
| secret | Provides a mechanism to manage manifest as secret |
+----------+---------------------------------------------------------------------------------+
| validate | Provides options for yaml validation |
+----------+---------------------------------------------------------------------------------+
| init | Create a structure for the yaml documents |
+----------+---------------------------------------------------------------------------------+
| apply | Label document set for specific purpose. Will drive specific target deliveries |
+----------+---------------------------------------------------------------------------------+
- ``*airshipctl* document **secret**``
In Airhsip 2.0 all the manifest document are stored in the cluster kubernetes etcd store. Documents are either CRD's or
Armada YAML . All YAML documents that are not CRD's will be encapsulated as a kubernetes secret documengt , to be consumed by Armada.
.. note::
*IF we have Armada Operator completed as part of Airship 2.0, then even those documents will be CRD's
in which case this command might only serve to trigger existing peleg functionality*
*airshipctl* will manage manifests as secrets locally and deliver them to the
appropriate context.
Should be able to be used in situations where it cannot deliver the secret.
This command would also server as the entry point for the
.. note::
*Should it care, should secrets be throw away / ephemeral
on the side of *airshipctl**
**SECRET OPTIONS**
+---------------------+-----------+-------------------------------------------+
| NAME | SHORTHAND | USAGE |
+=====================+===========+===========================================+
| get | g | Get a specific secret |
+---------------------+-----------+-------------------------------------------+
| create | c | Create the secret from the manifest |
+---------------------+-----------+-------------------------------------------+
| list | l | List all secret for the specific manifest |
+---------------------+-----------+-------------------------------------------+
| *delete* | d | Delete a specific secret for the manifest |
+---------------------+-----------+-------------------------------------------+
| generate-passphrase | | Command to generate a passphrase |
+---------------------+-----------+-------------------------------------------+
| rotate-passphrases | | Rotate passphrases |
+---------------------+-----------+-------------------------------------------+
| generate-pki | | Generate certificates |
+---------------------+-----------+-------------------------------------------+
| generate-passphrase | | Rotate certificates |
+---------------------+-----------+-------------------------------------------+
- ``*airshipctl* document **revision**``
.. note::
*Was is a revision, is it a name that by convention has some
timestamp and version, where the name has a relationship to the
secret document*
**REVISION FLAGS**
+----------+-----------+-----------+----------------------------+
| NAME | SHORTHAND | DEFAULT | USAGE |
+==========+===========+===========+============================+
| name | n | | Revision name? label |
+----------+-----------+-----------+----------------------------+
**REVISION OPTIONS**
+----------+-----------+----------------------------+
| NAME | SHORTHAND | USAGE |
+==========+===========+============================+
| get | g | Get a specific revision |
+----------+-----------+----------------------------+
| compare | d | Compare between revisions |
+----------+-----------+----------------------------+
| list | l | list all revisions |
+----------+-----------+----------------------------+
| *delete* | d | Delete a specific revision |
+----------+-----------+----------------------------+
.. note::
*A comment was made to support rollback. I dont think that is applicable here. We can disregard or simply apply the previous revision as needed.*
- ``*airshipctl* document **validate**``
*airshipctl* will be able to trigger validations locally prior to delivering documents to a site.
**VALIDATE OPTIONS**
+-----------+-----------+-----------------------------------------------------------------------------------------------+
| NAME | SHORTHAND | USAGE |
+===========+===========+===============================================================================================+
| lint | l | YAML syntax and schema validation |
+-----------+-----------+-----------------------------------------------------------------------------------------------+
| policy | p | All required policy documents are in-place, and existing documents conform to those policies. |
| | | E.g. if a 3rd party document specifies a layer that is not present in the layering policy, |
| | | that will cause this validation to report an error.. |
+-----------+-----------+-----------------------------------------------------------------------------------------------+
- ``*airshipctl* document **init**``
Initialize document structure and bootstrap contents. It creates
a skeleton Airship document to allow new users to
begin building the definition for a new site.
Executing this command will create the following directory structure:
..<Yaml Target Directory>/
global/
software/
charts/
config/
manifests/
secrets/
passphrases/
publickey/
site/
baremetal/
baremetalhosts/
machines/
.. note::
*More detail as to what files or CRs can be automatically generated
in order to be used as templates, or simple because the information
in them is generic*
Cluster
-------
The CLI should support integration with a cluster-registry.
Much of the same information is available in a cluster-api generated
Cluster CRD that a cluster registry should be populated with.
The communities themselves are looking to align on this and we should ensure
*airshipctl* supports the ultimate path.
- ``*airshipctl* cluster``
**CLUSTER OPTIONS**
+----------+-----------+--------------------------------------------------------+
| NAME | SHORTHAND | USAGE |
+==========+===========+========================================================+
| status | s | |
+----------+-----------+--------------------------------------------------------+
| state | t | State of the site, CRD and their verions. Argo version |
+----------+-----------+--------------------------------------------------------+
.. note::
*Cluster might not be the best name for this, could be named site instead*
Workflow
--------
- ``*airshipctl* workflow``
**WORKFLOW OPTIONS**
+--------------+-----------+--------------------------------------------------------------+
| NAME | SHORTHAND | USAGE |
+==============+===========+==============================================================+
| sitemanage | m | Predefined Airship workflow for deploy and uopdate of sites |
+--------------+-----------+--------------------------------------------------------------+
| redeployhost | h | Predefined Airship workflow for redeploying a host includes |
| | | taking care of whatever needs to preserve workloads when |
| | | appropriate |
+--------------+-----------+--------------------------------------------------------------+
| init | i | Install Argo CRD in contextual cluster |
+--------------+-----------+--------------------------------------------------------------+
| list | l | List workflows |
+--------------+-----------+--------------------------------------------------------------+
| delete | d | Delete a custom workflow |
+--------------+-----------+--------------------------------------------------------------+
| apply | a | Create a new custom workflow |
+--------------+-----------+--------------------------------------------------------------+
The definition of a site state is a collection of yaml. This predefined workflows will
be responsible for effecting the state depicted by this yaml. Whether this
is a 1st time deployment, or an expansion or simple software stack configuration
changes. This workflow will be responsible for delivering and executing the
intentions described by the YAML.
- ``*airshipctl* workflow **sitemanage**``
**SITEMANAGE OPTIONS**
+----------+-----------+---------------------------+
| NAME | SHORTHAND | USAGE |
+==========+===========+===========================+
| .. | .. | |
+----------+-----------+---------------------------+
This workflow essentially will encapsulate the logic required to prepare and
post configure a host after redeployment. In essence a host should be
redeployable by simple altering the BareMetalHost specification and the
associated MachineSpec definition. The reason for a specific workflow, is
that there could be tasks to be performed in preparation of the redeploy,
and after the redeploy occurs.
- ``*airshipctl* workflow **redeployhost**``
**SITEMANAGEREDEPLOYHOST OPTIONS**
+----------+-----------+---------------------------+
| NAME | SHORTHAND | USAGE |
+==========+===========+===========================+
| host | | |
+----------+-----------+---------------------------+
Implementation
==============
.. image:: ../../images/airshipctl.png
:width: 600
Dependencies
============
References
==========
.