diff --git a/docs/source/API.rst b/docs/source/API.rst index 5e697703..f52b9c93 100644 --- a/docs/source/API.rst +++ b/docs/source/API.rst @@ -28,8 +28,7 @@ functionality in Shipyard. Standards used by the API ------------------------- -See `UCP API -conventions `__ +See `API Conventions`_ Notes on examples ----------------- @@ -258,20 +257,19 @@ Entity Structure ^^^^^^^^^^^^^^^^ The response will be the list of validations from all downstream systems that perform validation during the commit process. The structure will match the -error response object described in the `UCP API -conventions `__ -and will be an aggregation of each UCP component’s responses. +error response object described in the `API Conventions`_ and will be an +aggregation of each validating component’s responses. POST /v1.0/commitconfigdocs ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Synchronous. Performs the commit of the Shipyard Buffer to the Committed -Documents. This invokes each of the UCP components to examine the Shipyard -Buffer version of the configuration documents and aggregate the responses. -While performing this commit, further POSTing of configdocs, or other commits -may not be invoked (Shipyard will block those requests with a 409 response). If -there are any failures to validate, the Shipyard Buffer and Committed Documents -will remain unchanged. If successful, the Shipyard Buffer will be cleared, and -the Committed documents will be updated. +Documents. This invokes each of the validating components to examine the +Shipyard Buffer version of the configuration documents and aggregate the +responses. While performing this commit, further POSTing of configdocs, or +other commits may not be invoked (Shipyard will block those requests with a 409 +response). If there are any failures to validate, the Shipyard Buffer and +Committed Documents will remain unchanged. If successful, the Shipyard Buffer +will be cleared, and the Committed documents will be updated. .. note:: @@ -1033,3 +1031,6 @@ Example [2018-04-11 07:30:43,892] {{cli.py:374}} INFO - Running on host airflow-worker-0.airflow-worker-discovery.ucp.svc.cluster.local [2018-04-11 07:30:43,945] {{base_task_runner.py:98}} INFO - Subtask: [2018-04-11 07:30:43,944] {{python_operator.py:90}} INFO - Done. Returned value was: None [2018-04-11 07:30:43,992] {{base_task_runner.py:98}} INFO - Subtask: """) + + +.. _API Conventions: https://airshipit.readthedocs.io/en/latest/api-conventions.html diff --git a/docs/source/API_action_commands.rst b/docs/source/API_action_commands.rst index f8616791..f8c03f42 100644 --- a/docs/source/API_action_commands.rst +++ b/docs/source/API_action_commands.rst @@ -34,12 +34,9 @@ configuration documents. Steps, conceptually: Prevents concurrent site modifications by conflicting actions/workflows. #. Preflight checks - Ensures all UCP components are in a responsive state. -#. Get design version - Uses Deckhand to discover the latest committed version of design for - the site. + Ensures all Airship components are in a responsive state. #. Validate design - Asks each involved UCP component to validate the design. This ensures + Asks each involved Airship component to validate the design. This ensures that the previously committed design is valid at the present time. #. Drydock build Orchestrates the Drydock component to configure hardware and the @@ -92,12 +89,11 @@ Schemas DeploymentConfiguration_ schema - Provides for validation of the deployment-configuration documents -Documents -~~~~~~~~~ -deployment-configuration - Allows for specification of configurable options -used by the site deployment related workflows, including the timeouts used for -various steps, and the name of the armada manifest that will be used during the -deployment/update. +Deployment Configuration +~~~~~~~~~~~~~~~~~~~~~~~~ +Allows for specification of configurable options used by the site deployment +related workflows, including the timeouts used for various steps, and the name +of the armada manifest that will be used during the deployment/update. A `sample deployment-configuration`_ shows a completely specified example. @@ -114,8 +110,8 @@ Supported values: - deployment_strategy: - Not currently used, placeholder for specifying how server deployments - should be grouped. (e.g. all-at-once, by-rack, by-label) + The name of the deployment strategy document to be used. There is a default + deployment strategy that is used if this field is not present. - deploy_interval: @@ -201,6 +197,20 @@ Supported values: The name of the Armada manifest document that the workflow will use during site deployment activities. e.g.:'full-site' -.. _DeploymentConfiguration: https://github.com/att-comdev/shipyard/blob/master/shipyard_airflow/schemas/deploymentConfiguration.yaml -.. _`sample deployment-configuration`: https://github.com/att-comdev/shipyard/blob/master/tests/unit/yaml_samples/deploymentConfiguration_full_valid.yaml -.. _`Default configuration values`: https://github.com/att-comdev/shipyard/blob/a88a5cf15a81de32707fee49d7a626908a8c8f38/shipyard_airflow/plugins/deployment_configuration_operator.py#L47 +Deployment Strategy +~~~~~~~~~~~~~~~~~~~ +The deployment strategy document is optionally specified in the Deployment +Configuration and provides a way to group, sequence, and test the deployments +of groups of hosts deployed using `Drydock`_. The `deployment strategy design`_ +provides details for the structures and usage of the deployment strategy. +A `sample deployment-strategy`_ shows one possible strategy, in the context of +the Shipyard unit testing. +The `DeploymentStrategy`_ schema is a more formal definition of this document. + +.. _`Default configuration values`: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/plugins/deployment_configuration_operator.py +.. _DeploymentConfiguration: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/schemas/deploymentConfiguration.yaml +.. _DeploymentStrategy: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/schemas/deploymentStrategy.yaml +.. _`deployment strategy design`: https://airshipit.readthedocs.io/en/latest/blueprints/deployment-grouping-baremetal.html +.. _Drydock: https://git.airshipit.org/cgit/airship-drydock +.. _`sample deployment-configuration`: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/tests/unit/yaml_samples/deploymentConfiguration_full_valid.yaml +.. _`sample deployment-strategy`: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/tests/unit/yaml_samples/deploymentStrategy_full_valid.yaml diff --git a/docs/source/client_user_guide.rst b/docs/source/client_user_guide.rst index 14b166b3..3fd28b98 100644 --- a/docs/source/client_user_guide.rst +++ b/docs/source/client_user_guide.rst @@ -97,8 +97,7 @@ Setup Server Components ~~~~~~~~~~~~~~~~~ Use of the Shipyard client requires that a working installation of the Shipyard -API is available. This may be accomplished by using the basic-ucp_ setup. -Detailed instructions are not part of this guide. +API is available. See :ref:`shipyard_deployment_guide` Local Environment ~~~~~~~~~~~~~~~~~ @@ -153,6 +152,8 @@ response:: the target Shipyard API/Keystone environment. - The ``shipyard`` and ``password`` values are the insecure values used by default if not overridden by the installation of Shipyard. +- The dev_minimal manifest deployment from Airship-in-a-bottle referenced in + the deployment guide provides a set of credentials that can be used. ``Configure hosts file, if necessary``:: @@ -196,9 +197,9 @@ The output will resemble the following:: Use Case: Ingest Site Design ---------------------------- -Shipyard serves as the entrypoint for a deployment of the Undercloud Platform -(UCP). One can imagine the following activities representing part of the -lifecycle of a group of servers that the UCP would serve as the control plane: +Shipyard serves as the entrypoint for a deployment of Airship. One can imagine +the following activities representing part of the lifecycle of a group of +servers for which Airship would serve as the control plane: Definition A group of servers making up a ``site`` has been identified. Designs covering @@ -207,10 +208,10 @@ Definition Preparation The site is assembled, racking, and wiring is completed, and the hardware is readied for operation. The ``Genesis Node`` is preinstalled with an - (Ubuntu 16.04) image. The Promenade_ genesis process similar to the - basic-ucp_ process is configured on and then run on the ``Genesis Node``. + (Ubuntu 16.04) image. Airship is deployed; See + :ref:`shipyard_deployment_guide` - At this point, the UCP is ready for use. This is the when the Shipyard API + At this point, Airship is ready for use. This is the when the Shipyard API is available for use. Load Configuration Documents @@ -218,7 +219,7 @@ Load Configuration Documents with Shipyard, perhaps by using the CLI. The operator loads ``configdocs`` which are a product of the definition step. These ``configdocs`` are declarative set of YAML documents using a format compatible with - Deckhand_ and containing information usable by the other UCP components. + Deckhand_ and containing information usable by the other Airship components. The interaction with Shipyard could happen as follows:: @@ -252,7 +253,7 @@ of an action:: $ shipyard commit configdocs -During this command, the other UCP components are contacted to validate the +During this command, the other Airship components are contacted to validate the designs in Deckhand. If the validations are not successful, Shipyard will not mark the revision as committed. @@ -271,7 +272,6 @@ mark the revision as committed. site-specific collection that combines with the other two collections to fully define the site. - Use Case: Deploy Site --------------------- Continuing the lifecycle steps from the Ingest Site Design use case, the @@ -314,14 +314,14 @@ action, the operator can query the status and results of the action:: ... More information is returned than shown here - for sake of abbreviation. The -process of maintenance is very similar to the process of deploying a site. +process of maintenance (update_site) is very similar to the process of +deploying a site. -.. _api-client: https://github.com/att-comdev/shipyard/tree/master/shipyard_client/api_client -.. _Armada: https://github.com/att-comdev/armada -.. _basic-ucp: https://github.com/att-comdev/ucp-integration/tree/master/manifests/basic_ucp -.. _Deckhand: http://deckhand.readthedocs.io/en/latest/ -.. _Divingbell: https://github.com/att-comdev/divingbell -.. _Drydock: https://github.com/att-comdev/drydock +.. _api-client: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_client +.. _Armada: https://git.airshipit.org/cgit/airship-armada +.. _Deckhand: https://git.airshipit.org/cgit/airship-deckhand +.. _Divingbell: https://git.airshipit.org/cgit/airship-divingbell +.. _Drydock: https://git.airshipit.org/cgit/airship-drydock .. _Keystone: https://developer.openstack.org/api-ref/identity/index.html -.. _Promenade: https://github.com/att-comdev/promenade +.. _Promenade: https://git.airshipit.org/cgit/airship-promenade diff --git a/docs/source/deployment_guide.rst b/docs/source/deployment_guide.rst index d7e31d16..1c81838b 100644 --- a/docs/source/deployment_guide.rst +++ b/docs/source/deployment_guide.rst @@ -23,109 +23,29 @@ Deployment Guide Shipyard is still under active development and this guide will evolve along the way -The current deployment makes use of the `ucp-integration`_ repository to set up -the underlaying Kubernetes infrastructure, Ceph and UCP components. This approach -sets up an 'All-In-One' UCP environment that allows developers to bring up Shipyard -and the rest of the UCP components on a single Ubuntu 16.04 Virtual Machine. +Deployment +---------- -.. note:: - The minimum recommended size of the VM is 4 vCPUs, 16GB of RAM with 64GB disk space +The current deployment makes use of the `airship-in-a-bottle`_ project to +set up the underlaying Kubernetes infrastructure, container networking +(Calico), disk provisioner (Ceph or NFS), and Airship components that are used +by Shipyard. +The `dev_minimal`_ manifest is the recommended manifest. Please see the +README.txt that exists in that manifest's directory. -Pre-Deployment Preparations ---------------------------- - -#. Set up ``etc/hosts`` on a freshly installed Ubuntu 16.04 Virtual Machine with the - IP and hostname of the VM:: - - HOST_IFACE=$(ip route | grep "^default" | head -1 | awk '{ print $5 }') - LOCAL_IP=$(ip addr | awk "/inet/ && /${HOST_IFACE}/{sub(/\/.*$/,\"\",\$2); print \$2}") - cat << EOF | sudo tee -a /etc/hosts - ${LOCAL_IP} $(hostname) - EOF - -#. Clone the `ucp-integration`_ repository - - -Production Deployment ---------------------- - -We will use this approach if we are not making any changes to Shipyard and would like to bring up -the UCP environment as it is - -#. Switch to root user after performing the steps in the *Pre-Deployment Preparations* section:: - - sudo -i - cd /home/ubuntu/ucp-integration/manifests/basic_ucp/ - -#. Export the variables that are unique to the environment. For instance, we can do the following - to update the environment variables for a VM with hostname **node1** that is assigned an IP of - **30.30.30.4** on interface **ens3** (network **30.30.30.0/24**):: - - export GENESIS_NODE_NAME=node1 - export CEPH_CLUSTER_NET=30.30.30.0/24 - export CEPH_PUBLIC_NET=30.30.30.0/24 - export GENESIS_NODE_IP=30.30.30.4 - export NODE_NET_IFACE=ens3 - -#. Start the UCP deployment:: - - ./deploy_ucp.sh - - -Dev Environment Deployment --------------------------- - -We will use this approach if we want to bring up a dev environment that allows us to test our own -dags and operators in Airflow. Changes to Shipyard API/CLI will require a rebuild of the Shipyard -images with the updates. We will need to reference to the custom image in our environment variables -so that it does not point to the image in the Master branch. - -#. Create the following directories on the target host machine:: - - mkdir -p /var/tmp/airflow/dags - mkdir -p /var/tmp/airflow/plugins - mkdir -p /var/tmp/airflow/logs - -#. Copy the `rest_api_plugin`_ into the newly created plugins directory, i.e. ``/var/tmp/airflow/plugins`` - so that it can be loaded by Airflow during startup. **Note** that other custom operators - should be added to the directory as required. - - **Note** that custom dags should be added into the newly created dags directory, i.e. ``/var/tmp/airflow/dags`` - -#. Switch to root user after performing Step 1 and 2:: - - sudo -i - cd /home/ubuntu/ucp-integration/manifests/basic_ucp/ - -#. Export the variables that are unique to the environment. For instance, we can do the below - to update the environment variables for a VM with hostname **node1** that is assigned an IP - of **30.30.30.4** on interface **ens3** (network **30.30.30.0/24**). - - Update image references in the environment variables if we want to test a new Shipyard and - Airflow image, e.g. image v0.1.0 as a result of code changes:: - - export SHIPYARD_PROD_DEPLOY=false - export GENESIS_NODE_NAME=node1 - export CEPH_CLUSTER_NET=30.30.30.0/24 - export CEPH_PUBLIC_NET=30.30.30.0/24 - export GENESIS_NODE_IP=30.30.30.4 - export NODE_NET_IFACE=ens3 - export SHIPYARD_IMAGE="attcomdev/shipyard:v0.1.0" - export AIRFLOW_IMAGE="attcomdev/airflow:v0.1.0" - -#. Start the UCP deployment:: - - ./deploy_ucp.sh +This approach sets up an 'All-In-One' Airship environment that allows +developers to bring up Shipyard and the rest of the Airship components on a +single Ubuntu Virtual Machine. +The deployment is fully automated and can take a while to complete (it can take +30 minutes to an hour for a full deployment to complete) Post Deployment --------------- -#. The deployment is fully automated and can take a while to complete (it can take 30 minutes - to an hour for a full deployment to complete) - -#. The environment should resemble the following after executing the required steps:: +#. The environment should resemble the following after executing the required + steps:: # sudo kubectl get pods -n ucp NAME READY STATUS RESTARTS AGE @@ -182,5 +102,6 @@ Post Deployment ucp-ucp-memcached 1 Fri Dec 1 10:02:44 2017 DEPLOYED memcached-0.1.0 ucp -.. _ucp-integration: https://github.com/att-comdev/ucp-integration -.. _rest_api_plugin: https://github.com/att-comdev/shipyard/blob/master/shipyard_airflow/plugins/rest_api_plugin.py +.. _airship-in-a-bottle: https://git.airshipit.org/cgit/airship-in-a-bottle +.. _dev_minimal: https://git.airshipit.org/cgit/airship-in-a-bottle/tree/manifests/dev_minimal +.. _rest_api_plugin: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/plugins/rest_api_plugin.py diff --git a/docs/source/index.rst b/docs/source/index.rst index 9c80b9d3..8ca021dd 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -19,7 +19,7 @@ Welcome to Shipyard's documentation! ==================================== Shipyard is a directed acyclic graph controller for Kubernetes and OpenStack -control plane life-cycle management. +control plane life-cycle management, and is part of the `Airship`_ platform. User's Guide ============ @@ -37,3 +37,5 @@ Shipyard Configuration Guide CLI client_user_guide deployment_guide + +.. _Airship: https://airshipit.org