Add udpate_software action to Shipyard

Provides an action that can be invoked by a user to deploy only the
software portion of a design.

Change-Id: I880bdc245064364dfdd6a482a3cf2d2a293f6c0d
This commit is contained in:
Bryan Strassner 2018-07-03 16:09:03 -05:00
parent 29d8810465
commit 70eb1cef10
25 changed files with 414 additions and 88 deletions

View File

@ -54,6 +54,22 @@ update_site
Applies a new committed configuration to the environment. The steps of Applies a new committed configuration to the environment. The steps of
update_site mirror those of :ref:`deploy_site`. update_site mirror those of :ref:`deploy_site`.
.. _update_software:
update_software
~~~~~~~~~~~~~~~
Triggers an update of the software in a site, using the latest committed
configuration documents. Steps, conceptually:
#. Concurrency check
Prevents concurrent site modifications by conflicting
actions/workflows.
#. Validate design
Asks each involved Airship component to validate the design. This ensures
that the previously committed design is valid at the present time.
#. Armada build
Orchestrates Armada to configure software on the nodes as designed.
Actions under development Actions under development
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~

View File

@ -93,11 +93,11 @@ be several versions of documents in a site that are accessible via this API:
default. (This behavior can be overridden by query parameters issued by the default. (This behavior can be overridden by query parameters issued by the
user of Shipyard) user of Shipyard)
- The "Last Site Action" version represents the version of documents associated - The "Last Site Action" version represents the version of documents associated
with the last successful or failed site action. Site actions include 'deploy_site' with the last successful or failed site action.
and 'update_site'.
- The "Successful Site Action" version represents the version of documents - The "Successful Site Action" version represents the version of documents
associated with the last successful site action. Site actions include 'deploy_site' associated with the last successful site action.
and 'update_site'. - Site actions include ``deploy_site``, ``update_site``, and
``update_software``.
All versions of documents rely upon Deckhand for storage. Shipyard uses the All versions of documents rely upon Deckhand for storage. Shipyard uses the
tagging features of Deckhand to find the appropriate Committed Documents, tagging features of Deckhand to find the appropriate Committed Documents,

View File

@ -582,9 +582,9 @@ get configdocs
Retrieve documents loaded into Shipyard. The possible options include last Retrieve documents loaded into Shipyard. The possible options include last
committed, last site action, last successful site action and retrieval from committed, last site action, last successful site action and retrieval from
the Shipyard Buffer. Site actions include deploy_site and update_site. Note the Shipyard Buffer. Site actions include ``deploy_site``, ``update_site`` and
that we can only select one of the options when we retrieve the documents ``update_software``. Note that only one option may be selected when retrieving
for a particular collection. the documents for a particular collection.
The command will compare the differences between the revisions specified if The command will compare the differences between the revisions specified if
the collection option is not specified. Note that we can only compare between the collection option is not specified. Note that we can only compare between

View File

@ -132,7 +132,7 @@ class DocumentValidator(metaclass=abc.ABCMeta):
self.doc_name, self.doc_name,
self.schema) self.schema)
# only proceed to validating the document if it is present. # only proceed to validating the document if it is present.
LOG.debug("Generic document validaton complete. Proceeding to " LOG.debug("Generic document validation complete. Proceeding to "
"specific validation") "specific validation")
self.do_validate() self.do_validate()
except DocumentLookupError as dle: except DocumentLookupError as dle:

View File

@ -24,13 +24,15 @@ from shipyard_airflow.common.document_validators.document_validator_manager \
import DocumentValidationManager import DocumentValidationManager
from shipyard_airflow.control import service_clients from shipyard_airflow.control import service_clients
from shipyard_airflow.control.validators.validate_deployment_configuration \ from shipyard_airflow.control.validators.validate_deployment_configuration \
import ValidateDeploymentConfiguration import ValidateDeploymentConfigurationBasic
from shipyard_airflow.control.validators.validate_deployment_configuration \
import ValidateDeploymentConfigurationFull
from shipyard_airflow.errors import ApiError from shipyard_airflow.errors import ApiError
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
def validate_site_action(action): def validate_site_action_full(action):
"""Validates that the deployment configuration is correctly set up """Validates that the deployment configuration is correctly set up
Checks: Checks:
@ -46,24 +48,53 @@ def validate_site_action(action):
""" """
validator = _SiteActionValidator( validator = _SiteActionValidator(
dh_client=service_clients.deckhand_client(), dh_client=service_clients.deckhand_client(),
action=action action=action,
full_validation=True
)
validator.validate()
def validate_site_action_basic(action):
"""Validates that the DeploymentConfiguration is present
Checks:
- The deployment configuration from Deckhand using the design version
- If the deployment configuration is missing, error
"""
validator = _SiteActionValidator(
dh_client=service_clients.deckhand_client(),
action=action,
full_validation=False
) )
validator.validate() validator.validate()
class _SiteActionValidator: class _SiteActionValidator:
"""The validator object setup and used by the validate_site_action function """The validator object used by the validate_site_action_<x> functions
""" """
def __init__(self, dh_client, action): def __init__(self, dh_client, action, full_validation=True):
self.action = action self.action = action
self.doc_revision = self._get_doc_revision() self.doc_revision = self._get_doc_revision()
self.cont_on_fail = str(self._action_param( self.cont_on_fail = str(self._action_param(
'continue-on-fail')).lower() == 'true' 'continue-on-fail')).lower() == 'true'
self.doc_val_mgr = DocumentValidationManager( if full_validation:
dh_client, # Perform a complete validation
self.doc_revision, self.doc_val_mgr = DocumentValidationManager(
[(ValidateDeploymentConfiguration, 'deployment-configuration')] dh_client,
) self.doc_revision,
[(ValidateDeploymentConfigurationFull,
'deployment-configuration')]
)
else:
# Perform a basic validation only
self.doc_val_mgr = DocumentValidationManager(
dh_client,
self.doc_revision,
[(ValidateDeploymentConfigurationBasic,
'deployment-configuration')]
)
def validate(self): def validate(self):
results = self.doc_val_mgr.validate() results = self.doc_val_mgr.validate()

View File

@ -44,11 +44,15 @@ def _action_mappings():
return { return {
'deploy_site': { 'deploy_site': {
'dag': 'deploy_site', 'dag': 'deploy_site',
'validators': [action_validators.validate_site_action] 'validators': [action_validators.validate_site_action_full]
}, },
'update_site': { 'update_site': {
'dag': 'update_site', 'dag': 'update_site',
'validators': [action_validators.validate_site_action] 'validators': [action_validators.validate_site_action_full]
},
'update_software': {
'dag': 'update_software',
'validators': [action_validators.validate_site_action_basic]
}, },
'redeploy_server': { 'redeploy_server': {
'dag': 'redeploy_server', 'dag': 'redeploy_server',

View File

@ -35,7 +35,7 @@ from shipyard_airflow.control.helpers.deckhand_client import (
from shipyard_airflow.control.service_endpoints import ( from shipyard_airflow.control.service_endpoints import (
Endpoints, get_endpoint, get_token) Endpoints, get_endpoint, get_token)
from shipyard_airflow.control.validators.validate_deployment_configuration \ from shipyard_airflow.control.validators.validate_deployment_configuration \
import ValidateDeploymentConfiguration import ValidateDeploymentConfigurationFull
from shipyard_airflow.errors import ApiError, AppError from shipyard_airflow.errors import ApiError, AppError
CONF = cfg.CONF CONF = cfg.CONF
@ -497,12 +497,13 @@ class ConfigdocsHelper(object):
return _format_validations_to_status(resp_msgs, error_count) return _format_validations_to_status(resp_msgs, error_count)
def _get_shipyard_validations(self, revision_id): def _get_shipyard_validations(self, revision_id):
# Run Shipyard's own validations # Run Shipyard's own validations.
try: try:
sy_val_mgr = DocumentValidationManager( sy_val_mgr = DocumentValidationManager(
service_clients.deckhand_client(), service_clients.deckhand_client(),
revision_id, revision_id,
[(ValidateDeploymentConfiguration, 'deployment-configuration')] [(ValidateDeploymentConfigurationFull,
'deployment-configuration')]
) )
return sy_val_mgr.validate() return sy_val_mgr.validate()
except Exception as ex: except Exception as ex:

View File

@ -26,14 +26,29 @@ from .validate_deployment_strategy import ValidateDeploymentStrategy
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
class ValidateDeploymentConfiguration(DocumentValidator): class ValidateDeploymentConfigurationBasic(DocumentValidator):
"""Validates the DeploymentConfiguration.""" """Validates that the DeploymentConfiguration is present
The base DocumentValidator ensures the document is present.
The Schema validation done separately ensures that the Armada Manifest
document is specified.
"""
def __init__(self, **kwargs): def __init__(self, **kwargs):
super().__init__(**kwargs) super().__init__(**kwargs)
schema = "shipyard/DeploymentConfiguration/v1" schema = "shipyard/DeploymentConfiguration/v1"
missing_severity = "Error" missing_severity = "Error"
def do_validate(self):
self.error_status = False
class ValidateDeploymentConfigurationFull(
ValidateDeploymentConfigurationBasic):
"""Validates the DeploymentConfiguration
Includes a triggered check for DeploymentStrategy
"""
def do_validate(self): def do_validate(self):
try: try:
dep_strat_nm = ( dep_strat_nm = (
@ -42,7 +57,10 @@ class ValidateDeploymentConfiguration(DocumentValidator):
self.add_triggered_validation(ValidateDeploymentStrategy, self.add_triggered_validation(ValidateDeploymentStrategy,
dep_strat_nm) dep_strat_nm)
except KeyError: except (KeyError, TypeError):
# need to check both KeyError for missing 'deployment_strategy'
# and TypeError for not subscriptable exception when
# 'physical_provisioner' is None
self.val_msg_list.append(self.val_msg( self.val_msg_list.append(self.val_msg(
name="DeploymentStrategyNotSpecified", name="DeploymentStrategyNotSpecified",
error=False, error=False,
@ -55,4 +73,4 @@ class ValidateDeploymentConfiguration(DocumentValidator):
"'all-at-once' is assumed, and deployment strategy will " "'all-at-once' is assumed, and deployment strategy will "
"not be further validated") "not be further validated")
self.error_status = False super().do_validate()

View File

@ -46,6 +46,11 @@ def _get_node_lookup(revision_id):
).lookup ).lookup
def _get_deployment_group_manager(groups, revision_id):
"""Retrieves the deployment group manager"""
return DeploymentGroupManager(groups, _get_node_lookup(revision_id))
class ValidateDeploymentStrategy(DocumentValidator): class ValidateDeploymentStrategy(DocumentValidator):
"""Validates the deployment strategy""" """Validates the deployment strategy"""
def __init__(self, **kwargs): def __init__(self, **kwargs):
@ -57,7 +62,7 @@ class ValidateDeploymentStrategy(DocumentValidator):
def do_validate(self): def do_validate(self):
groups = self.doc_dict['groups'] groups = self.doc_dict['groups']
try: try:
DeploymentGroupManager(groups, _get_node_lookup(self.revision)) _get_deployment_group_manager(groups, self.revision)
except DeploymentGroupCycleError as dgce: except DeploymentGroupCycleError as dgce:
self.val_msg_list.append(self.val_msg( self.val_msg_list.append(self.val_msg(
name=dgce.__class__.__name__, name=dgce.__class__.__name__,

View File

@ -105,6 +105,7 @@ class CommonStepFactory(object):
dag=self.dag) dag=self.dag)
def get_validate_site_design(self, def get_validate_site_design(self,
targets=None,
task_id=dn.VALIDATE_SITE_DESIGN_DAG_NAME): task_id=dn.VALIDATE_SITE_DESIGN_DAG_NAME):
"""Generate the validate site design step """Generate the validate site design step
@ -115,7 +116,8 @@ class CommonStepFactory(object):
subdag=validate_site_design( subdag=validate_site_design(
self.parent_dag_name, self.parent_dag_name,
task_id, task_id,
args=self.default_args), args=self.default_args,
targets=targets),
task_id=task_id, task_id=task_id,
on_failure_callback=step_failure_handler, on_failure_callback=step_failure_handler,
dag=self.dag) dag=self.dag)

View File

@ -0,0 +1,74 @@
# Copyright 2018 AT&T Intellectual Property. All other rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from datetime import timedelta
import airflow
from airflow import DAG
from common_step_factory import CommonStepFactory
from validate_site_design import SOFTWARE
"""update_software
The top-level orchestration DAG for updating only the software components
using the Undercloud platform.
"""
PARENT_DAG_NAME = 'update_software'
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'start_date': airflow.utils.dates.days_ago(1),
'email': [''],
'email_on_failure': False,
'email_on_retry': False,
'provide_context': True,
'retries': 0,
'retry_delay': timedelta(seconds=30),
}
dag = DAG(PARENT_DAG_NAME, default_args=default_args, schedule_interval=None)
step_factory = CommonStepFactory(parent_dag_name=PARENT_DAG_NAME,
dag=dag,
default_args=default_args)
action_xcom = step_factory.get_action_xcom()
concurrency_check = step_factory.get_concurrency_check()
deployment_configuration = step_factory.get_deployment_configuration()
validate_site_design = step_factory.get_validate_site_design(
targets=[SOFTWARE]
)
armada_build = step_factory.get_armada_build()
decide_airflow_upgrade = step_factory.get_decide_airflow_upgrade()
upgrade_airflow = step_factory.get_upgrade_airflow()
skip_upgrade_airflow = step_factory.get_skip_upgrade_airflow()
create_action_tag = step_factory.get_create_action_tag()
# DAG Wiring
deployment_configuration.set_upstream(action_xcom)
validate_site_design.set_upstream([
concurrency_check,
deployment_configuration
])
armada_build.set_upstream(validate_site_design)
decide_airflow_upgrade.set_upstream(armada_build)
decide_airflow_upgrade.set_downstream([
upgrade_airflow,
skip_upgrade_airflow
])
create_action_tag.set_upstream([
upgrade_airflow,
skip_upgrade_airflow
])

View File

@ -21,7 +21,11 @@ from airflow.operators import PromenadeValidateSiteDesignOperator
from config_path import config_path from config_path import config_path
def validate_site_design(parent_dag_name, child_dag_name, args): BAREMETAL = 'baremetal'
SOFTWARE = 'software'
def validate_site_design(parent_dag_name, child_dag_name, args, targets=None):
"""Subdag to delegate design verification to the UCP components """Subdag to delegate design verification to the UCP components
There is no wiring of steps - they all execute in parallel There is no wiring of steps - they all execute in parallel
@ -30,32 +34,44 @@ def validate_site_design(parent_dag_name, child_dag_name, args):
'{}.{}'.format(parent_dag_name, child_dag_name), '{}.{}'.format(parent_dag_name, child_dag_name),
default_args=args) default_args=args)
deckhand_validate_docs = DeckhandValidateSiteDesignOperator( if targets is None:
targets = [BAREMETAL, SOFTWARE]
# Always add Deckhand validations
DeckhandValidateSiteDesignOperator(
task_id='deckhand_validate_site_design', task_id='deckhand_validate_site_design',
shipyard_conf=config_path, shipyard_conf=config_path,
main_dag_name=parent_dag_name, main_dag_name=parent_dag_name,
retries=1, retries=1,
dag=dag) dag=dag
)
drydock_validate_docs = DrydockValidateDesignOperator( if BAREMETAL in targets:
task_id='drydock_validate_site_design', # Add Drydock and Promenade validations
shipyard_conf=config_path, DrydockValidateDesignOperator(
main_dag_name=parent_dag_name, task_id='drydock_validate_site_design',
retries=1, shipyard_conf=config_path,
dag=dag) main_dag_name=parent_dag_name,
retries=1,
dag=dag
)
armada_validate_docs = ArmadaValidateDesignOperator( PromenadeValidateSiteDesignOperator(
task_id='armada_validate_site_design', task_id='promenade_validate_site_design',
shipyard_conf=config_path, shipyard_conf=config_path,
main_dag_name=parent_dag_name, main_dag_name=parent_dag_name,
retries=1, retries=1,
dag=dag) dag=dag
)
promenade_validate_docs = PromenadeValidateSiteDesignOperator( if SOFTWARE in targets:
task_id='promenade_validate_site_design', # Add Armada validations
shipyard_conf=config_path, ArmadaValidateDesignOperator(
main_dag_name=parent_dag_name, task_id='armada_validate_site_design',
retries=1, shipyard_conf=config_path,
dag=dag) main_dag_name=parent_dag_name,
retries=1,
dag=dag
)
return dag return dag

View File

@ -28,7 +28,8 @@ DAG_RUN_SELECT_RUNNING_SQL = ("select dag_id, execution_date "
AIRFLOW_DB = 'airflows_own_db' AIRFLOW_DB = 'airflows_own_db'
# each set in this list of sets indicates DAGs that shouldn't execute together # each set in this list of sets indicates DAGs that shouldn't execute together
CONFLICTING_DAG_SETS = [set(['deploy_site', 'update_site', 'redeploy_server'])] CONFLICTING_DAG_SETS = [set(['deploy_site', 'update_site', 'update_software',
'redeploy_server'])]
def find_conflicting_dag_set(dag_name, conflicting_dag_sets=None): def find_conflicting_dag_set(dag_name, conflicting_dag_sets=None):

View File

@ -111,7 +111,7 @@ class DeckhandCreateSiteActionTagOperator(DeckhandBaseOperator):
task = ['armada_build'] task = ['armada_build']
task_result = {} task_result = {}
if self.main_dag_name == 'update_site': if self.main_dag_name in ['update_site', 'update_software']:
# NOTE: We will check the final state of the 'armada_build' task # NOTE: We will check the final state of the 'armada_build' task
# as a 'success' means that all tasks preceding it would either # as a 'success' means that all tasks preceding it would either
# be in 'skipped' or 'success' state. A failure of 'armada_build' # be in 'skipped' or 'success' state. A failure of 'armada_build'
@ -119,8 +119,8 @@ class DeckhandCreateSiteActionTagOperator(DeckhandBaseOperator):
# to determine the success/failure of the 'deploy_site' workflow # to determine the success/failure of the 'deploy_site' workflow
# with the final state of the 'armada_build' task. # with the final state of the 'armada_build' task.
# #
# NOTE: The 'update_site' workflow contains additional steps for # NOTE: The 'update_site' and 'update_software' workflows contain
# upgrading of worker pods. # additional steps for upgrading of worker pods.
for k in ['skip_upgrade_airflow', 'upgrade_airflow']: for k in ['skip_upgrade_airflow', 'upgrade_airflow']:
task.append(k) task.append(k)

View File

@ -108,6 +108,8 @@ class UcpHealthCheckOperator(BaseOperator):
""" """
# If Drydock health check fails and continue-on-fail, continue # If Drydock health check fails and continue-on-fail, continue
# and create xcom key 'drydock_continue_on_fail' # and create xcom key 'drydock_continue_on_fail'
# Note that 'update_software' does not interact with Drydock, and
# therefore does not use the continue-on-fail option.
if (component == service_endpoint.DRYDOCK and if (component == service_endpoint.DRYDOCK and
self.action_info['parameters'].get( self.action_info['parameters'].get(
'continue-on-fail', 'false').lower() == 'true' and 'continue-on-fail', 'false').lower() == 'true' and

View File

@ -18,8 +18,14 @@ import yaml
import pytest import pytest
from shipyard_airflow.common.deployment_group.errors import (
DeploymentGroupCycleError,
InvalidDeploymentGroupError,
InvalidDeploymentGroupNodeLookupError
)
from shipyard_airflow.control.action.action_validators import ( from shipyard_airflow.control.action.action_validators import (
validate_site_action validate_site_action_basic,
validate_site_action_full
) )
from shipyard_airflow.errors import ApiError from shipyard_airflow.errors import ApiError
from tests.unit.common.deployment_group.node_lookup_stubs import node_lookup from tests.unit.common.deployment_group.node_lookup_stubs import node_lookup
@ -70,10 +76,10 @@ class TestActionValidator:
@mock.patch("shipyard_airflow.control.validators." @mock.patch("shipyard_airflow.control.validators."
"validate_deployment_strategy._get_node_lookup", "validate_deployment_strategy._get_node_lookup",
return_value=node_lookup) return_value=node_lookup)
def test_validate_site_action(self, *args): def test_validate_site_action_full(self, *args):
"""Test the function that runs the validator class""" """Test the function that runs the validator class"""
try: try:
validate_site_action({ validate_site_action_full({
'id': '123', 'id': '123',
'name': 'deploy_site', 'name': 'deploy_site',
'committed_rev_id': 1 'committed_rev_id': 1
@ -87,12 +93,12 @@ class TestActionValidator:
@mock.patch("shipyard_airflow.control.validators." @mock.patch("shipyard_airflow.control.validators."
"validate_deployment_strategy._get_node_lookup", "validate_deployment_strategy._get_node_lookup",
return_value=node_lookup) return_value=node_lookup)
def test_validate_site_action_cycle(self, *args): def test_validate_site_action_full_cycle(self, *args):
"""Test the function that runs the validator class with a """Test the function that runs the validator class with a
deployment strategy that has a cycle in the groups deployment strategy that has a cycle in the groups
""" """
with pytest.raises(ApiError) as apie: with pytest.raises(ApiError) as apie:
validate_site_action({ validate_site_action_full({
'id': '123', 'id': '123',
'name': 'deploy_site', 'name': 'deploy_site',
'committed_rev_id': 1 'committed_rev_id': 1
@ -107,12 +113,12 @@ class TestActionValidator:
@mock.patch("shipyard_airflow.control.validators." @mock.patch("shipyard_airflow.control.validators."
"validate_deployment_strategy._get_node_lookup", "validate_deployment_strategy._get_node_lookup",
return_value=node_lookup) return_value=node_lookup)
def test_validate_site_action_missing_dep_strat(self, *args): def test_validate_site_action_full_missing_dep_strat(self, *args):
"""Test the function that runs the validator class with a missing """Test the function that runs the validator class with a missing
deployment strategy - specified, but not present deployment strategy - specified, but not present
""" """
with pytest.raises(ApiError) as apie: with pytest.raises(ApiError) as apie:
validate_site_action({ validate_site_action_full({
'id': '123', 'id': '123',
'name': 'deploy_site', 'name': 'deploy_site',
'committed_rev_id': 1 'committed_rev_id': 1
@ -121,21 +127,21 @@ class TestActionValidator:
assert apie.value.error_list[0]['name'] == 'DocumentNotFoundError' assert apie.value.error_list[0]['name'] == 'DocumentNotFoundError'
@mock.patch("shipyard_airflow.control.service_clients.deckhand_client", @mock.patch("shipyard_airflow.control.service_clients.deckhand_client",
return_value=fake_dh_doc_client('clean'), ds_name='defaulted') return_value=fake_dh_doc_client('clean', ds_name='defaulted'))
@mock.patch("shipyard_airflow.control.validators." @mock.patch("shipyard_airflow.control.validators."
"validate_deployment_strategy._get_node_lookup", "validate_deployment_strategy._get_node_lookup",
return_value=node_lookup) return_value=node_lookup)
def test_validate_site_action_default_dep_strat(self, *args): def test_validate_site_action_full_default_dep_strat(self, *args):
"""Test the function that runs the validator class with a defaulted """Test the function that runs the validator class with a defaulted
deployment strategy (not specified) deployment strategy (not specified)
""" """
try: try:
validate_site_action({ validate_site_action_full({
'id': '123', 'id': '123',
'name': 'deploy_site', 'name': 'deploy_site',
'committed_rev_id': 1 'committed_rev_id': 1
}) })
except: except Exception:
# any exception is a failure # any exception is a failure
assert False assert False
@ -149,7 +155,7 @@ class TestActionValidator:
deployment strategy that has a cycle in the groups deployment strategy that has a cycle in the groups
""" """
with pytest.raises(ApiError) as apie: with pytest.raises(ApiError) as apie:
validate_site_action({ validate_site_action_full({
'id': '123', 'id': '123',
'name': 'deploy_site' 'name': 'deploy_site'
}) })
@ -160,17 +166,81 @@ class TestActionValidator:
@mock.patch("shipyard_airflow.control.validators." @mock.patch("shipyard_airflow.control.validators."
"validate_deployment_strategy._get_node_lookup", "validate_deployment_strategy._get_node_lookup",
return_value=node_lookup) return_value=node_lookup)
def test_validate_site_action_continue_failure(self, *args): def test_validate_site_action_full_continue_failure(self, *args):
"""Test the function that runs the validator class with a defaulted """Test the function that runs the validator class with a missing
deployment strategy (not specified) deployment strategy (not specified), but continue-on-fail specified
""" """
try: try:
validate_site_action({ validate_site_action_full({
'id': '123', 'id': '123',
'name': 'deploy_site', 'name': 'deploy_site',
'committed_rev_id': 1, 'committed_rev_id': 1,
'parameters': {'continue-on-fail': 'true'} 'parameters': {'continue-on-fail': 'true'}
}) })
except: except Exception:
# any exception is a failure # any exception is a failure
assert False assert False
@mock.patch("shipyard_airflow.control.service_clients.deckhand_client",
return_value=fake_dh_doc_client('clean', ds_name='not-there'))
@mock.patch("shipyard_airflow.control.validators."
"validate_deployment_strategy._get_node_lookup",
return_value=node_lookup)
def test_validate_site_action_basic_missing_dep_strat(self, *args):
"""Test the function that runs the validator class with a missing
deployment strategy - specified, but not present. This should be
ignored by the basic valdiator
"""
try:
validate_site_action_basic({
'id': '123',
'name': 'deploy_site',
'committed_rev_id': 1
})
except Exception:
# any exception is a failure
assert False
@mock.patch("shipyard_airflow.control.service_clients.deckhand_client",
return_value=fake_dh_doc_client('clean'))
@mock.patch("shipyard_airflow.control.validators."
"validate_deployment_strategy._get_node_lookup",
return_value=node_lookup)
def test_validate_site_action_dep_strategy_exceptions(self, *args):
"""Test the function that runs the validator class for exceptions"""
to_catch = [InvalidDeploymentGroupNodeLookupError,
InvalidDeploymentGroupError, DeploymentGroupCycleError]
for exc in to_catch:
with mock.patch(
"shipyard_airflow.control.validators."
"validate_deployment_strategy._get_deployment_group_manager",
side_effect=exc()
):
with pytest.raises(ApiError) as apie:
validate_site_action_full({
'id': '123',
'name': 'deploy_site',
'committed_rev_id': 1
})
assert apie.value.description == 'InvalidConfigurationDocuments'
assert apie.value.error_list[0]['name'] == (exc.__name__)
@mock.patch("shipyard_airflow.control.service_clients.deckhand_client",
return_value=fake_dh_doc_client('clean'))
@mock.patch("shipyard_airflow.control.validators."
"validate_deployment_strategy._get_node_lookup",
return_value=node_lookup)
@mock.patch("shipyard_airflow.control.validators."
"validate_deployment_strategy._get_deployment_group_manager",
side_effect=TypeError())
def test_validate_site_action_dep_strategy_exception_other(self, *args):
"""Test the function that runs the validator class"""
with pytest.raises(ApiError) as apie:
validate_site_action_full({
'id': '123',
'name': 'deploy_site',
'committed_rev_id': 1
})
assert apie.value.description == 'InvalidConfigurationDocuments'
assert apie.value.error_list[0]['name'] == (
'DocumentValidationProcessingError')

View File

@ -313,7 +313,7 @@ def test_create_action():
# with invalid input. fail. # with invalid input. fail.
with mock.patch('shipyard_airflow.control.action.action_validators' with mock.patch('shipyard_airflow.control.action.action_validators'
'.validate_site_action') as validator: '.validate_site_action_full') as validator:
try: try:
action = action_resource.create_action( action = action_resource.create_action(
action={'name': 'broken', action={'name': 'broken',
@ -330,7 +330,7 @@ def test_create_action():
# with valid input and some parameters # with valid input and some parameters
with mock.patch('shipyard_airflow.control.action.action_validators' with mock.patch('shipyard_airflow.control.action.action_validators'
'.validate_site_action') as validator: '.validate_site_action_full') as validator:
try: try:
action = action_resource.create_action( action = action_resource.create_action(
action={'name': 'deploy_site', action={'name': 'deploy_site',
@ -351,7 +351,7 @@ def test_create_action():
# with valid input and no parameters # with valid input and no parameters
with mock.patch('shipyard_airflow.control.action.action_validators' with mock.patch('shipyard_airflow.control.action.action_validators'
'.validate_site_action') as validator: '.validate_site_action_full') as validator:
try: try:
action = action_resource.create_action( action = action_resource.create_action(
action={'name': 'deploy_site'}, action={'name': 'deploy_site'},
@ -382,7 +382,7 @@ def test_create_action_validator_error():
# with valid input and some parameters # with valid input and some parameters
with mock.patch('shipyard_airflow.control.action.action_validators' with mock.patch('shipyard_airflow.control.action.action_validators'
'.validate_site_action', '.validate_site_action_full',
side_effect=ApiError(title='bad')): side_effect=ApiError(title='bad')):
with pytest.raises(ApiError) as apie: with pytest.raises(ApiError) as apie:
action = action_resource.create_action( action = action_resource.create_action(

View File

@ -222,7 +222,7 @@ def get_version(ctx, buffer, committed, last_site_action,
'successful or failed site action\n' 'successful or failed site action\n'
'--successful-site-action for the documents associated with the ' '--successful-site-action for the documents associated with the '
'last successful site action\n' 'last successful site action\n'
'Site actions include deploy_site and update_site.') 'Site actions are deploy_site, update_site, and update_software')
elif len(optional_site_parameters) == 1: elif len(optional_site_parameters) == 1:
return optional_site_parameters[0] return optional_site_parameters[0]

View File

@ -37,9 +37,12 @@ The workflow actions that may be invoked using Shipyard
deploy_site: Triggers the initial deployment of a site using the latest deploy_site: Triggers the initial deployment of a site using the latest
committed configuration documents. committed configuration documents.
update_site: Triggers the initial deployment of a site, using the latest update_site: Triggers the update to a deployment of a site, using the latest
committed configuration documents. committed configuration documents.
update_software: Starts an update that only exercises the software portion of
the commited configuration documents.
redeploy_server: Using parameters to indicate which server(s), triggers a redeploy_server: Using parameters to indicate which server(s), triggers a
redeployment of servers to the last committed design and redeployment of servers to the last committed design and
secrets. secrets.

View File

@ -18,9 +18,11 @@ from arrow.parser import ParserError
def check_action_command(ctx, action_command): def check_action_command(ctx, action_command):
"""Verifies the action command is valid""" """Verifies the action command is valid"""
if action_command not in ['deploy_site', 'update_site', 'redeploy_server']: valid_commands = ['deploy_site', 'update_site', 'update_software',
'redeploy_server']
if action_command not in valid_commands:
ctx.fail('Invalid action command. The action commands available are ' ctx.fail('Invalid action command. The action commands available are '
'deploy_site, update_site, and redeploy_server.') ' {}'.format(', '.join(valid_commands)))
def check_control_action(ctx, action): def check_control_action(ctx, action):

View File

@ -176,6 +176,7 @@ def test_check_action_commands():
ctx = Mock(side_effect=Exception("failed")) ctx = Mock(side_effect=Exception("failed"))
input_checks.check_action_command(ctx, 'deploy_site') input_checks.check_action_command(ctx, 'deploy_site')
input_checks.check_action_command(ctx, 'update_site') input_checks.check_action_command(ctx, 'update_site')
input_checks.check_action_command(ctx, 'update_software')
input_checks.check_action_command(ctx, 'redeploy_server') input_checks.check_action_command(ctx, 'redeploy_server')
ctx.fail.assert_not_called() ctx.fail.assert_not_called()

View File

@ -13,8 +13,8 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
# This is a common script that is used by the deploy_site, update_site # This is a common script that is used by the deploy_site, update_site,
# and redeploy_server scripts # update_software and redeploy_server scripts
set -ex set -ex

View File

@ -13,13 +13,13 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
# NOTE: If user is executing deploy_site, update_site or redeploy_server # NOTE: If a user is executing deploy_site, update_site, update_software or
# workflow from outside the cluster, e.g. from a remote jump server, then # redeploy_server workflow from outside the cluster, e.g. from a remote jump
# he/she will need to ensure that the DNS server is able to resolve the # server, then the user will need to ensure that the DNS server is able to
# FQDN of the Shipyard and Keystone public URL (both will be pointing to # resolve the FQDN of the Shipyard and Keystone public URL (both will be
# the IP of the Ingress Controller). If the DNS resolution is not available, # pointing to the IP of the Ingress Controller). If the DNS resolution is not
# the user will need to ensure that the /etc/hosts file is properly updated # available, the user will need to ensure that the /etc/hosts file is properly
# before setting up the environment variables and running the worflow. # updated before setting up the environment variables and running the worflow.
# Define Variable # Define Variable
namespace="${namespace:-ucp}" namespace="${namespace:-ucp}"

58
tools/shipyard.sh Executable file
View File

@ -0,0 +1,58 @@
#!/bin/bash
# Copyright 2018 AT&T Intellectual Property. All other rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -e
# User can run the script as they would execute the Shipyard CLI.
# For instance, to run the 'shipyard get actions' command, user can execute
# the following command after setting up the required environment variables:
#
# $ ./tools/shipyard.sh get actions
#
# NOTE: If user is executing the script from outside the cluster, e.g. from
# a remote jump server, then he/she will need to ensure that the DNS server
# is able to resolve the FQDN of the Shipyard and Keystone public URL (both
# will be pointing to the IP of the Ingress Controller). If the DNS resolution
# is not available, the user will need to ensure that the /etc/hosts file is
# properly updated before running the script.
# Commands requiring files as input utilize the pwd mounted into the container
# as the /target directory, e.g.:
#
# $ ./tools/shipyard.sh create configdocs design --filename=/target/afile.yaml
# Get the path of the directory where the script is located
# Source Base Docker Command
SHIPYARD_HOSTPATH=${SHIPYARD_HOSTPATH:-"/target"}
NAMESPACE="${NAMESPACE:-ucp}"
SHIPYARD_IMAGE="${SHIPYARD_IMAGE:-quay.io/airshipit/shipyard:master}"
# Define Base Docker Command
base_docker_command=$(cat << EndOfCommand
sudo docker run -t --rm --net=host
-e http_proxy=${HTTP_PROXY}
-e https_proxy=${HTTPS_PROXY}
-e OS_AUTH_URL=${OS_AUTH_URL:-http://keystone.${NAMESPACE}.svc.cluster.local:80/v3}
-e OS_USERNAME=${OS_USERNAME:-shipyard}
-e OS_USER_DOMAIN_NAME=${OS_USER_DOMAIN_NAME:-default}
-e OS_PASSWORD=${OS_PASSWORD:-password}
-e OS_PROJECT_DOMAIN_NAME=${OS_PROJECT_DOMAIN_NAME:-default}
-e OS_PROJECT_NAME=${OS_PROJECT_NAME:-service}
EndOfCommand
)
# Execute Shipyard CLI
${base_docker_command} -v "$(pwd)":"${SHIPYARD_HOSTPATH}" "${SHIPYARD_IMAGE}" $@

22
tools/update_software.sh Executable file
View File

@ -0,0 +1,22 @@
#!/bin/bash
# Copyright 2017 AT&T Intellectual Property. All other rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -ex
# Source environment variables
source set_env
# Execute shipyard action for update_software
bash execute_shipyard_action.sh 'update_software'