.. WARNING: Add no lines of text between the label immediately following
.. and the title.

.. _deploy-software-releases-using-the-cli-1ea02eb230e5:

======================================
Deploy Software Releases using the CLI
======================================

After a software release has been uploaded to the central repository, it can be
deployed first on the System Controller and then on a specific subcloud or
across a group of subclouds through different orchestration methods.

Deploy a Software Release on the System Controller
--------------------------------------------------

Follow the instructions at
:ref:`orchestrated-deployment-host-software-deployment-d234754c7d20` to
complete the software deployment.

.. note::

    The step to upload the software release can be skipped since it has already
    been done.

It is recommended to upgrade Kubernetes following the deployment of a major
release. See :ref:`the-kubernetes-update-orchestration-process`.

The System Controller is now ready for new subcloud deployment.

.. note::

    Starting release |prod-ver|, low-latency configuration via remote install
    is no longer supported.

    For new subcloud deployments that involve remote install using Redfish,
    avoid using ``install_type`` of 4 and 5 in ``install-values.yaml``.

    For existing subclouds that were originally deployed with remote install,
    use :command:`dcmanager subcloud update --install-values` to update
    ``install_type`` so they can be restored in the future. Ensure that the
    software version in ``install_values`` matches the new version.

Orchestrate the Deployment of a Software Release Across the Subclouds
---------------------------------------------------------------------

After the System Controller has been successfully updated with a new major or
patch software release; distributed software deployment orchestration can be
initiated to bring the managed subclouds to the same software level as the
System Controller. The subclouds must, however, be prestaged with the target
software before orchestrated deployment can take place.

For more information on Prestaging Subcloud Orchestration see
:ref:`orchestrate-subcloud-prestage-using-the-cli-eb516473582f`.

Before orchestrating a major software release deployment across the subclouds,
it is strongly recommended that a backup of each subcloud to be upgraded is
taken. See
:ref:`backup-a-subcloud-group-of-subclouds-using-dcmanager-cli-f12020a8fc42`
for instructions.

.. rubric:: |prereq|

The following conditions must be met for the orchestrated software deployment
across the subclouds to be successful.

-   All target subclouds are managed, online and healthy. The Kubernetes
    cluster on each subcloud is fully operational and there are no management
    affecting alarms.

-   Subcloud certificates are up-to-date or will not expire during the
    deployment of the new software release.

-   All target subclouds have been prestaged with the software release and the
    new container images required for that release. See
    :ref:`orchestrate-subcloud-prestage-using-the-cli-eb516473582f` for
    instructions.

-   For the deployment of major software release, ensure that the controller-0
    is the active controller on each subcloud.

-   |Optional| **Only for Major Release** Upload the new software release to
    the central software release repository:

    .. code-block:: none

        ~(keystone_admin)]$ software --os-region-name SystemController upload <bootimage.iso> <bootimage.sig>

.. note::

    If the target patch release to be deployed on the subcloud is not the
    latest patch release already deployed on the System Controller, the
    subcloud will need to be prestaged again before deploying any subsequent
    releases.

.. note::

    The sysadmin and admin passwords must be set to the same value prior to
    starting an upgrade from |prod-long| Release |v_r9| to |prod| Release |v_r10|.

.. rubric:: |proc|

#.  Review software sync status of the subclouds.

    After new software release has been deployed on the System Controller, wait
    for approximately 60 seconds for the ``software_sync_status`` of all
    subclouds to be updated.

    To identify which subclouds are deploy-current (in-sync), use the
    :command:`dcmanager subcloud list` command. For example:

    .. code-block:: none

        ~(keystone_admin)]$ dcmanager subcloud list
        +----+-----------+--------------+--------------------+-------------+
        | id | name      | management   | availability       | sync        |
        +----+-----------+--------------+--------------------+-------------+
        |  1 | subcloud-1| managed      | online             | out-of-sync |
        |  2 | subcloud-2| managed      | online             | out-of-sync |
        |  3 | subcloud-3| managed      | online             | out-of-sync |
        |  4 | subcloud-4| managed      | online             | out-of-sync |
        +----+-----------+--------------+--------------------+-------------+

    To see synchronization details for a subcloud, use the following command:

    .. code-block:: none

        ~(keystone_admin)]$ dcmanager subcloud show subcloud1
        +-----------------------------+----------------------------------+
        | Field                       | Value                            |
        +-----------------------------+----------------------------------+
        | id                          | 2                                |
        | name                        | subcloud1-stx-latest             |
        | description                 | Ottawa Site                      |
        | location                    | YOW                              |
        | software_version            | 24.09                            |
        | management                  | unmanaged                        |
        | availability                | online                           |
        | deploy_status               | complete                         |
        | management_subnet           | 192.168.101.0/24                 |
        | management_start_ip         | 192.168.101.2                    |
        | management_end_ip           | 192.168.101.50                   |
        | management_gateway_ip       | 192.168.101.1                    |
        | systemcontroller_gateway_ip | 192.168.204.101                  |
        | group_id                    | 1                                |
        | peer_group_id               | None                             |
        | created_at                  | 2025-01-21 18:34:30.410350       |
        | updated_at                  | 2025-01-21 19:03:08.836619       |
        | backup_status               | None                             |
        | backup_datetime             | None                             |
        | prestage_status             | None                             |
        | prestage_versions           | None                             |
        | dc-cert_sync_status         | in-sync                          |
        | firmware_sync_status        | unknown                          |
        | identity_sync_status        | unknown                          |
        | kubernetes_sync_status      | unknown                          |
        | kube-rootca_sync_status     | unknown                          |
        | platform_sync_status        | unknown                          |
        | software_sync_status        | unknown                          |
        | region_name                 | 0aa48e9b72bf48bbaad9a624cdd0cacb |
        +-----------------------------+----------------------------------+

#.  To create a sw-deploy strategy, use the :command:`dcmanager sw-deploy-strategy create <release_id>`
    command.

    The sw-deploy strategy for a |prod-dc| system controls how updates/upgrades
    are applied to subclouds.

    .. code-block:: none

        ~(keystone_admin)]$ dcmanager sw-deploy-strategy create \
        [--subcloud-apply-type <type>] \
        [–-max-parallel-subclouds <i>] \
        [–-stop-on-failure <level>] \
        [--group group] \
        [--release-id RELEASE_ID] \
        [--snapshot] \
        [--rollback] \
        [--with-delete] \
        [--delete-only]  \
        [<subcloud>]

    where:

    **subcloud-apply-type**
        **parallel** or **serial**— determines whether the subclouds will be
        processed in parallel or serially.

        If this is not specified using the CLI, the values for
        :command:`subcloud_update_type` defined for each subcloud group will
        be used by default.

    **max-parallel-subclouds**
        Sets the maximum number of subclouds that can be upgraded in parallel
        (default 2).

        If this is not specified using the CLI, the values for
        :command:`max_parallel_subclouds` defined for each subcloud group
        will be used by default.

    **stop-on-failure**
        **true**\(default) or **false**— determines whether upgrade
        orchestration failure for a subcloud prevents application to subsequent
        subclouds.

    **group**
        Optionally pass the name or ID of a subcloud group to the
        :command:`dcmanager sw-deploy-strategy create` command. This results in a
        strategy that is only applied to all subclouds in the specified group.
        The subcloud group values are used for subcloud apply type and max
        parallel subclouds parameters.

    **snapshot**
        Creates a snapshot of the main logical volumes so that, in the case of
        a rollback, the snapshots can be restored in order to speed up the
        overall rollback procedure. By default, the rollback procedure does not
        use snapshots. Instead, it rolls back the individual file changes on
        the main logical volumes.

    **rollback**
        Specifies that :command:`sw-deploy-strategy` should run a rollback of the
        existing (un-deleted) software deployment on the subclouds.

        This is valid after a failed :command:`sw-deploy-strategy` on a
        subcloud or after a successful :command:`sw-deploy-strategy` on a
        subcloud that did not delete the software deployment using
        **--with-delete** or **--delete-only**.

    **with-delete**
        Specifies that the software deployments created on the subclouds as
        part of the strategy apply should be deleted at the end of a successful
        subcloud software deployment. With this option, the subclouds' software
        deployments cannot be rolled back.

        By default, the software deployment is not deleted, that is, a subcloud
        software deployment rollback is possible, if required.

    **delete-only**
        Specifies that :command:`sw-deploy-strategy` should only run the
        software deployment delete operation on subclouds. This command is used
        to clean up a previous default :command:`sw-deploy-strategy` where the
        subclouds' software deployments were not deleted.

    For example:

    .. only:: starlingx

        .. code-block:: none

            ~(keystone_admin)]$ dcmanager sw-deploy-strategy create --release-id
            +------------------------+----------------------------+
            | Field                  | Value                      |
            +------------------------+----------------------------+
            | strategy type          | sw-deploy                  |
            | subcloud apply type    | None                       |
            | max parallel subclouds | 2                          |
            | stop on failure        | False                      |
            | release_id             | starlingx-25.09.0          |
            | snapshot               | False                      |
            | rollback               | False                      |
            | delete_option          | None                       |
            | state                  | initial                    |
            | created_at             | 2025-08-21T22:30:53.282940 |
            | updated_at             | None                       |
            +------------------------+----------------------------+

    .. only:: partner

        .. include:: /_includes/deploy-software-releases-using-the-cli.rest
           :start-after: sw-deploy-strategy-create-begin
           :end-before: sw-deploy-strategy-create-end

#.  To show the settings for the ``sw-deploy-strategy``, use the
    :command:`dcmanager sw-deploy-strategy show` command.

    For example:

    .. code-block:: none

        ~(keystone_admin)]$ dcmanager sw-deploy-strategy show
        +------------------------+----------------------------+
        | Field                  | Value                      |
        +------------------------+----------------------------+
        | strategy type          | sw-deploy                  |
        | subcloud apply type    | parallel                   |
        | max parallel subclouds | 2                          |
        | stop on failure        | False                      |
        | state                  | initial                    |
        | created_at             | 2020-02-02T14:42:13.822499 |
        | updated_at             | None                       |
        +------------------------+----------------------------+

    .. note::
        The values for `subcloud apply type` and `max parallel subclouds` will
        be taken from the subcloud group if specified through the ``--group``
        parameter.

#.  Review the sw-deploy strategy for the subclouds.

    To show the subclouds that will be processed when the sw-deploy strategy is
    applied, use the :command:`dcmanager strategy-step list` command. For
    example:

    .. code-block:: none

        ~(keystone_admin)]$ dcmanager strategy-step list
        +----------------------+-------+---------+---------+------------+-------------+
        | cloud                | stage | state   | details | started_at | finished_at |
        +----------------------+-------+---------+---------+------------+-------------+
        | subcloud1-stx-latest |     1 | initial |         | None       | None        |
        +----------------------+-------+---------+---------+------------+-------------+

    .. note::

        All the subclouds that are included in the same stage will be deployed
        in parallel.

#.  To apply the software deploy strategy, use the :command:`dcmanager sw-deploy-strategy apply`
    command.

    .. only:: starlingx

        .. code-block:: none

            ~(keystone_admin)]$ dcmanager sw-deploy-strategy apply
            +------------------------+----------------------------+
            | Field                  | Value                      |
            +------------------------+----------------------------+
            | subcloud apply type    | None                       |
            | max parallel subclouds | 2                          |
            | stop on failure        | False                      |
            | release_id             | starlingx-25.09.0          |
            | snapshot               | False                      |
            | rollback               | False                      |
            | delete_option          | None                       |
            | state                  | applying                   |
            | created_at             | 2025-08-21T22:30:53.282940 |
            | updated_at             | None                       |
            +------------------------+----------------------------+

    .. only:: partner

        .. include:: /_includes/deploy-software-releases-using-the-cli.rest
           :start-after: sw-deploy-strategy-apply-begin
           :end-before: sw-deploy-strategy-apply-end

#.  To show the step currently being performed on each of the subclouds, use
    the :command:`dcmanager strategy-step list` command.

    For example:

    .. code-block:: none

        ~(keystone_admin)]$ dcmanager strategy-step list
        +------------------+--------+-----------------------+-------------------+----------------------------+----------------------------+
        | cloud            | stage  | state                 | details           | started_at                 | finished_at                |
        +------------------+--------+-----------------------+-------------------+----------------------------+----------------------------+
        | subcloud-1       | 3      | complete              |                   | 2024-11-06 12:57:20.342429 | 2024-11-06 12:57:41.054664 |
        | subcloud-4       | 2      | apply VIM sw-deploy   |                   | 2024-11-06 12:57:20.342429 | None                       |
        | subcloud-5       | 1      | initial               |                   | None                       | None                       |
        | subcloud-6       | 1      | initial               |                   | None                       | None                       |
        +------------------+--------+-------------+-----------------------------+----------------------------+----------------------------+

#.  To show the step currently being performed on a subcloud, use the
    :command:`dcmanager strategy-step show` <subcloud> command.

    .. code-block:: none

        ~(keystone_admin)]$ dcmanager strategy-step show <subcloud>

#.  When all the subclouds within the distributed software deploy orchestration
    indicate they have entered either the complete or failed state, delete the
    sw-deploy strategy, using the :command:`dcmanager sw-deploy-strategy delete`
    command.

    .. only:: starlingx

        .. code-block:: none

            ~(keystone_admin)]$ dcmanager sw-deploy-strategy delete
            +------------------------+----------------------------+
            | Field                  | Value                      |
            +------------------------+----------------------------+
            | subcloud apply type    | None                       |
            | max parallel subclouds | 2                          |
            | stop on failure        | False                      |
            | release_id             | starlingx-25.09.0          |
            | snapshot               | False                      |
            | rollback               | False                      |
            | delete_option          | None                       |
            | state                  | deleting                   |
            | created_at             | 2025-08-21T22:30:53.282940 |
            | updated_at             | None                       |
            +------------------------+----------------------------+

    .. only:: partner

        .. include:: /_includes/deploy-software-releases-using-the-cli.rest
           :start-after: sw-deploy-strategy-delete-begin
           :end-before: sw-deploy-strategy-delete-end

.. rubric:: |postreq|

If the subclouds are upgraded with a major software release, it is recommended
to also upversion Kubernetes right after the software deployment. See
:ref:`kubernetes-upversion-orchestration`.


Error Handling During an Orchestrated Subcloud Software Deployment
------------------------------------------------------------------

If a failure occurs, follow the general steps below:

#.  Allow the failed strategy to complete on its own.

#.  Check the output using :command:`dcmanager strategy-step list` command.

#.  To see the cause of failure for a subcloud, use the :command:`dcmanager subcloud errors <subcloud-name/subcloud-id>`
    command.

#.  Resolve the condition that caused the failure.

#.  Retry the orchestration by deleting the failed strategy and create a new
    one for the failed subcloud(s).