Kubernetes Upgrade Optimization - Simplex (ds8)
Added information on Kubernetes multi-version upgrade Change-Id: I45a984881a7b77c92ab41778963b2a05cd0e7707 Signed-off-by: Ngairangbam Mili <ngairangbam.mili@windriver.com>
This commit is contained in:
parent
f6dda4b80d
commit
50ef0a9b48
@ -0,0 +1,359 @@
|
||||
.. _configuring-kubernetes-multi-version-upgrade-orchestration-aio-b0b59a346466:
|
||||
|
||||
=========================================================================
|
||||
Configure Kubernetes Multi-Version Upgrade Cloud Orchestration for AIO-SX
|
||||
=========================================================================
|
||||
|
||||
You can configure Kubernetes multi-version upgrade orchestration strategy using
|
||||
the :command:`sw-manager` command. This feature is enabled from
|
||||
|prod| |k8s-multi-ver-orch-strategy-release| and is supported only for the |AIO-SX| system.
|
||||
|
||||
.. note::
|
||||
|
||||
You require administrator privileges to use the :command:`sw-manager` command.
|
||||
You must log in to the active controller as **user sysadmin** and source the script
|
||||
by using the :command:`source /etc/platform/openrc` command to obtain administrator
|
||||
privileges. Do not use :command:`sudo`.
|
||||
|
||||
.. note::
|
||||
|
||||
Management-affecting alarms cannot be ignored using relaxed alarm rules
|
||||
during an orchestrated Kubernetes version upgrade operation. For a list of
|
||||
management-affecting alarms, see |fault-doc|: :ref:`100-series-alarm-messages-starlingx`.
|
||||
To display management-affecting active alarms, use the :command:`fm alarm-list --mgmt_affecting`
|
||||
command.
|
||||
|
||||
During an orchestrated Kubernetes version upgrade operation, the following
|
||||
alarms are ignored even when the default strict restrictions are selected:
|
||||
|
||||
.. _noc1590162360081-ul-vhg-jxs-tlb:
|
||||
|
||||
- 100.103: Memory threshold exceeded
|
||||
|
||||
- 200.001: Locked host
|
||||
|
||||
- 280.001: Subcloud resource off-line
|
||||
|
||||
- 280.002: Subcloud resource out-of-sync
|
||||
|
||||
- 700.004: |VM| stopped
|
||||
|
||||
- 750.006: Configuration change requires reapply of cert-manager
|
||||
|
||||
- 900.001: Patch in progress
|
||||
|
||||
- 900.007: Kube upgrade in progress
|
||||
|
||||
- 900.401: kube-upgrade-auto-apply-inprogress
|
||||
|
||||
|
||||
You can use ``help`` for the overall commands and also for each sub-command.
|
||||
|
||||
For example:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ sw-manager kube-upgrade-strategy –help
|
||||
usage: sw-manager kube-upgrade-strategy [-h] ...
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
Kubernetes Update Commands:
|
||||
create Create a strategy
|
||||
delete Delete a strategy
|
||||
apply Apply a strategy
|
||||
abort Abort a strategy
|
||||
show Show a strategy
|
||||
|
||||
.. rubric:: |prereq|
|
||||
|
||||
.. _noc1590162360081-ul-ls2-pxs-tlb:
|
||||
|
||||
- Hosts that need to be upgraded must be in the **unlocked-enabled** state.
|
||||
|
||||
- If you are using NetApp Trident, ensure that your NetApp version is
|
||||
compatible with Trident 22.07 before upgrading Kubernetes to version
|
||||
|kube-ver| and after updating |prod| to version |prod-ver|. For more
|
||||
information, see :ref:`upgrade-the-netapp-trident-software-c5ec64d213d3`.
|
||||
|
||||
.. only:: partner
|
||||
|
||||
.. include:: /_includes/configuring-kubernetes-update-orchestration.rest
|
||||
|
||||
.. rubric:: |proc|
|
||||
|
||||
#. List available upgrades.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ system kube-version-list
|
||||
+-----------------+--------+-------------+
|
||||
| Version | Target | State |
|
||||
+-----------------+--------+-------------+
|
||||
| v1.21.8 | True | active |
|
||||
| v1.22.5 | False | available |
|
||||
| v1.23.1 | False | available |
|
||||
| v1.24.4 | False | available |
|
||||
+-----------------+--------+-------------+
|
||||
|
||||
#. Confirm that the system is healthy.
|
||||
|
||||
Check the current system health status, resolve any alarms and other issues
|
||||
reported by the :command:`system health-query-kube-upgrade` command. Then,
|
||||
recheck the system health status to confirm that all **System Health**
|
||||
fields are set to **OK**.
|
||||
|
||||
By default, the upgrade process cannot be run and is not recommended to be
|
||||
run with active alarms present. Use the :command:`system kube-upgrade-start --force`
|
||||
command to force the upgrade process to start and ignore non-management-affecting
|
||||
alarms.
|
||||
|
||||
.. note::
|
||||
|
||||
It is strongly recommended that you clear your system of all alarms
|
||||
before doing an upgrade. While the :command:`--force` option is
|
||||
available to run the upgrade, it is a best practice to clear any
|
||||
alarms.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system health-query-kube-upgrade
|
||||
System Health:
|
||||
All hosts are provisioned: [OK]
|
||||
All hosts are unlocked/enabled: [OK]
|
||||
All hosts have current configurations: [OK]
|
||||
All hosts are patch current: [OK]
|
||||
No alarms: [OK]
|
||||
All kubernetes nodes are ready: [OK]
|
||||
All kubernetes control plane pods are ready: [OK]
|
||||
All kubernetes applications are in a valid state: [OK]
|
||||
|
||||
#. Create the strategy.
|
||||
|
||||
The Kubernetes multi-version upgrade orchestration strategy :command:`create`
|
||||
command creates a series of stages with steps that apply the Kubernetes
|
||||
version upgrade.
|
||||
|
||||
Specify the desired Kubernetes version in ``--to-version``
|
||||
(usually the highest version available in the system).
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ sw-manager kube-upgrade-strategy create --to-version v1.24.4
|
||||
Strategy Kubernetes Upgrade Strategy:
|
||||
strategy-uuid: f03f5944-ee79-4047-8d2e-68bfa6775210
|
||||
controller-apply-type: serial
|
||||
storage-apply-type: serial
|
||||
worker-apply-type: serial
|
||||
default-instance-action: stop-start
|
||||
alarm-restrictions: strict
|
||||
current-phase: build
|
||||
current-phase-completion: 0%
|
||||
state: building
|
||||
inprogress: true
|
||||
|
||||
where:
|
||||
|
||||
``--to-version``
|
||||
The version of Kubernetes to upgrade to, for example, ``v1.24.4``.
|
||||
This argument is required.
|
||||
|
||||
``--controller-apply-type`` and ``--storage-apply-type``
|
||||
These options cannot be changed from ``serial`` because Kubernetes
|
||||
upgrade concurrency is only supported for worker hosts.
|
||||
|
||||
.. note::
|
||||
|
||||
Setting the Kubernetes version upgrade apply type is supported only
|
||||
for hosts with the worker function. Any attempt to modify the
|
||||
controller or storage apply type will be rejected.
|
||||
|
||||
``--worker-apply-type``
|
||||
This option specifies the host concurrency of the Kubernetes version
|
||||
upgrade strategy:
|
||||
|
||||
- serial (default): worker hosts will be patched one at a time
|
||||
|
||||
- parallel: worker hosts will be upgraded in parallel
|
||||
|
||||
- At most, ``parallel`` will be upgraded at the same time
|
||||
|
||||
- At most, half of the hosts in a host aggregate will be upgraded
|
||||
at the same time
|
||||
|
||||
- ignore: worker hosts will not be upgraded; strategy create will fail
|
||||
|
||||
Worker hosts with no instances are upgraded before worker hosts with
|
||||
instances.
|
||||
|
||||
``--max-parallel-worker-hosts``
|
||||
This option applies to the parallel worker apply type selection to
|
||||
specify the maximum worker hosts to upgrade in parallel (minimum: 2, maximum: 10).
|
||||
|
||||
``--alarm-restrictions``
|
||||
This option sets how the Kubernetes version upgrade orchestration
|
||||
behaves when alarms are present.
|
||||
|
||||
To display management-affecting active alarms, use the :command:`fm alarm-list --mgmt_affecting` command.
|
||||
|
||||
``strict`` (default)
|
||||
The default strict option will result in the failure of patch orchestration if
|
||||
there are any alarms present in the system (except for a small list of alarms).
|
||||
|
||||
``relaxed``
|
||||
This option allows orchestration to proceed even if alarms are present, as
|
||||
long as none of these alarms are management affecting.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ sw-manager kube-upgrade-strategy create --help
|
||||
usage:sw-manager kube-upgrade-strategy [-h]
|
||||
--to-version <kubernetesVersion>
|
||||
[--controller-apply-type {ignore}]
|
||||
[--storage-apply-type {ignore}]
|
||||
[--worker-apply-type
|
||||
{serial,parallel,ignore}]
|
||||
[--max-parallel-worker-hosts
|
||||
{2,3,4,5,6,7,8,9,10}]
|
||||
[--instance-action {migrate,stop-start}]
|
||||
[--alarm-restrictions {strict,relaxed}]
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
--controller-apply-type {serial,ignore}
|
||||
defaults to serial
|
||||
--storage-apply-type {serial,ignore}
|
||||
defaults to serial
|
||||
--worker-apply-type {serial,parallel,ignore}
|
||||
defaults to serial
|
||||
--max-parallel-worker-hosts {2,3,4,5,6,7,8,9,10}
|
||||
maximum worker hosts to update in parallel
|
||||
--instance-action {migrate,stop-start}
|
||||
defaults to stop-start
|
||||
--alarm-restrictions {strict,relaxed}
|
||||
defaults to strict
|
||||
|
||||
|
||||
#. |optional| Display the strategy in summary, if required. The Kubernetes
|
||||
upgrade strategy :command:`show` command displays the strategy in a summary.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ sw-manager kube-upgrade-strategy show
|
||||
Strategy Kubernetes Upgrade Strategy:
|
||||
strategy-uuid: f03f5944-ee79-4047-8d2e-68bfa6775210
|
||||
controller-apply-type: serial
|
||||
storage-apply-type: serial
|
||||
worker-apply-type: serial
|
||||
default-instance-action: stop-start
|
||||
alarm-restrictions: strict
|
||||
current-phase: build
|
||||
current-phase-completion: 100%
|
||||
state: ready-to-apply
|
||||
build-result: success
|
||||
build-reason:
|
||||
|
||||
The :command:`show` strategy subcommand displays a summary of the current
|
||||
state of the strategy. A complete view of the strategy can be shown using
|
||||
the ``--details`` option.
|
||||
|
||||
The strategy steps and stages are displayed using the ``--details`` option.
|
||||
|
||||
#. Apply the strategy.
|
||||
|
||||
Kubernetes multi-version upgrade orchestration strategy :command:`apply` command
|
||||
runs the strategy stages and steps consecutively until the Kubernetes
|
||||
upgrade on all the hosts in the strategy is completed.
|
||||
|
||||
|
||||
- Use the ``-stage-id`` option to specify a specific stage to apply one
|
||||
at a time.
|
||||
|
||||
.. note::
|
||||
|
||||
When applying a single stage, only the next stage will be applied.
|
||||
You cannot skip stages.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ sw-manager kube-upgrade-strategy apply
|
||||
Strategy Kubernetes upgrade Strategy:
|
||||
strategy-uuid: f03f5944-ee79-4047-8d2e-68bfa6775210
|
||||
controller-apply-type: serial
|
||||
storage-apply-type: serial
|
||||
worker-apply-type: serial
|
||||
default-instance-action: stop-start
|
||||
alarm-restrictions: strict
|
||||
current-phase: apply
|
||||
current-phase-completion: 0%
|
||||
state: applying
|
||||
inprogress: true
|
||||
|
||||
- Use the :command:`kube-upgrade-show` command to monitor Kubernetes
|
||||
upgrade state and percentage completion.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ system kube-upgrade-show
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 1272e9cc-1a15-4a53-bb5a-d47494729068 |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | downloading-images |
|
||||
| created_at | 2023-09-25T18:32:10.820488+00:00 |
|
||||
| updated_at | 2023-09-25T18:32:10.885709+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
You will see the ``state`` property transition through values, such as
|
||||
``downloading-images``, ``downloaded-images``, ``upgraded-networking``,
|
||||
and ``upgraded-first-master``.
|
||||
|
||||
#. |optional| Abort the strategy, if required. This is only used to stop and
|
||||
abort the entire strategy.
|
||||
|
||||
The Kubernetes version upgrade strategy :command:`abort` command can be
|
||||
used to abort the Kubernetes version upgrade strategy after the current
|
||||
step of the currently applying stage is completed.
|
||||
|
||||
#. Confirm that the upgrade has completed successfully.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ system kube-upgrade-show
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 1272e9cc-1a15-4a53-bb5a-d47494729068 |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | upgrade-complete |
|
||||
| created_at | 2023-09-25T18:52:10.885709+00:00 |
|
||||
| updated_at | 2023-09-25T18:52:11.673259+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
~(keystone_admin)$ system kube-version-list
|
||||
+-----------------+--------+-------------+
|
||||
| Version | Target | State |
|
||||
+-----------------+--------+-------------+
|
||||
| v1.21.8 | False | unavailable |
|
||||
| v1.22.5 | False | unavailable |
|
||||
| v1.23.1 | False | unavailable |
|
||||
| v1.24.4 | True | active |
|
||||
+-----------------+--------+-------------+
|
||||
|
||||
#. Delete the strategy.
|
||||
|
||||
.. note::
|
||||
|
||||
After the Kubernetes multi-version upgrade orchestration strategy has been
|
||||
applied (or aborted), it must be deleted before another Kubernetes
|
||||
version upgrade strategy can be created. If a Kubernetes version
|
||||
upgrade strategy application fails, you must address the issue that
|
||||
caused the failure, then delete and re-create the strategy before
|
||||
attempting to apply it again.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ sw-manager kube-upgrade-strategy delete
|
||||
Strategy deleted.
|
@ -74,6 +74,7 @@ Manual Kubernetes Version Upgrade
|
||||
:maxdepth: 1
|
||||
|
||||
manual-kubernetes-components-upgrade
|
||||
manual-kubernetes-multi-version-upgrade-in-aio-sx-13e05ba19840
|
||||
|
||||
----------------------------------------------
|
||||
Kubernetes Version Upgrade Cloud Orchestration
|
||||
@ -85,6 +86,7 @@ Kubernetes Version Upgrade Cloud Orchestration
|
||||
about-kubernetes-orchestrated-upgrades
|
||||
the-kubernetes-update-orchestration-process
|
||||
configuring-kubernetes-update-orchestration
|
||||
configuring-kubernetes-multi-version-upgrade-orchestration-aio-b0b59a346466
|
||||
handling-kubernetes-update-orchestration-failures
|
||||
|
||||
----------------------------------
|
||||
|
@ -0,0 +1,422 @@
|
||||
.. _manual-kubernetes-multi-version-upgrade-in-aio-sx-13e05ba19840:
|
||||
|
||||
=================================================
|
||||
Manual Kubernetes Multi-Version Upgrade in AIO-SX
|
||||
=================================================
|
||||
|
||||
You can upgrade the Kubernetes version on a running system from one supported
|
||||
version to another.
|
||||
|
||||
|AIO-SX| now supports multi-version Kubernetes upgrades. In this model,
|
||||
Kubernetes is upgraded by two or more versions after disabling applications and
|
||||
then applications are enabled again. This is faster than upgrading Kubernetes
|
||||
one version at a time. Also, the upgrade can be aborted and reverted to the
|
||||
original version. This feature is supported only for |AIO-SX|.
|
||||
|
||||
|AIO-SX| supports the Kubernetes multi-version upgrade. Thus, Kubernetes can be
|
||||
upgraded from the lowest version to the highest version available in the
|
||||
system. This feature is not supported in the system which is not |AIO-SX|.
|
||||
|
||||
.. note::
|
||||
|
||||
Each |prod| release supports two or more consecutive Kubernetes releases.
|
||||
The default version on a fresh install will always be the latest Kubernetes
|
||||
release supported by a |prod| release. Upgrades from previous releases will
|
||||
always start with the same Kubernetes version as the highest version from
|
||||
the release you are upgrading from.
|
||||
|
||||
.. rubric:: |prereq|
|
||||
|
||||
- The system must be clear of alarms.
|
||||
|
||||
- All hosts must be unlocked, enabled, and available.
|
||||
|
||||
- All Kubernetes pods must be ready.
|
||||
|
||||
- The installed applications must be compatible with the new Kubernetes
|
||||
versions that the system will upgrade to.
|
||||
|
||||
- If you are using NetApp Trident in |prod| |prod-ver| and if you have
|
||||
upgraded from the |prod| previous version, ensure that your NetApp backend
|
||||
version is compatible with Trident 22.07. Follow the steps in
|
||||
:ref:`upgrade-the-netapp-trident-software-c5ec64d213d3` to upgrade Trident
|
||||
to 22.07 before upgrading Kubernetes to version 1.24.
|
||||
|
||||
.. rubric:: |proc|
|
||||
|
||||
#. List the available Kubernetes versions.
|
||||
|
||||
On a fresh install of |prod| |prod-ver|, the following output appears:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-version-list
|
||||
+---------+--------+-------------+
|
||||
| Version | Target | State |
|
||||
+---------+--------+-------------+
|
||||
| v1.21.8 | False | unavailable |
|
||||
| v1.22.5 | False | unavailable |
|
||||
| v1.23.1 | False | unavailable |
|
||||
| v1.24.4 | True | active |
|
||||
+---------+--------+-------------+
|
||||
|
||||
If |prod| was upgraded to |prod-ver|, the following output appears:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-version-list
|
||||
+---------+--------+-------------+
|
||||
| Version | Target | State |
|
||||
+---------+--------+-------------+
|
||||
| v1.21.8 | True | active |
|
||||
| v1.22.5 | False | available |
|
||||
| v1.23.1 | False | available |
|
||||
| v1.24.4 | False | available |
|
||||
+---------+--------+-------------+
|
||||
|
||||
The following meanings apply to the output shown:
|
||||
|
||||
**Target**
|
||||
|
||||
Target is either true or false. Target will be true only for the active
|
||||
Kubernetes version.
|
||||
|
||||
**State**
|
||||
|
||||
State can be one of the following:
|
||||
|
||||
*active*: The version is running everywhere.
|
||||
|
||||
*partial*: The version is running somewhere.
|
||||
|
||||
*available*: The version can be upgraded.
|
||||
|
||||
*unavailable*: The version is not available for upgrading.
|
||||
|
||||
#. Confirm that the system is healthy.
|
||||
|
||||
Check the current system health status, resolve any alarms and other issues
|
||||
reported by the :command:`system health-query-kube-upgrade` command, and
|
||||
recheck the system health status to confirm that all the **System Health**
|
||||
fields are set to *OK*.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system health-query-kube-upgrade
|
||||
System Health:
|
||||
All hosts are provisioned: [OK]
|
||||
All hosts are unlocked/enabled: [OK]
|
||||
All hosts have current configurations: [OK]
|
||||
All hosts are patch current: [OK]
|
||||
Ceph Storage Healthy: [OK]
|
||||
No alarms: [OK]
|
||||
All kubernetes nodes are ready: [OK]
|
||||
All kubernetes control plane pods are ready: [OK]
|
||||
Required patches are applied: [OK]
|
||||
License valid for upgrade: [OK]
|
||||
All kubernetes applications are in a valid state: [OK]
|
||||
Active controller is controller-0: [OK]
|
||||
|
||||
#. Start the Kubernetes multi-version upgrade.
|
||||
|
||||
Specify the desired target version available to upgrade.
|
||||
|
||||
For example:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-start 1.24.4
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | upgrade-started |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
The upgrade process checks the applied/available updates, the upgrade
|
||||
path, the system health, the installed applications compatibility, and
|
||||
validates that the system is ready for an upgrade.
|
||||
|
||||
.. warning::
|
||||
The command :command:`system kube-upgrade-start --force` causes the
|
||||
upgrade process to ignore non-management-affecting alarms.
|
||||
Kubernetes cannot be upgraded if there are management-affecting alarms.
|
||||
|
||||
#. Download the Kubernetes images.
|
||||
|
||||
For example:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-download-images
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 065e683a-13a3-4229-b3c7-701f90216a3d |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | downloading-images |
|
||||
| created_at | 2023-08-24T02:33:47.049826+00:00 |
|
||||
| updated_at | None |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
#. Confirm that the download has completed.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-show
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 065e683a-13a3-4229-b3c7-701f90216a3d |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | downloaded-images |
|
||||
| created_at | 2023-08-24T02:33:47.049826+00:00 |
|
||||
| updated_at | 2023-08-24T02:38:16.374677+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
#. Upgrade Kubernetes networking.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-networking
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | bf3f9c80-0cec-49a0-91ef-dd86c9bb8fe8 |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | upgrading-networking |
|
||||
| created_at | 2023-08-24T02:33:47.049826+00:00 |
|
||||
| updated_at | 2023-08-24T02:38:16.374677+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
The state **upgraded-networking** will be entered when the networking
|
||||
upgrade has completed.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-show
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 065e683a-13a3-4229-b3c7-701f90216a3d |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | upgraded-networking |
|
||||
| created_at | 2023-08-24T02:33:47.049826+00:00 |
|
||||
| updated_at | 2023-08-24T02:42:40.543522+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
#. |optional| Cordon
|
||||
|
||||
The :command:`kube-host-cordon` command will evict the regular pods from
|
||||
the host. This command will prevent the application from running on
|
||||
intermediate versions.
|
||||
|
||||
.. note::
|
||||
|
||||
This command will permanently evict the pods which are not in namespaces.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-host-cordon controller-0
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 065e683a-13a3-4229-b3c7-701f90216a3d |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | cordon-started |
|
||||
| created_at | 2023-08-24T02:45:32.257231+00:00 |
|
||||
| updated_at | 2023-08-24T02:45:32.257231+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-show
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 065e683a-13a3-4229-b3c7-701f90216a3d |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | cordon-complete |
|
||||
| created_at | 2023-08-24T02:45:32.257231+00:00 |
|
||||
| updated_at | 2023-08-24T11:47:56.178266+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
The state **cordon-complete** will be entered when the host cordon has
|
||||
completed.
|
||||
|
||||
#. Upgrade the control plane on controller-0.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-host-upgrade controller-0 control-plane
|
||||
+-----------------------+-------------------------+
|
||||
| Property | Value |
|
||||
+-----------------------+-------------------------+
|
||||
| control_plane_version | v1.21.8 |
|
||||
| hostname | controller-0 |
|
||||
| id | 1 |
|
||||
| kubelet_version | v1.21.8 |
|
||||
| personality | controller |
|
||||
| status | upgrading-control-plane |
|
||||
| target_version | v1.24.4 |
|
||||
+-----------------------+-------------------------+
|
||||
|
||||
Check if the control plane version upgrade status is changed to *None*.
|
||||
This verifies that the control plane has been successfully upgraded to the next
|
||||
version.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-host-upgrade-list
|
||||
+----+---------------+------------+----------------+-----------------------+-----------------+--------------+
|
||||
| id | hostname | personality| target_version | control_plane_version | kubelet_version | status |
|
||||
+----+---------------+------------+----------------+-----------------------+-----------------+--------------+
|
||||
| 1 | controller-0 | controller | v1.22.5 | v1.22.5 | v1.21.8 | None |
|
||||
+----+---------------+---+--------+----------------+-----------------------+-----------------+--------------+
|
||||
|
||||
#. Upgrade kubelet on controller-0.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-host-upgrade controller-0 kubelet
|
||||
+-----------------------+-------------------------+
|
||||
| Property | Value |
|
||||
+-----------------------+-------------------------+
|
||||
| control_plane_version | v1.22.5 |
|
||||
| hostname | controller-0 |
|
||||
| id | 1 |
|
||||
| kubelet_version | v1.21.8 |
|
||||
| personality | controller |
|
||||
| status | upgrading-kubelet |
|
||||
| target_version | v1.22.5 |
|
||||
+-----------------------+-------------------------+
|
||||
|
||||
Check the status of the kubelet upgrade.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-host-upgrade-list
|
||||
+----+---------------+------------+----------------+-----------------------+-----------------+------------------+
|
||||
| id | hostname | personality| target_version | control_plane_version | kubelet_version | status |
|
||||
+----+---------------+------------+----------------+-----------------------+-----------------+------------------+
|
||||
| 1 | controller-0 | controller | v1.22.5 | v1.22.5 | v1.22.5 | upgraded-kubelet |
|
||||
+----+---------------+---+--------+----------------+-----------------------+-----------------+------------------+
|
||||
|
||||
The status **upgraded-kubelet** will be entered when the kubelet upgrade
|
||||
has completed.
|
||||
|
||||
Repeat steps 9 and 10 to reach the target Kubernetes version. For example, in
|
||||
this case, we need to repeat steps 9 and 10 twice for the remaining
|
||||
versions v1.23.1 and v1.24.4.
|
||||
|
||||
#. |optional| Uncordon
|
||||
|
||||
Skip this step if you did not perform step 8.
|
||||
|
||||
The :command:`kube-host-uncordon` command will allow the regular pods on the
|
||||
host again.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-host-uncordon controller-0
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 065e683a-13a3-4229-b3c7-701f90216a3d |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | uncordon-started |
|
||||
| created_at | 2023-08-24T11:56:56.178266+00:00 |
|
||||
| updated_at | 2023-08-24T11:56:56.178266+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-show
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 065e683a-13a3-4229-b3c7-701f90216a3d |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | uncordon-complete |
|
||||
| created_at | 2023-08-24T11:56:56.178266+00:00 |
|
||||
| updated_at | 2023-08-24T11:58:35.136866+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
The state **uncordon-complete** will be entered when the host uncordon has
|
||||
completed.
|
||||
|
||||
#. Complete the Kubernetes upgrade.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-complete
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 065e683a-13a3-4229-b3c7-701f90216a3d |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | upgrade-complete |
|
||||
| created_at | 2023-08-24T02:33:47.049826+00:00 |
|
||||
| updated_at | 2023-08-24T02:55:18.122620+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
#. Remove the alarm 900.007 (Kubernetes upgrade in progress) if it is still
|
||||
running after the upgrade.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-delete
|
||||
|
||||
------------------------
|
||||
Kubernetes Upgrade Abort
|
||||
------------------------
|
||||
|
||||
If you want to abort the Kubernetes upgrade after the upgrade has started, run
|
||||
the following command:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-abort
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 065e683a-13a3-4229-b3c7-701f90216a3d |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.22.4 |
|
||||
| state | upgrade-aborting |
|
||||
| created_at | 2023-06-26T18:44:46.854319+00:00 |
|
||||
| updated_at | 2023-08-24T02:55:18.122620+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
To check the status of the abort operation, run the following command:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ system kube-upgrade-show
|
||||
+--------------+--------------------------------------+
|
||||
| Property | Value |
|
||||
+--------------+--------------------------------------+
|
||||
| uuid | 065e683a-13a3-4229-b3c7-701f90216a3d |
|
||||
| from_version | v1.21.8 |
|
||||
| to_version | v1.24.4 |
|
||||
| state | upgrade-aborted |
|
||||
| created_at | 2023-08-24T07:10:02.578787+00:00 |
|
||||
| updated_at | 2023-08-24T07:24:00.429794+00:00 |
|
||||
+--------------+--------------------------------------+
|
||||
|
||||
.. note::
|
||||
|
||||
- The upgrade abort operation reverts all the Kubernetes version upgrades and
|
||||
shows the same state the Kubernetes was in before the upgrade started.
|
||||
|
||||
- Once the Kubernetes upgrade is completed, it cannot be aborted.
|
Loading…
Reference in New Issue
Block a user