Removed CMK

CMK topics removed from usertasks and adminstasks sections.
Updates based on base review comments.
Patchset 2 update

Signed-off-by: Ron Stone <ronald.stone@windriver.com>
Change-Id: I8f7d20a4b79126c26cf86489259831bd2763c37b
Signed-off-by: Ron Stone <ronald.stone@windriver.com>
This commit is contained in:
Ron Stone 2021-05-25 13:49:01 -04:00
parent ef1c5ac068
commit 5c3f0f5df4
13 changed files with 26 additions and 547 deletions

View File

@ -0,0 +1,5 @@
.. usage-limitation-begin
.. usage-limitation-end
.. changes-relative-to-root-begin
.. changes-relative-to-root-end

View File

@ -1,17 +0,0 @@
.. pdb1561551141102
.. _about-cpu-manager-for-kubernetes:
================================
About CPU Manager for Kubernetes
================================
The CPU Manager for Kubernetes \(CMK\) feature provides cooperative management
of CPU affinity for Kubernetes workloads requiring predictable performance.
For more information about CMK, see the project page at `https://github.com/intel/CPU-Manager-for-Kubernetes <https://github.com/intel/CPU-Manager-for-Kubernetes>`__.
.. note::
The installation instructions on the CMK project page are incomplete.
Refer instead to :ref:`Install and Run CPU Manager for Kubernetes <installing-and-running-cpu-manager-for-kubernetes>`.

View File

@ -1,35 +0,0 @@
======================================
|prod-long| Kubernetes Admin Tutorials
======================================
- :ref:`About the Admin Tutorials <about-the-admin-tutorials>`
- Application Management
- :ref:`Helm Package Manager <kubernetes-admin-tutorials-helm-package-manager>`
- :ref:`StarlingX Application Package Manager <kubernetes-admin-tutorials-tarlingx-application-package-manager>`
- :ref:`Application Commands and Helm Overrides <application-commands-and-helm-overrides>`
- Local Docker Registry
- :ref:`Local Docker Registry <local-docker-registry>`
- :ref:`Authentication and Authorization <kubernetes-admin-tutorials-authentication-and-authorization>`
- :ref:`Installing/Updating the Docker Registry Certificate <installing-updating-the-docker-registry-certificate>`
- :ref:`Setting up a Public Repository <setting-up-a-public-repository>`
- :ref:`Freeing Space in the Local Docker Registry <freeing-space-in-the-local-docker-registry>`
- Optimizing Application Performance
- :ref:`Kubernetes CPU Manager Policies <kubernetes-cpu-manager-policies>`
- :ref:`Isolating CPU Cores to Enhance Application Performance <isolating-cpu-cores-to-enhance-application-performance>`
- :ref:`Kubernetes Topology Manager Policies <kubernetes-topology-manager-policies>`
- Intel's CPU Manager for Kubernetes \(CMK\)
- :ref:`About CPU Manager for Kubernetes <about-cpu-manager-for-kubernetes>`
- :ref:`Installing and Running CPU Manager for Kubernetes <installing-and-running-cpu-manager-for-kubernetes>`
- :ref:`Removing CPU Manager for Kubernetes <removing-cpu-manager-for-kubernetes>`
- :ref:`Uninstalling CPU Manager for Kubernetes on IPv6 <uninstalling-cpu-manager-for-kubernetes-on-ipv6>`

View File

@ -51,24 +51,13 @@ Optimize application performance
isolating-cpu-cores-to-enhance-application-performance
kubernetes-topology-manager-policies
--------------------------
CPU Manager for Kubernetes
--------------------------
.. toctree::
:maxdepth: 1
about-cpu-manager-for-kubernetes
installing-and-running-cpu-manager-for-kubernetes
removing-cpu-manager-for-kubernetes
uninstalling-cpu-manager-for-kubernetes-on-ipv6
--------------
Metrics Server
--------------
.. toctree::
:maxdepth: 1
kubernetes-admin-tutorials-metrics-server

View File

@ -1,239 +0,0 @@
.. jme1561551450093
.. _installing-and-running-cpu-manager-for-kubernetes:
==========================================
Install and Run CPU Manager for Kubernetes
==========================================
You must install Helm charts and label worker nodes appropriately before using
CMK.
.. rubric:: |context|
Perform the following steps to enable CMK on a cluster.
.. rubric:: |proc|
#. Apply the **cmk-node** label to each worker node to be managed using CMK.
For example:
.. code-block:: none
~(keystone)admin)$ system host-lock worker-0
~(keystone)admin)$ system host-label-assign worker-0 cmk-node=enabled
+-------------+--------------------------------------+
| Property | Value |
+-------------+--------------------------------------+
| uuid | 2909d775-cd6c-4bc1-8268-27499fe38d5e |
| host_uuid | 1f00d8a4-f520-41ee-b608-1b50054b1cd8 |
| label_key | cmk-node |
| label_value | enabled |
+-------------+--------------------------------------+
~(keystone)admin)$ system host-unlock worker-0
#. Perform the following steps if you have not specified CMK at Ansible
Bootstrap in the localhost.yml file:
#. On the active controller, run the following command to generate the
username and password to be used for Docker login.
This command generates the username and password to be used for Docker
login.
.. code-block:: none
$ sudo python /usr/share/ansible/stx-ansible/playbooks/roles/common/push-docker-images/files/get_registry_auth.py 625619392498.dkr.ecr.us-west-2.amazonaws.com <Access_Key_ID_from_Wind_Share> <Secret_Access_Key_from_Wind_Share>
#. Run the Docker login command:
.. code-block:: none
~(keystone)admin)$ sudo docker login 625619392498.dkr.ecr.us-west-2.amazonaws.com -u AWS -p <password_returned_from_first_cmd>
#. Pull the CMK image from the AWS registry.
.. code-block:: none
~(keystone)admin)$ sudo docker pull 625619392498.dkr.ecr.us-west-2.amazonaws.com/docker.io/wind-river/cmk:WRCP.20.01-v1.3.1-15-ge3df769-1
#. Tag the image, by using the following command:
.. code-block:: none
~(keystone)admin)$ sudo docker image tag 625619392498.dkr.ecr.us-west-2.amazonaws.com/docker.io/wind-river/cmk:WRCP.20.01-v1.3.1-15-ge3df769-1 registry.local:9001/docker.io/wind-river/cmk:WRCP.20.01-v1.3.1-15-ge3df769-1
#. Authenticate the local registry, by using the following command:
.. code-block:: none
~(keystone)admin)$ sudo docker login registry.local:9001 -u admin -p <admin_passwd>
#. Push the image, by using the following command:
.. code-block:: none
~(keystone)admin)$ sudo docker image push registry.local:9001/docker.io/wind-river/cmk:WRCP.20.01-v1.3.1-15-ge3df769-1
#. On all configurations with two controllers, after the CMK Docker image has
been pulled, tagged \(with the local registry\), and pushed \(to the local
registry\), the admin user should log in to the inactive controller and run
the following commands:
For example:
.. code-block:: none
~(keystone)admin)$ sudo docker login registry.local:9001 -u admin -p <admin_passwd>
~(keystone)admin)$ sudo docker image pull tis-lab-registry.cumulus.wrs.com:9001/wrcp-staging/docker.io/wind-river/cmk:WRCP.20.01-v1.3.1-15-ge3df769-1
#. Configure any isolated CPUs on worker nodes in order to reduce host OS
impacts on latency for tasks running on Isolated CPUs.
Any container tasks running on isolated CPUs will have to explicitly manage
their own affinity, the process scheduler will ignore them completely.
.. note::
The following commands are examples only, the admin user must specify
the number of CPUs per processor based on the node CPU topology.
.. code-block:: none
~(keystone)admin)$ system host-lock worker-1
~(keystone)admin)$ system host-cpu-modify -f platform -p0 1 worker-1
~(keystone)admin)$ system host-cpu-modify -f application-isolated -p0 15 worker-1
~(keystone)admin)$ system host-cpu-modify -f application-isolated -p1 15 worker-1
~(keystone)admin)$ system host-unlock worker-1
This sets one platform core and 15 application-isolated cores on NUMA node
0, and 15 application-isolated cores on NUMA node 1. At least one CPU must
be left unspecified, which will cause it to be an application CPU.
#. Run the /opt/extracharts/cpu-manager-k8s-setup.sh helper script to install
the CMK Helm charts used to configure the system for CMK.
#. Before running this command, untar files listed in /opt/extracharts.
.. code-block:: none
~(keystone)admin)$ cd /opt/extracharts
~(keystone)admin)$ sudo tar -xvf cpu-manager-k8s-init-1.3.1.tgz
~(keystone)admin)$ sudo tar -xvf cpu-manager-k8s-webhook-1.3.1.tgz
~(keystone)admin)$ sudo tar -xvf cpu-manager-k8s-1.3.1.tgz
#. Run the script.
The script is located in the /opt/extracharts directory of the active
controller.
For example:
.. code-block:: none
~(keystone)admin)$ cd /opt/extracharts
~(keystone)admin)$ ./cpu-manager-k8s-setup.sh
The following actions are performed:
- The **cpu-manager-k8s-init** chart is installed. This will create a
service account and set up rules-based access control.
- A webhook is created to insert the appropriate resources into pods
that request CMK resources. \(This will result in one pod running.\)
- A daemonset is created for the per-CMK-node pod that will handle
all CMK operations on that node.
- **cmk-webhook-deployment** is launched on the controller and
**cpu-manager-k8s-cmk-default** is launched on the worker.
By default, each node will have one available CPU allocated to the
shared pool, and all the rest allocated to the exclusive pool. The
platform CPUs will be ignored.
#. Add more CPUs to the shared pool.
#. Override the allocation via per-node Helm chart overrides on the
**cpu-manager-k8s** Helm chart.
.. code-block:: none
$ cat <<EOF > /home/sysadmin/worker-0-cmk-overrides.yml
# For NUM_EXCLUSIVE_CORES a value of -1 means
# "all available cores after infra and shared
# cores have been allocated".
# NUM_SHARED_CORES must be at least 1.
conf:
cmk:
NUM_EXCLUSIVE_CORES: -1
NUM_SHARED_CORES: 1
overrides:
cpu-manager-k8s_cmk:
hosts:
- name: worker-0
conf:
cmk:
NUM_SHARED_CORES: 2
EOF
#. Apply the override.
.. code-block:: none
$ helm upgrade cpu-manager cpu-manager-k8s --reuse-values -f /home/sysadmin/worker-0-cmk-overrides.yml
#. After CMK has been installed, run the following command to patch the
webhook to pull the image, if required for future use:
.. code-block:: none
~(keystone)admin)$ kubectl -n kube-system patch deploy cmk-webhook-deployment \
-p '{"spec":{"template":{"spec":{"containers":[{"name":"cmk-webhook",\
"imagePullPolicy":"IfNotPresent"}]}}}}'
.. rubric:: |postreq|
Once CMK is set up, you can run workloads as described at `https://github.com/intel/CPU-Manager-for-Kubernetes <https://github.com/intel/CPU-Manager-for-Kubernetes>`__,
with the following caveats:
- When using CMK, the application pods should not specify requests or limits
for the **cpu** resource.
When running a container with :command:`cmk isolate --pool=exclusive`, the
**cpu** resource should be superseded by the
:command:`cmk.intel.com/exclusive-cores` resource.
When running a container with :command:`cmk isolate --pool=shared` or
:command:`cmk isolate --pool=infra`, the **cpu** resource has no meaning as
Kubelet assumes it has access to all the CPUs rather than just the
**infra** or **shared** ones and this confuses the resource tracking.
- There is a known issue with resource tracking if a node with running
CMK-isolated applications suffers an uncontrolled reboot. The suggested
workaround is to wait for it to come back up, then lock/unlock the node.
- When using the :command:`cmk isolate --socket-id` command to run an
application on a particular socket, there can be complications with
scheduling because the Kubernetes scheduler isn't NUMA-aware. A pod can be
scheduled to a kubernetes node that has enough resources across all NUMA
nodes, but then a container trying to run :command:`cmk isolate --socket-id=<X>`
can lead to a run-time error if there are not enough resources on that
particular NUMA node:
.. code-block:: none
~(keystone)admin)$ kubectl logs cmk-isolate-pod
[6] Failed to execute script cmk
Traceback (most recent call last):
File "cmk.py", line 162, in <module> main()
File "cmk.py", line 127, in main args["--socket-id"])
File "intel/isolate.py", line 57, in isolate.format(pool_name))
SystemError: Not enough free cpu lists in pool
.. From step 1
.. xbooklink For more information on node labeling, see |node-doc|: :ref:`Configure Node Labels from the CLI <assigning-node-labels-from-the-cli>`.
.. From step 2
.. xreflink For more information, see |inst-doc|: :ref:`Bootstrap and Deploy Cloud Platform <bootstrapping-and-deploying-starlingx>`.

View File

@ -10,8 +10,7 @@ Isolate the CPU Cores to Enhance Application Performance
which are completely isolated from the host process scheduler.
This allows you to customize Kubernetes CPU management when policy is set to
static, or when using CMK with policy set to none so that high-performance,
low-latency applications run with optimal efficiency.
static so that low-latency applications run with optimal efficiency.
The following restrictions apply when using application-isolated cores in the
Horizon Web interface and sysinv:
@ -37,10 +36,11 @@ All SMT siblings on a core will have the same assigned function. On host boot,
any CPUs designated as isolated will be specified as part of the isolcpu kernel
boot argument, which will isolate them from the process scheduler.
The use of application-isolated cores is only applicable when using the static
Kubernetes CPU Manager policy, or when using CMK. For more information,
see :ref:`Kubernetes CPU Manager Policies <kubernetes-cpu-manager-policies>`,
or :ref:`Install and Run CPU Manager for Kubernetes <installing-and-running-cpu-manager-for-kubernetes>`.
.. only:: partner
.. include:: /_includes/isolating-cpu-cores-to-enhance-application-performance.rest
:start-after: usage-limitation-begin
:end-before: usage-limitation-end
When using the static CPU manager policy before increasing the number of
platform CPUs or changing isolated CPUs to application CPUs on a host, ensure
@ -53,4 +53,6 @@ functions. On AIO Simplex systems, you must explicitly delete the pods.
.. only:: partner
.. include:: ../_includes/isolating-cpu-cores-to-enhance-application-performance.rest
.. include:: /_includes/isolating-cpu-cores-to-enhance-application-performance.rest
:start-after: changes-relative-to-root-begin
:end-before: changes-relative-to-root-end

View File

@ -1,75 +0,0 @@
.. fuq1561551658529
.. _removing-cpu-manager-for-kubernetes:
=================================
Remove CPU Manager for Kubernetes
=================================
You can uninstall CMK by removing related Helm charts in the reverse order of
their installation.
.. rubric:: |proc|
#. Delete **cmk manager**.
#. Run the :command:`helm delete` command.
.. code-block:: none
~(keystone)admin)$ helm delete --purge
release "cpu-manager" deleted
#. Ensure that any pods in the Terminating state have deleted before
proceeding to the next step. The pods being terminated are in the
**kube-system** namespace.
For example:
.. code-block:: none
~(keystone)admin)$ kubectl get pods -n kube-system | grep cmk
cmk-setup 0/1 Completed 0 71m
cmk-uninstall-2z29p 0/1 ContainerCreating 0 4s
cmk-webhook-deployment-778c787679-7bpw2 1/1 Running 0 71m
cpu-manager-k8s-cmk-compute-0-5621f953-pchjr 3/3 Terminating 0 38
~(keystone)admin)$ kubectl get pods -n kube-system | grep cmk
cmk-setup 0/1 Completed 0 72m
cmk-webhook-deployment-778c787679-7bpw2 1/1 Running 0 72m
#. Delete **cmk-manager-webhook**.
#. Run the :command:`helm delete` command.
.. code-block:: none
~(keystone)admin)$ helm delete cmk-webhook --purge
#. Ensure that any pods in the Terminating state have been deleted before
proceeding to the next step.
.. code-block:: none
~(keystone)admin)$ kubectl get pods -n kube-system | grep cmk
cmk-uninstall-webhook 0/1 Completed 0 11s
cmk-webhook-deployment-778c787679-7bpw2 1/1 Terminating 0 73m
~(keystone)admin)$ kubectl get pods -n kube-system | grep cmk
cmk-uninstall-webhook 0/1 Completed 0 49s
#. Delete **cmk-manager-init**. Run the :command:`helm delete` command.
.. code-block:: none
~(keystone)admin)$ helm delete cmk-manager-init --purge
release "cpu-manager-init" deleted
.. rubric:: |result|
The CPU Manager for Kubernetes is now deleted.
.. seealso::
:ref:`Uninstall CPU Manager for Kubernetes on IPv6 <uninstalling-cpu-manager-for-kubernetes-on-ipv6>`

View File

@ -1,20 +0,0 @@
.. mbd1576786954045
.. _uninstalling-cpu-manager-for-kubernetes-on-ipv6:
===============================================
Uninstalling CPU Manager for Kubernetes on IPv6
===============================================
You will have to run some additional uninstall steps for IPv6 configurations.
When uninstalling CMK on an IPv6 system, first follow the steps at
:ref:`Removing CPU Manager for Kubernetes <removing-cpu-manager-for-kubernetes>`,
then run the following commands:
.. code-block:: none
~(keystone_admin)]$ kubectl delete pod/cmk-uninstall-webhook -n kube-system
~(keystone_admin)]$ kubectl delete ds cmk-uninstall -n kube-system
~(keystone_admin)]$ kubectl delete pod delete-uninstall -n kube-system

View File

@ -1,65 +0,0 @@
=====================================
|prod-long| Kubernetes User Tutorials
=====================================
- :ref:`About the User Tutorials <about-the-user-tutorials>`
- Accessing the System
- :ref:`Overview <kubernetes-user-tutorials-overview>`
- :ref:`Remote CLI Access <remote-cli-access>`
- :ref:`Configuring Container-backed Remote CLIs and Clients <kubernetes-user-tutorials-configuring-container-backed-remote-clis-and-clients>`
- :ref:`Using Container-backed Remote CLIs and Clients <using-container-based-remote-clis-and-clients>`
- :ref:`Installing Kubectl and Helm Clients Directly on a Host <kubernetes-user-tutorials-installing-kubectl-and-helm-clients-directly-on-a-host>`
- :ref:`Configuring Remote Helm Client <configuring-remote-helm-client>`
- :ref:`Accessing the GUI <kubernetes-user-tutorials-accessing-the-gui>`
- :ref:`Accessing the Kubernetes Dashboard <accessing-the-kubernetes-dashboard>`
- :ref:`REST API Access <kubernetes-user-tutorials-rest-api-access>`
- Application Management
- :ref:`Helm Package Manager <kubernetes-user-tutorials-helm-package-manager>`
- Local Docker Registry
- :ref:`Authentication and Authorization <kubernetes-user-tutorials-authentication-and-authorization>`
- :ref:`Using an Image from the Local Docker Registry in a Container Spec <using-an-image-from-the-local-docker-registry-in-a-container-spec>`
- :ref:`NodePort Usage Restrictions <nodeport-usage-restrictions>`
- :ref:`Cert Manager <kubernetes-user-tutorials-cert-manager>`
- :ref:`LetsEncrypt Example <letsencrypt-example>`
- Vault Secret and Data Management
- :ref:`Vault Overview <kubernetes-user-tutorials-vault-overview>`
- :ref:`Vault Aware <vault-aware>`
- :ref:`Vault Unaware <vault-unaware>`
- Using Kata Container Runtime
- Usage
- :ref:`Overview <cloud-platform-kubernetes-user-tutorials-overview>`
- :ref:`Specifying Kata Container Runtime in Pod Spec <specifying-kata-container-runtime-in-pod-spec>`
- :ref:`Known Limitations <known-limitations>`
- Adding Persistent Volume Claims
- :ref:`Creating Persistent Volume Claims <kubernetes-user-tutorials-creating-persistent-volume-claims>`
- :ref:`Mounting Persistent Volumes in Containers <kubernetes-user-tutorials-mounting-persistent-volumes-in-containers>`
- Adding an SRIOV Interface to a Container
- :ref:`Creating Network Attachment Definitions <creating-network-attachment-definitions>`
- :ref:`Using Network Attachment Definitions in a Container <using-network-attachment-definitions-in-a-container>`
- Managing CPU Resource Usage of Containers
- :ref:`Using Kubernetes CPU Manager Static Policy <using-kubernetes-cpu-manager-static-policy>`
- :ref:`Using Intel's CPU Manager for Kubernetes (CMK) <using-intels-cpu-manager-for-kubernetes-cmk>`
- :ref:`Uninstalling CMK <uninstalling-cmk>`

View File

@ -132,6 +132,15 @@ Ceph File System Provisioner
kubernetes-user-tutorials-create-readwritemany-persistent-volume-claims
kubernetes-user-tutorials-mount-readwritemany-persistent-volumes-in-containers
--------------------------------
Optimize application performance
--------------------------------
.. toctree::
:maxdepth: 1
using-kubernetes-cpu-manager-static-policy
----------------------------------------
Adding an SRIOV interface to a container
----------------------------------------
@ -142,16 +151,6 @@ Adding an SRIOV interface to a container
creating-network-attachment-definitions
using-network-attachment-definitions-in-a-container
--------------------------
CPU Manager for Kubernetes
--------------------------
.. toctree::
:maxdepth: 1
using-kubernetes-cpu-manager-static-policy
using-intels-cpu-manager-for-kubernetes-cmk
uninstalling-cmk
**************
Metrics Server
@ -160,4 +159,4 @@ Metrics Server
.. toctree::
:maxdepth: 1
kubernetes-user-tutorials-metrics-server
kubernetes-user-tutorials-metrics-server

View File

@ -1,33 +0,0 @@
.. usq1569263366388
.. _uninstalling-cmk:
=============
Uninstall CMK
=============
You can uninstall the CPU Manager for Kubernetes from the command line.
.. rubric:: |proc|
#. Delete **cmk**.
.. code-block:: none
% helm delete --purge cmk
Wait for all pods in the terminating state to be deleted before proceeding.
#. Delete **cmk-webhook**.
.. code-block:: none
% helm delete --purge cmk-webhook
Wait for all pods in the terminating state to be deleted before proceeding.
#. Delete **cmk-init**.
.. code-block:: none
% helm delete --purge cmk-init

View File

@ -1,32 +0,0 @@
.. nnj1569261145380
.. _using-intels-cpu-manager-for-kubernetes-cmk:
==============================================
Use Intel's CPU Manager for Kubernetes \(CMK\)
==============================================
Use the CMK user manual to run a workload via CMK.
See `https://github.com/intel/CPU-Manager-for-Kubernetes/blob/master/docs/user.md#pod-configuration-on-the-clusters-with-cmk-mutating-webhook-kubernetes-v190
<https://github.com/intel/CPU-Manager-for-Kubernetes/blob/master/docs/user.md#pod-configuration-on-the-clusters-with-cmk-mutating-webhook-kubernetes-v190>`__ for detailed instructions.
.. xreflink See Kubernetes Admin Tasks: :ref:`Kubernetes CPU Manager Static Policy
<isolating-cpu-cores-to-enhance-application-performance>` for details on how
to enable this CPU management mechanism.
The basic workflow is to:
.. _using-intels-cpu-manager-for-kubernetes-cmk-ul-xcq-cwb-2jb:
#. Request the number of exclusive cores you want as:
.. code-block:: none
cmk.intel.com/exclusive-cores
#. Run your workload as:
.. code-block:: none
/opt/bin/cmk isolate --pool=exclusive <workload>