Kata Container Support with Debian OS
Update Kata containers docs Fix conflict Story: 2010765 Task: 49554 Change-Id: I07e31de31baca504e073efc5358ab664d9355909 Signed-off-by: Elisamara Aoki Goncalves <elisamaraaoki.goncalves@windriver.com>
This commit is contained in:
Elisamara Aoki Goncalves
parent
ecef035a3a
commit
1eae56ba7f
@ -35,7 +35,6 @@ Kubernetes Operation
|
||||
k8s_persistent_vol_claims
|
||||
k8s_sriov_config
|
||||
k8s_gpu_device_plugin
|
||||
kata_container
|
||||
|
||||
-------------------
|
||||
OpenStack Operation
|
||||
|
||||
@ -1,156 +0,0 @@
|
||||
===============
|
||||
Kata Containers
|
||||
===============
|
||||
|
||||
This guide describes how to run Kata Containers with Kubernetes on StarlingX.
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
--------
|
||||
Overview
|
||||
--------
|
||||
|
||||
StarlingX has supported Kata Containers in master since January 2020, and coming
|
||||
Release 4.0 will include it. Release 3.0 and before will not support it.
|
||||
|
||||
To support Kata Containers, pods are created by containerd instead of Docker.
|
||||
Also containerd is configured to support both runc and Kata Containers, while the
|
||||
default runtime is still runc. If you want to launch a pod with Kata Containers,
|
||||
you must declare it explicitly.
|
||||
|
||||
---------------------------------
|
||||
Run Kata Containers in Kubernetes
|
||||
---------------------------------
|
||||
|
||||
There are two methods to run Kata Containers in Kubernetes: by runtime class or
|
||||
by annotation. Runtime class is supported in Kubernetes since v1.12.0, and it is
|
||||
the recommended method for running Kata Containers.
|
||||
|
||||
To run by runtime class, create a RuntimeClass with ``handler`` set to ``kata``.
|
||||
Then reference this class in the pod spec, as shown in the following example:
|
||||
|
||||
::
|
||||
|
||||
kind: RuntimeClass
|
||||
apiVersion: node.k8s.io/v1
|
||||
metadata:
|
||||
name: kata-containers
|
||||
handler: kata
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: busybox-runtime
|
||||
spec:
|
||||
runtimeClassName: kata-containers
|
||||
containers:
|
||||
- name: busybox
|
||||
command:
|
||||
- sleep
|
||||
- "3600"
|
||||
image: busybox
|
||||
|
||||
To run a pod with Kata Containers by annotation, set ``io.kubernetes.cri.untrusted-workload``
|
||||
to ``true`` in the annotations section of a pod spec.
|
||||
|
||||
.. note::
|
||||
|
||||
This method is deprecated and may not be supported in future
|
||||
Kubernetes releases. We recommend using the RuntimeClass method.
|
||||
|
||||
Example of using annotation:
|
||||
|
||||
::
|
||||
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: busybox-untrusted
|
||||
annotations:
|
||||
io.kubernetes.cri.untrusted-workload: "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: busybox
|
||||
command:
|
||||
- sleep
|
||||
- "3600"
|
||||
image: busybox
|
||||
|
||||
--------------------------------
|
||||
Kata configuration in containerd
|
||||
--------------------------------
|
||||
.. note::
|
||||
|
||||
No action is required for end user. This section just shows the
|
||||
configuration in containerd to support Kata Containers.
|
||||
|
||||
Containerd's configuration file ``/etc/containerd/config.toml`` is customized
|
||||
to support Kata Containers.
|
||||
|
||||
For RuntimeClass, here is the configuration in config.toml:
|
||||
|
||||
::
|
||||
|
||||
[plugins.cri.containerd.runtimes.kata]
|
||||
runtime_type = "io.containerd.kata.v2"
|
||||
|
||||
For annotation, here is the configuration in config.toml:
|
||||
|
||||
::
|
||||
|
||||
[plugins.cri.containerd.runtimes.untrusted]
|
||||
runtime_type = "io.containerd.kata.v2"
|
||||
|
||||
In this example, ``kata.v2`` means ``shimv2(containerd-shim-kata-v2)``, which
|
||||
helps Kubernetes to launch pods and OCI-compatible containers with one shim per
|
||||
pod.
|
||||
|
||||
-------------------------
|
||||
Check Kata Containers use
|
||||
-------------------------
|
||||
|
||||
Here are two methods to check whether the pod is running with Kata Containers
|
||||
or not:
|
||||
|
||||
#. Run ``uname -a`` in both container and host. The host kernel version should
|
||||
be 4.18.0, while the container kernel version should be 4.19 or higher.
|
||||
For normal container, the host kernel version is the same as the container.
|
||||
|
||||
#. Run ``ps aux`` in the host. A normal container is triggered by
|
||||
containerd-shim-runc-v1, while Kata Containers is triggered by
|
||||
containerd-shim-kata-v2.
|
||||
|
||||
----------
|
||||
References
|
||||
----------
|
||||
|
||||
* For technical details about how Kata Containers is implemented on StarlingX,
|
||||
refer to:
|
||||
|
||||
* Spec file: https://opendev.org/starlingx/specs/src/branch/master/doc/source/specs/stx-3.0/approved/containerization-2006145-kata-containers-integration.rst
|
||||
* Story: https://storyboard.openstack.org/#!/story/2006145
|
||||
* Patches: https://review.opendev.org/#/q/topic:kata+projects:starlingx
|
||||
|
||||
* Kata Containers is supported for Kubernetes only, since Kubernetes is the
|
||||
only supported container orchestration tool in StarlingX. Kata Container
|
||||
support for Docker is not implemented in StarlingX. The Docker runtime also
|
||||
may be removed in future releases of StarlingX, since all containers in
|
||||
StarlingX are run by Kubernetes at this time.
|
||||
|
||||
To try Kata Containers with Docker in StarlingX, refer to this link:
|
||||
https://github.com/kata-containers/documentation/blob/master/install/docker/centos-docker-install.md
|
||||
|
||||
* To support Kata Containers, the CRI runtime in Kubernetes was switched from
|
||||
``dockershim`` to ``containerd``. This means you cannot view/operate a
|
||||
Kubernetes container with the Docker client. You must use ``crictl``
|
||||
instead, which supports commands that are similar to Docker commands. There
|
||||
is no difference in kubectl commands before and after the switch to
|
||||
containerd.
|
||||
|
||||
* More information is available at:
|
||||
|
||||
* Kata Containers: https://katacontainers.io/
|
||||
* containerd: https://containerd.io/
|
||||
* Kubernetes RuntimeClass: https://kubernetes.io/docs/concepts/containers/runtime-class/
|
||||
@ -63,17 +63,13 @@ and volumes through pod security policies, for example:
|
||||
Kata Containers
|
||||
---------------
|
||||
|
||||
.. note::
|
||||
|
||||
Kata Containers will not be supported in |prod-long| |prod-ver|.
|
||||
|
||||
Kata containers are an optional capability on |prod| that provide a secure
|
||||
container runtime with lightweight virtual machines that feel and perform like
|
||||
containers, but provide stronger workload isolation. For improved performance
|
||||
with relation to isolation, Kata containers leverages hardware-enforced isolation
|
||||
with virtualization VT extensions.
|
||||
with relation to isolation, Kata containers leverages hardware-enforced
|
||||
isolation with virtualization VT extensions.
|
||||
|
||||
For more information, see :ref:`starlingx-kubernetes-user-tutorials-overview`.
|
||||
For more information, see :ref:`kata_container`.
|
||||
|
||||
---------------------
|
||||
Pod Security Policies
|
||||
|
||||
@ -6,14 +6,84 @@
|
||||
Kata Containers Overview
|
||||
========================
|
||||
|
||||
.. note::
|
||||
|
||||
Kata Containers will not be supported in |prod-long| |prod-ver|.
|
||||
|
||||
|prod| uses a **containerd** :abbr:`CRI (Container Runtime Interface)` that supports
|
||||
both runc and Kata Container runtimes. The default runtime is runc. If you want
|
||||
to launch a pod that uses the Kata Container runtime, you must declare it
|
||||
explicitly.
|
||||
|prod| uses a **containerd** :abbr:`CRI (Container Runtime Interface)` that
|
||||
supports both runc and Kata Container runtimes. The default runtime is runc. If
|
||||
you want to launch a pod that uses the Kata Container runtime, you must declare
|
||||
it explicitly. You can see more details on how to specify the use of Kata
|
||||
Container in :ref:`specifying-kata-container-runtime-in-pod-spec`.
|
||||
|
||||
For more information about Kata containers, see `https://katacontainers.io/
|
||||
<https://katacontainers.io/>`__.
|
||||
|
||||
|
||||
This guide describes how to run Kata Containers with Kubernetes on StarlingX.
|
||||
|
||||
.. _kata_container:
|
||||
|
||||
---------------------------------
|
||||
Run Kata Containers in Kubernetes
|
||||
---------------------------------
|
||||
|
||||
There are two methods to run Kata Containers in Kubernetes: by runtime class or
|
||||
by annotation. Runtime class is supported in Kubernetes since v1.12.0 or
|
||||
higher, and it is the recommended method for running Kata Containers.
|
||||
|
||||
To run by runtime class, create a RuntimeClass with ``handler`` set to ``kata``.
|
||||
Then reference this class in the pod spec, as shown in the following example:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
kind: RuntimeClass
|
||||
apiVersion: node.k8s.io/v1
|
||||
metadata:
|
||||
name: kata-containers
|
||||
handler: kata
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: busybox-runtime
|
||||
spec:
|
||||
runtimeClassName: kata-containers
|
||||
containers:
|
||||
- name: busybox
|
||||
command:
|
||||
- sleep
|
||||
- "3600"
|
||||
image: busybox
|
||||
|
||||
Alternatively, set the ``io.kubernetes.cri.untrusted-workload`` to ``true``, to
|
||||
run a pod with Kata Containers.
|
||||
|
||||
.. note::
|
||||
|
||||
This method is deprecated and may not be supported in future Kubernetes
|
||||
releases. We recommend using the RuntimeClass method.
|
||||
|
||||
Example of using annotation:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: busybox-untrusted
|
||||
annotations:
|
||||
io.kubernetes.cri.untrusted-workload: "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: busybox
|
||||
command:
|
||||
- sleep
|
||||
- "3600"
|
||||
image: busybox
|
||||
|
||||
|
||||
-------------------------
|
||||
Check Kata Containers use
|
||||
-------------------------
|
||||
|
||||
You can check whether the pod is running with Kata Containers or not by running
|
||||
``ps aux`` in the host. A normal container is triggered by
|
||||
``containerd-shim-runc-v1``, while Kata Containers is triggered by
|
||||
``containerd-shim-kata-v2``.
|
||||
|
||||
@ -6,10 +6,6 @@
|
||||
Known Kata Container Limitations
|
||||
================================
|
||||
|
||||
.. note::
|
||||
|
||||
Kata Containers will not be supported in |prod-long| |prod-ver|.
|
||||
|
||||
This section describes the known limitations when using Kata containers.
|
||||
|
||||
.. _known-limitations-section-tsh-tl1-zlb:
|
||||
@ -19,11 +15,12 @@ SR-IOV Support
|
||||
--------------
|
||||
|
||||
A minimal **kernel** and **rootfs** for Kata containers are shipped with
|
||||
|prod-long|, and are present at /usr/share/kata-containers/. To enable certain
|
||||
kernel features such as :abbr:`IOMMU (I/O memory management unit)`, and desired
|
||||
network kernel modules, a custom kernel image, and rootfs has to be built. For
|
||||
more information, see `https://katacontainers.io/
|
||||
<https://katacontainers.io/>`__.
|
||||
|prod-long|, and can be found at ``/usr/share/kata-containers/``. To enable
|
||||
certain kernel features such as :abbr:`IOMMU (I/O memory management unit)`, and
|
||||
desired network kernel modules, a custom kernel image, and rootfs has to be
|
||||
built. However, many tests carried out using this solution were unsuccessful.
|
||||
More details of this solution can be seen in:
|
||||
https://bugs.launchpad.net/starlingx/+bug/1867927.
|
||||
|
||||
.. _known-limitations-section-ngp-ty3-bmb:
|
||||
|
||||
@ -42,11 +39,17 @@ Hugepages
|
||||
|
||||
.. _known-limitations-ul-uhz-xy3-bmb:
|
||||
|
||||
- Similar to the SR-IOV limitation, hugepage support must be configured in a
|
||||
- Similar to the |SRIOV| limitation, hugepage support must be configured in a
|
||||
custom Kata kernel.
|
||||
|
||||
- The size and number of hugepages must be written using the
|
||||
:command:`io.katacontainers.config.hypervisor.kernel_params` annotation.
|
||||
|
||||
- Creating a **hugetlbfs** mount for hugepages in the Kata container is
|
||||
- Creating a ``hugetlbfs`` mount for hugepages in the Kata container is
|
||||
specific to the end user application.
|
||||
|
||||
.. note::
|
||||
|
||||
This solution does not work in version v3.1.3 of the Kata Containers. A
|
||||
pod can be started with hugepages specified, but none are actually
|
||||
allocated to the Kata |VM|.
|
||||
@ -6,66 +6,62 @@
|
||||
Specify Kata Container Runtime in Pod Spec
|
||||
==========================================
|
||||
|
||||
.. note::
|
||||
|
||||
Kata Containers will not be supported in |prod-long| |prod-ver|.
|
||||
|
||||
You can specify the use of Kata Container runtime in your pod specification by
|
||||
runtime class or by annotation.
|
||||
|
||||
.. rubric:: |proc|
|
||||
|
||||
* Do one of the following:
|
||||
Choose one of the methods below:
|
||||
|
||||
.. table::
|
||||
:widths: auto
|
||||
.. table::
|
||||
:widths: auto
|
||||
|
||||
+--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+
|
||||
| **To use the runtime class method:** | #. Create a RuntimeClass with handler set to kata. |
|
||||
| | |
|
||||
| | #. Reference this class in the pod spec, as shown in the following example: |
|
||||
| | |
|
||||
| | .. code-block:: none |
|
||||
| | |
|
||||
| | kind: RuntimeClass |
|
||||
| | apiVersion: node.k8s.io/v1 |
|
||||
| | metadata: |
|
||||
| | name: kata-containers |
|
||||
| | handler: kata |
|
||||
| | --- |
|
||||
| | apiVersion: v1 |
|
||||
| | kind: Pod |
|
||||
| | metadata: |
|
||||
| | name: busybox-runtime |
|
||||
| | spec: |
|
||||
| | runtimeClassName: kata-containers |
|
||||
| | containers: |
|
||||
| | - name: busybox |
|
||||
| | command: |
|
||||
| | - sleep |
|
||||
| | - "3600" |
|
||||
| | image: busybox |
|
||||
+--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+
|
||||
| **To use the annotation method:** | Set io.kubernetes.cri.untrusted-workload to true in the annotations section of a pod spec. |
|
||||
| | |
|
||||
| | For example: |
|
||||
| | |
|
||||
| | .. code-block:: none |
|
||||
| | |
|
||||
| | apiVersion: v1 |
|
||||
| | kind: Pod |
|
||||
| | metadata: |
|
||||
| | name: busybox-untrusted |
|
||||
| | annotations: |
|
||||
| | io.kubernetes.cri.untrusted-workload: "true" |
|
||||
| | spec: |
|
||||
| | containers: |
|
||||
| | - name: busybox |
|
||||
| | command: |
|
||||
| | - sleep |
|
||||
| | - "3600" |
|
||||
| | image: busybox |
|
||||
| | |
|
||||
| | .. note:: |
|
||||
| | This method is deprecated and may not be supported in future releases. |
|
||||
+--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+
|
||||
+--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+
|
||||
| **To use the runtime class method:** | #. Create a RuntimeClass with handler set to kata. |
|
||||
| | |
|
||||
| | #. Reference this class in the pod spec, as shown in the following example: |
|
||||
| | |
|
||||
| | .. code-block:: none |
|
||||
| | |
|
||||
| | kind: RuntimeClass |
|
||||
| | apiVersion: node.k8s.io/v1 |
|
||||
| | metadata: |
|
||||
| | name: kata-containers |
|
||||
| | handler: kata |
|
||||
| | --- |
|
||||
| | apiVersion: v1 |
|
||||
| | kind: Pod |
|
||||
| | metadata: |
|
||||
| | name: busybox-runtime |
|
||||
| | spec: |
|
||||
| | runtimeClassName: kata-containers |
|
||||
| | containers: |
|
||||
| | - name: busybox |
|
||||
| | command: |
|
||||
| | - sleep |
|
||||
| | - "3600" |
|
||||
| | image: busybox |
|
||||
+--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+
|
||||
| **To use the annotation method:** | Set io.kubernetes.cri.untrusted-workload to true in the annotations section of a pod spec. |
|
||||
| | |
|
||||
| | For example: |
|
||||
| | |
|
||||
| | .. code-block:: none |
|
||||
| | |
|
||||
| | apiVersion: v1 |
|
||||
| | kind: Pod |
|
||||
| | metadata: |
|
||||
| | name: busybox-untrusted |
|
||||
| | annotations: |
|
||||
| | io.kubernetes.cri.untrusted-workload: "true" |
|
||||
| | spec: |
|
||||
| | containers: |
|
||||
| | - name: busybox |
|
||||
| | command: |
|
||||
| | - sleep |
|
||||
| | - "3600" |
|
||||
| | image: busybox |
|
||||
| | |
|
||||
| | .. note:: |
|
||||
| | This method is deprecated and may not be supported in future releases. |
|
||||
+--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+
|
||||
Loading…
x
Reference in New Issue
Block a user