The current starlingx platform containers implementation by definition
does not have any cpu request and are allocated on reserved cpus, that
are not under kubernetes management, however they are still allocated
on the same cgroup of application containers (k8s-infra).
This causes an imbalance on cpushares calculations performed by
kubernetes. As a result, platform containers have a much lower priority
on kernel cpu scheduling, and in case of resource contention
(AIO-SX with 1 platform core), they suffer a harmful degradation,
causing probe failures and latency spikes on kube-apiserver.
This change allocates platform pods on a dedicated cgroup
(called k8s-infra-stx), so a balanced cpu shares distribution between
platform and application pods (running on k8s-infra cgroup) can be
achieved.
Test Plan (AIO-SX 1p core):
PASS: Build ISO and fresh install
PASS: The k8s-infra-stx cgroups directories are created:
ls -la /sys/fs/cgroup/*/k8s-infra-stx/kubepods
PASS: Launch some starlingx apps, confirm that their pods are allocated
on the k8s-infra-stx cgroup
PASS: Launch VDU simulation app (on application cores) with the
stx apps still running, verify that no kube-apiserver latency spikes nor
probe failures occurs in the next hours.
PASS: Kubernetes conformance test.
PASS: <sudo kube-cpusets> output includes platform pods
PASS: <sudo kube-memory> output includes platform pods
PASS: Run following make tests successfully:
make test WHAT=./pkg/kubelet/cm GOFLAGS="-v"
make test WHAT=./pkg/kubelet/cm/cpumanager GOFLAGS="-v"
make test WHAT=./pkg/kubelet/cm/cpumanager/state GOFLAGS="-v"
make test WHAT=./pkg/kubelet/cm/cpumanager/topology GOFLAGS="-v"
make test WHAT=./pkg/kubelet/cm/topologymanager GOFLAGS="-v"
make test WHAT=./pkg/kubelet/cm/devicemanager GOFLAGS="-v"
make test WHAT=./pkg/kubelet/cm/memorymanager GOFLAGS="-v"
make test WHAT=./pkg/kubelet/kuberuntime GOFLAGS="-v"
make test WHAT=./cmd/kubeadm/app/constants GOFLAGS="-v"
make test WHAT=./cmd/kubeadm/app/phases/controlplane GOFLAGS="-v"
make test WHAT=./pkg/kubelet/cm GOFLAGS="-v"
make test WHAT=./cmd/kubeadm/app/phases/addons/dns/ GOFLAGS="-v"
Story: 2011377
Task: 52738
Change-Id: I0153daf8b676af45d5a26283eef4732c42042479
Signed-off-by: Alyson Deives Pereira <alyson.deivespereira@windriver.com>
d567446047