This story will pin the infrastructure and openstack pods to the
platform cores for worker nodes and All-in-one servers.
This configures systemd system.conf parameter
CPUAffinity=<platform_cpus> by generating
/etc/systemd/system.conf.d/platform-cpuaffinity.conf .
All services launch tasks with the appropriate cpu affinity.
This creates the cgroup called 'k8s-infra' for the following subset
of controllers ('cpuacct', 'cpuset', 'cpu', 'memory', 'systemd').
This configures custom cpuset.cpus (i.e., cpuset) and cpuset.mems
(i.e., nodeset) based on sysinv platform configurable cores. This is
generated by puppet using sysinv host cpu information and is stored
to the hieradata variables:
- platform::kubernetes::params::k8s_cpuset
- platform::kubernetes::params::k8s_nodeset
This creates the cgroup called 'machine.slice' for the controller
'cpuset' and sets cpuset.cpus and cpuset.mems to the parent values.
This prevents VMs from inheriting those settings from libvirt.
Note: systemd automatically mounts cgroups and all available
resource controllers, so the new puppet code does not need to do
that.
Kubelet is now launched with --cgroup-root /k8s-infra by configuring
kubeadm.yaml with the option: cgroupRoot: "/k8s-infra" .
For openstack based worker nodes including AIO
(i.e., host-label openstack-compute-node=enabled):
- the k8s cpuset and nodeset include the assigned platform cores
For non-openstack based worker nodes including AIO:
- the k8s cpuset and nodeset include all cpus except the assigned
platform cores. This will be refined in a later update since
we need isolate cpusets of k8s infrastructure from other pods.
The cpuset topology can be viewed with the following:
sudo systemd-cgls cpuset
The task cpu affinity can be verified with the following:
ps-sched.sh
The dynamic affining of platform tasks during start-up is disabled,
that code requires cleanup, and likely no longer required
since we are using systemd CPUAffinity and cgroups.
This includes a few small fixes to enable testing of this feature:
- facter platform_res_mem was updated to not require 'memtop', since
that depends on existance of numa nodes. This was failing on QEMU
environment when the host does not have Numa nodes. This occurs
when there is no CPU topology specified.
- cpumap_functions.sh updated parameter defaults so that calling
bash scripts may enable 'set -u' undefined variable checking.
- the generation of platform_cpu_list did not have all threads.
- the cpulist-to-ranges inline code was incorrect; in certain
senarios the rstrip(',') would take out the wrong commas.
Story: 2004762
Task: 28879
Change-Id: I6fd21bac59fc2d408132905b88710da48aa8d928
Signed-off-by: Jim Gauld <james.gauld@windriver.com>
209e346ab4