859e8eb730
In order to minimize latency as much as possible, we want to allow kubernetes containers to make use of CPUs which have been specified as "isolated" via the kernel boot args. This commit creates an isolcpus device plugin, which detects the isolated CPUs and exports them to kubelet via the device plugin API. See kubernetes/plugins/isolcpus-device-plugin/files/README.md for more information on the behaviour and design choices for this commit. When we move to a newer version of the Intel device plugin manager we may be able to simplify some of this. See the above README.md file for details. Change-Id: I3bfe04ab6e7fbafefa63f6dc43cb2ed79a52579f Story: 2008760 Task: 42165 Signed-off-by: Chris Friesen <chris.friesen@windriver.com> |
||
|---|---|---|
| .. | ||
| intel/intel-device-plugins-for-kubernetes/pkg | ||
| kubernetes/pkg/kubelet | ||
| vendor | ||
| go.mod | ||
| go.sum | ||
| isolcpu_plugin.conf | ||
| isolcpu_plugin.service | ||
| isolcpu.go | ||
| LICENSE | ||
| README.md | ||
Isolated CPUs Device Plugin for Kubernetes
About
This code implements a Kubernetes device plugin. The plugin detects all CPUs specified via "isolcpus=X" in the kernel boot args, and exports them to Kubernetes as custom devices using the deviceplugin API.
It makes heavy use of the Intel device plugin manager from github.com/intel/intel-device-plugins-for-kubernetes and credit is due to them for making a useful helper. A good example of how to use that framework can be found at https://github.com/intel/intel-device-plugins-for-kubernetes/blob/master/cmd/gpu_plugin/gpu_plugin.go
Implementation Notes
There are currently problems with using go modules for the deviceplugin API...it leads to an "go: error loading module requirements" error when running "go build". Accordingly, it was necessary to copy a number of files from external packages. As part of this work I also updated the deviceplugin API files to the latest versions to pick up in-development upstream changes.
The "intel/intel-device-plugins-for-kubernetes" subdirectory corresponds to "github.com/intel/intel-device-plugins-for-kubernetes".
The "kubernetes" subdirectory corresponds to "k8s.io/kubernetes"
In an ideal world, these two subdirectories would not be needed, and instead we would simply include the following imports in isolcpu.go:
"github.com/intel/intel-device-plugins-for-kubernetes/pkg/debug"
dpapi "github.com/intel/intel-device-plugins-for-kubernetes/pkg/deviceplugin"
pluginapi "k8s.io/kubernetes/pkg/kubelet/apis/deviceplugin/v1beta1"
"k8s.io/kubernetes/pkg/kubelet/cm/cpuset"
This would also require updating the Intel package to pick up the latest deviceplugin API so that the topology field is properly represented.
Build Notes
In order to avoid the need for a network connection to download dependencies at build time, I've chosen to include all the dependencies in the "vendor" directory. This is auto-generated by running "go mod vendor". The binary is then built with "go build -mod=vendor".