Add DSA plugin to app-intel-device-plugins

This commits adds helm chart of Intel DSA (Data Streaming Accelerator)
plugin to Intel Device Plugins app. Intel DSA is a high-performance data
copy and transformation accelerator integrated into processors starting
with 4th Generation Intel Xeon processors. It is targeted for
optimizing streaming data movement and transformation operations common
with applications for high-performance storage, networking, persistent
memory, and various data processing applications. The Intel DSA device
plugin discovers DSA work queues and presents them as node resources.

This commit also overrides default dsa helm chart value for
nodeFeatureRule to true, which labels the node with the help of NFD
(Node Feature Discovery) app. Node Feature Rule is defined in dsa.yaml
file which recognizes the device id of DSA hardware and label the node
with intel.feature.node.kubernetes.io/dsa: "true".

nodeSelector is used to match the label and create the pod of DSA plugin
in the node where the DSA devices (hardware) exists. So NodeAffinity and
Tolerations are not required to use.

Moreover, this commit patches the upstream helm chart to allow a custom
configuration for DSA device by setting the overrideConfig value via
system helm-chart-overrides. The default config is based on upstream
intel-device-plugins repository:

https://github.com/intel/intel-device-plugins-for-kubernetes/blob/release-0.29/deployments/dsa_plugin/overlays/dsa_initcontainer/dsa-config.yaml

Test Plan:
PASS: Build process is successful with creation of debian package.
PASS: Extract the content of stx-intel-device-plugins-helm_*.deb and
      check if the dsa helm tar file is present.
PASS: Upload using 'system application-upload' command
PASS: Check the status of charts using below command, it should
      be disabled.
      "system helm-override-list app_name --long"
PASS: Enable the charts using below command:
      "system helm-chart-attribute-modify --enabled true app_name
      chart_name namespace"
PASS: Apply the app using "system application-apply" command
PASS: Testing on DSA supported hardware (AIO-SX):
      After apply, DSA sample pod should be running.
      Verify using below command:
      "kubectl get pods -A | grep dsa"
PASS: Testing on DSA not supported hardware (AIO-SX):
      After apply, Daemonset should be running, but dsa plugin
      pod should not be created. Verify using below command.
      "kubectl get ds -A | grep dsa"
PASS: Tested app upload/apply/remove/delete in AIO-SX.
PASS: Override DSA config and confirm that it is applied via
      idxd-initcontainer after system application-apply

Story: 2011040
Task: 49559

Change-Id: I8bdb615ffd2b0a7d304a47e6056a8b67c1e8fd9a
Signed-off-by: Alyson Deives Pereira <alyson.deivespereira@windriver.com>
This commit is contained in:
Alyson Deives Pereira 2024-02-02 16:48:04 -03:00
parent c3b85be3eb
commit 70c3fb58e1
23 changed files with 593 additions and 3 deletions

View File

@ -1,4 +1,5 @@
intel-device-plugins-operator-helm
intel-device-plugins-dsa-helm
intel-device-plugins-qat-helm
python3-k8sapp-intel-device-plugins-operator
stx-intel-device-plugins-operator-helm

View File

@ -0,0 +1,5 @@
intel-device-plugins-dsa-helm (0.29-0) unstable; urgency=medium
* Initial release.
-- Alyson Deives Pereira <alyson.deivespereira@windriver.com> Wed, 31 Jan 2024 10:45:00 +0000

View File

@ -0,0 +1,15 @@
Source: intel-device-plugins-dsa-helm
Section: libs
Priority: optional
Maintainer: StarlingX Developers <starlingx-discuss@lists.starlingx.io>
Build-Depends: debhelper-compat (= 13),
helm
Standards-Version: 4.5.1
Homepage: https://www.starlingx.io
Package: intel-device-plugins-dsa-helm
Section: libs
Architecture: any
Depends: ${misc:Depends}
Description: StarlingX Intel Device Plugins DSA Helm Charts
This package contains helm charts for the Intel Device Plugins application.

View File

@ -0,0 +1,41 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: intel-device-plugins-operator-helm
Source: https://opendev.org/starlingx/app-intel-device-plugins/
Files: *
Copyright: (c) 2024 Wind River Systems, Inc
License: Apache-2
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
https://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.
On Debian-based systems the full text of the Apache version 2.0 license
can be found in `/usr/share/common-licenses/Apache-2.0'.
# If you want to use GPL v2 or later for the /debian/* files use
# the following clauses, or change it to suit. Delete these two lines
Files: debian/*
Copyright: 2023 Wind River Systems, Inc
License: Apache-2
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
https://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.
On Debian-based systems the full text of the Apache version 2.0 license
can be found in `/usr/share/common-licenses/Apache-2.0'.

View File

@ -0,0 +1,299 @@
From ad7bd1edd81250f6cfd8206c81ed4f26cd23f80c Mon Sep 17 00:00:00 2001
From: Alyson Deives Pereira <alyson.deivespereira@windriver.com>
Date: Fri, 1 Mar 2024 10:27:30 -0300
Subject: [PATCH] Enable custom DSA configuration via helm overrides
Enable custom configuration for DSA devices via helm overrides by
defining the overrideConfig value.
The default DSA configuration provided is based on the default config
from upstream intel-device-plugins repository:
https://github.com/intel/intel-device-plugins-for-kubernetes/blob/release-0.29/deployments/dsa_plugin/overlays/dsa_initcontainer/dsa-config.yaml
---
.../templates/_helpers.tpl | 3 +
.../templates/dsa-config.yaml | 232 ++++++++++++++++++
templates/dsa.yaml | 2 +-
values.yaml | 4 +-
4 files changed, 239 insertions(+), 2 deletions(-)
create mode 100644 templates/_helpers.tpl
create mode 100644 templates/dsa-config.yaml
diff --git a/templates/_helpers.tpl b/templates/_helpers.tpl
new file mode 100644
index 0000000..c5f1fd6
--- /dev/null
+++ b/templates/_helpers.tpl
@@ -0,0 +1,3 @@
+{{- define "dsa.provisioningConfig" -}}
+{{- printf "intel-dsa-config-%s" (.Values.overrideConfig | toJson | sha256sum) -}}
+{{- end -}}
\ No newline at end of file
diff --git a/templates/dsa-config.yaml b/templates/dsa-config.yaml
new file mode 100644
index 0000000..e70bf9d
--- /dev/null
+++ b/templates/dsa-config.yaml
@@ -0,0 +1,232 @@
+{{- /*
+based on
+deployments/dsa_plugin/overlays/dsa_initcontainer/dsa-config.yaml
+*/}}
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: {{ include "dsa.provisioningConfig" . }}
+data:
+{{- if .Values.overrideConfig }}
+ {{- .Values.overrideConfig | toYaml | nindent 2 }}
+{{- else }}
+ dsa.conf: |
+ [
+ {
+ "dev":"dsaX",
+ "read_buffer_limit":0,
+ "groups":[
+ {
+ "dev":"groupX.0",
+ "read_buffers_reserved":0,
+ "use_read_buffer_limit":0,
+ "read_buffers_allowed":8,
+ "grouped_workqueues":[
+ {
+ "dev":"wqX.0",
+ "mode":"dedicated",
+ "size":16,
+ "group_id":0,
+ "priority":10,
+ "block_on_fault":1,
+ "type":"user",
+ "name":"appX0",
+ "threshold":15
+ }
+ ],
+ "grouped_engines":[
+ {
+ "dev":"engineX.0",
+ "group_id":0
+ },
+ ]
+ },
+ {
+ "dev":"groupX.1",
+ "read_buffers_reserved":0,
+ "use_read_buffer_limit":0,
+ "read_buffers_allowed":8,
+ "grouped_workqueues":[
+ {
+ "dev":"wqX.1",
+ "mode":"dedicated",
+ "size":16,
+ "group_id":1,
+ "priority":10,
+ "block_on_fault":1,
+ "type":"user",
+ "name":"appX1",
+ "threshold":15
+ }
+ ],
+ "grouped_engines":[
+ {
+ "dev":"engineX.1",
+ "group_id":1
+ },
+ ]
+ },
+ {
+ "dev":"groupX.2",
+ "read_buffers_reserved":0,
+ "use_read_buffer_limit":0,
+ "read_buffers_allowed":8,
+ "grouped_workqueues":[
+ {
+ "dev":"wqX.2",
+ "mode":"dedicated",
+ "size":16,
+ "group_id":2,
+ "priority":10,
+ "block_on_fault":1,
+ "type":"user",
+ "name":"appX2",
+ "threshold":15
+ }
+ ],
+ "grouped_engines":[
+ {
+ "dev":"engineX.2",
+ "group_id":2
+ },
+ ]
+ },
+ {
+ "dev":"groupX.3",
+ "read_buffers_reserved":0,
+ "use_read_buffer_limit":0,
+ "read_buffers_allowed":8,
+ "grouped_workqueues":[
+ {
+ "dev":"wqX.3",
+ "mode":"dedicated",
+ "size":16,
+ "group_id":3,
+ "priority":10,
+ "block_on_fault":1,
+ "type":"user",
+ "name":"appX3",
+ "threshold":15
+ }
+ ],
+ "grouped_engines":[
+ {
+ "dev":"engineX.3",
+ "group_id":3
+ },
+ ]
+ },
+ ]
+ }
+ ]
+ dsa-controller-0.conf: |
+ [
+ {
+ "dev":"dsaX",
+ "read_buffer_limit":0,
+ "groups":[
+ {
+ "dev":"groupX.0",
+ "read_buffers_reserved":0,
+ "use_read_buffer_limit":0,
+ "read_buffers_allowed":8,
+ "grouped_workqueues":[
+ {
+ "dev":"wqX.0",
+ "mode":"shared",
+ "size":16,
+ "group_id":0,
+ "priority":10,
+ "block_on_fault":1,
+ "type":"user",
+ "name":"appX0",
+ "threshold":15
+ }
+ ],
+ "grouped_engines":[
+ {
+ "dev":"engineX.0",
+ "group_id":0
+ },
+ ]
+ },
+ {
+ "dev":"groupX.1",
+ "read_buffers_reserved":0,
+ "use_read_buffer_limit":0,
+ "read_buffers_allowed":8,
+ "grouped_workqueues":[
+ {
+ "dev":"wqX.1",
+ "mode":"shared",
+ "size":16,
+ "group_id":1,
+ "priority":10,
+ "block_on_fault":1,
+ "type":"user",
+ "name":"appX1",
+ "threshold":15
+ }
+ ],
+ "grouped_engines":[
+ {
+ "dev":"engineX.1",
+ "group_id":1
+ },
+ ]
+ },
+ {
+ "dev":"groupX.2",
+ "read_buffers_reserved":0,
+ "use_read_buffer_limit":0,
+ "read_buffers_allowed":8,
+ "grouped_workqueues":[
+ {
+ "dev":"wqX.2",
+ "mode":"shared",
+ "size":16,
+ "group_id":2,
+ "priority":10,
+ "block_on_fault":1,
+ "type":"user",
+ "name":"appX2",
+ "threshold":15
+ }
+ ],
+ "grouped_engines":[
+ {
+ "dev":"engineX.2",
+ "group_id":2
+ },
+ ]
+ },
+ {
+ "dev":"groupX.3",
+ "read_buffers_reserved":0,
+ "use_read_buffer_limit":0,
+ "read_buffers_allowed":8,
+ "grouped_workqueues":[
+ {
+ "dev":"wqX.3",
+ "mode":"shared",
+ "size":16,
+ "group_id":3,
+ "priority":10,
+ "block_on_fault":1,
+ "type":"user",
+ "name":"appX3",
+ "threshold":15
+ }
+ ],
+ "grouped_engines":[
+ {
+ "dev":"engineX.3",
+ "group_id":3
+ },
+ ]
+ },
+ ]
+ }
+ ]
+{{- end }}
\ No newline at end of file
diff --git a/templates/dsa.yaml b/templates/dsa.yaml
index 896b42c..411f867 100755
--- a/templates/dsa.yaml
+++ b/templates/dsa.yaml
@@ -14,7 +14,7 @@ spec:
sharedDevNum: {{ .Values.sharedDevNum }}
logLevel: {{ .Values.logLevel }}
nodeSelector: {{- .Values.nodeSelector | toYaml | nindent 4 }}
-
+ provisioningConfig: {{ include "dsa.provisioningConfig" . }}
---
{{ if eq .Values.nodeFeatureRule true }}
apiVersion: nfd.k8s-sigs.io/v1alpha1
diff --git a/values.yaml b/values.yaml
index 9598856..0fd9de5 100755
--- a/values.yaml
+++ b/values.yaml
@@ -14,4 +14,6 @@ logLevel: 4
nodeSelector:
intel.feature.node.kubernetes.io/dsa: 'true'
-nodeFeatureRule: false
\ No newline at end of file
+nodeFeatureRule: true
+annotations: {}
+overrideConfig: {}
\ No newline at end of file
--
2.34.1

View File

@ -0,0 +1 @@
0001-Enable-custom-DSA-configuration-via-helm-overrides.patch

View File

@ -0,0 +1,24 @@
#!/usr/bin/make -f
export DH_VERBOSE = 1
export ROOT = debian/tmp
export APP_FOLDER = $(ROOT)/usr/lib/helm
%:
dh $@
override_dh_auto_build:
mkdir -p intel-device-plugins-dsa
cp Chart.yaml intel-device-plugins-dsa
cp -r templates/ intel-device-plugins-dsa
cp values.yaml intel-device-plugins-dsa
make intel-device-plugins-dsa
override_dh_auto_install:
# Install the app tar file.
install -d -m 755 $(APP_FOLDER)
install -p -D -m 755 intel-device-plugins-dsa*.tgz $(APP_FOLDER)
override_dh_auto_test:

View File

@ -0,0 +1 @@
3.0 (quilt)

View File

@ -0,0 +1,14 @@
---
debname: intel-device-plugins-dsa-helm
debver: 0.29-0
dl_path:
name: intel-device-plugins-dsa-v0.29.0.tgz
url: https://github.com/intel/helm-charts/releases/download/intel-device-plugins-dsa-0.29.0/intel-device-plugins-dsa-0.29.0.tgz
sha256sum: cb653afb09f1830d211857846ee7d20a6b9f909174d2d64ebcb8f481c62f5609
src_files:
- intel-device-plugins-dsa-helm/files/Makefile
revision:
dist: $STX_DIST
GITREVCOUNT:
SRC_DIR: ${MY_REPO}/stx/app-intel-device-plugins/intel-device-plugins-dsa-helm
BASE_SRCREV: 99a5c80d3cd9d07b695ed57fbd29c72f1844f66a

View File

@ -0,0 +1,42 @@
#
# Copyright 2017 The Openstack-Helm Authors.
#
# Copyright (c) 2024 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
# It's necessary to set this because some environments don't link sh -> bash.
SHELL := /bin/bash
TASK := build
EXCLUDES := doc tests tools logs tmp
CHARTS := $(filter-out $(EXCLUDES), $(patsubst %/.,%,$(wildcard */.)))
.PHONY: $(EXCLUDES) $(CHARTS)
all: $(CHARTS)
$(CHARTS):
@if [ -d $@ ]; then \
echo; \
echo "===== Processing [$@] chart ====="; \
make $(TASK)-$@; \
fi
init-%:
if [ -f $*/Makefile ]; then make -C $*; fi
lint-%: init-%
if [ -d $* ]; then helm lint $*; fi
build-%: lint-%
if [ -d $* ]; then helm package $*; fi
clean:
@echo "Clean all build artifacts"
rm -f */templates/_partials.tpl */templates/_globals.tpl
rm -f *tgz */charts/*tgz
rm -rf */charts */tmpcharts
%:
@:

View File

@ -5,5 +5,5 @@ src_path: k8sapp_intel_device_plugins_operator
revision:
dist: $STX_DIST
GITREVCOUNT:
SRC_DIR: ${MY_REPO}/stx/app-intel-device-plugins/intel-device-plugins-operator-helm
SRC_DIR: ${MY_REPO}/stx/app-intel-device-plugins
BASE_SRCREV: 99a5c80d3cd9d07b695ed57fbd29c72f1844f66a

View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2023 Wind River Systems, Inc.
# Copyright (c) 2024 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
@ -14,6 +14,7 @@ HELM_NS_INTEL_DEVICE_PLUGINS_OPERATOR = 'intel-device-plugins-operator'
HELM_CHART_INTEL_DEVICE_PLUGINS_OPERATOR = 'intel-device-plugins-operator'
# Helm: Supported charts:
HELM_CHART_INTEL_DEVICE_PLUGINS_DSA = 'intel-device-plugins-dsa'
HELM_CHART_INTEL_DEVICE_PLUGINS_QAT = 'intel-device-plugins-qat'
# Label

View File

@ -0,0 +1,75 @@
#
# Copyright (c) 2024 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
from sysinv.common import exception
from sysinv.helm import base
from k8sapp_intel_device_plugins_operator.common import constants as app_constants
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
class IntelDevicePluginsDsaHelm(base.FluxCDBaseHelm):
"""Class to encapsulate helm operations for the intel-device-plugins-dsa chart."""
SUPPORTED_NAMESPACES = base.BaseHelm.SUPPORTED_NAMESPACES + \
[app_constants.HELM_NS_INTEL_DEVICE_PLUGINS_OPERATOR]
SUPPORTED_APP_NAMESPACES = {
app_constants.HELM_APP_INTEL_DEVICE_PLUGINS_OPERATOR:
base.BaseHelm.SUPPORTED_NAMESPACES +
[app_constants.HELM_NS_INTEL_DEVICE_PLUGINS_OPERATOR],
}
CHART = app_constants.HELM_CHART_INTEL_DEVICE_PLUGINS_DSA
HELM_RELEASE = app_constants.HELM_CHART_INTEL_DEVICE_PLUGINS_DSA
SERVICE_NAME = app_constants.HELM_CHART_INTEL_DEVICE_PLUGINS_DSA
def _is_enabled(self, app_name, chart_name, namespace):
"""
Check if the chart is enable at a system level
:param app_name: Application name
:param chart_name: Chart supplied with the application
:param namespace: Namespace where the chart will be executed
Returns true by default if an exception occurs as most charts are
enabled.
"""
return super(IntelDevicePluginsDsaHelm, self)._is_enabled(
app_name, chart_name, namespace)
def execute_kustomize_updates(self, operator):
"""
Update the elements of FluxCD kustomize manifests.
This allows a helm chart plugin to use the FluxCDKustomizeOperator to
make dynamic structural changes to the application manifest based on the
current conditions in the platform
Changes currenty include updates to the top level kustomize manifest to
disable helm releases.
:param operator:
an instance of the FluxCDKustomizeOperator
"""
if not self._is_enabled(operator.APP, self.CHART,
app_constants.HELM_NS_INTEL_DEVICE_PLUGINS_OPERATOR):
operator.helm_release_resource_delete(self.HELM_RELEASE)
def get_overrides(self, namespace=None):
overrides = {
app_constants.HELM_NS_INTEL_DEVICE_PLUGINS_OPERATOR: {}
}
if namespace in self.SUPPORTED_NAMESPACES:
return overrides[namespace]
elif namespace:
raise exception.InvalidHelmNamespace(
chart=self.CHART, namespace=namespace)
else:
return overrides

View File

@ -36,6 +36,7 @@ systemconfig.helm_applications =
systemconfig.helm_plugins.intel_device_plugins_operator =
001_intel-device-plugins-operator = k8sapp_intel_device_plugins_operator.helm.intel_device_plugins_operator:IntelDevicePluginsOperatorHelm
002_intel-device-plugins-qat = k8sapp_intel_device_plugins_operator.helm.intel_device_plugins_qat:IntelDevicePluginsQatHelm
003_intel-device-plugins-dsa = k8sapp_intel_device_plugins_operator.helm.intel_device_plugins_dsa:IntelDevicePluginsDsaHelm
systemconfig.fluxcd.kustomize_ops =
intel-device-plugins-operator = k8sapp_intel_device_plugins_operator.kustomize.kustomize_intel_device_plugins:IntelDevicePluginsFluxCDKustomizeOperator

View File

@ -6,6 +6,7 @@ Build-Depends: debhelper-compat (= 13),
helm,
python3-k8sapp-intel-device-plugins-operator,
python3-k8sapp-intel-device-plugins-operator-wheels,
intel-device-plugins-dsa-helm,
intel-device-plugins-qat-helm,
intel-device-plugins-operator-helm,
build-info

View File

@ -3,7 +3,7 @@ Upstream-Name: stx-intel-device-plugins-operator-helm
Source: https://opendev.org/starlingx/app-intel-device-plugins/
Files: *
Copyright: (c) 2023 Wind River Systems, Inc
Copyright: (c) 2024 Wind River Systems, Inc
License: Apache-2
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.

View File

@ -5,6 +5,7 @@ maintain_user_overrides: true
maintain_attributes: true
disabled_charts:
- intel-device-plugins-dsa
- intel-device-plugins-qat
upgrades:

View File

@ -0,0 +1,7 @@
#
# Copyright (c) 2024 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
nodeFeatureRule: true

View File

@ -0,0 +1,5 @@
#
# Copyright (c) 2024 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#

View File

@ -0,0 +1,36 @@
#
# Copyright (c) 2024 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
apiVersion: "helm.toolkit.fluxcd.io/v2beta1"
kind: HelmRelease
metadata:
name: intel-device-plugins-dsa
labels:
chart_group: intel-device-plugins-operator-charts
spec:
releaseName: intel-device-plugins-dsa
chart:
spec:
chart: intel-device-plugins-dsa
version: 0.29.0
sourceRef:
kind: HelmRepository
name: stx-platform
interval: 1m
timeout: 30m
test:
enable: false
install:
disableHooks: false
upgrade:
disableHooks: false
valuesFrom:
- kind: Secret
name: dsa-static-overrides
valuesKey: dsa-static-overrides.yaml
- kind: Secret
name: dsa-system-overrides
valuesKey: dsa-system-overrides.yaml

View File

@ -0,0 +1,18 @@
#
# Copyright (c) 2024 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
namespace: intel-device-plugins-operator
resources:
- helmrelease.yaml
secretGenerator:
- name: dsa-static-overrides
files:
- dsa-static-overrides.yaml
- name: dsa-system-overrides
files:
- dsa-system-overrides.yaml
generatorOptions:
disableNameSuffixHash: true

View File

@ -9,5 +9,6 @@ kind: Kustomization
namespace: intel-device-plugins-operator
resources:
- base
- intel-device-plugins-dsa
- intel-device-plugins-operator
- intel-device-plugins-qat