kernel-modules: re-enable qat and octeon_ep OOT drivers

The drivers qat and octeon_ep are known to only exist as OOT,
because the in-tree drivers of 6.6.7 kernel do not support the
virtual function for both qat and octeon_ep.

We encountered a few version compatibility issues after upgrading
the kernel to 6.6.7 version. We adapted drivers' code to the kernel
6.6.7 by referring to the upstream commits.

We also upgrade the qat to the version 1.1.40-00018 from 1.0.20-00008,
because there is a risk if we adapted the qat-1.0.20-00008 driver to
the kernel 6.6.7. The code changes involve the changes of the
initialization logic and also involve the several files changes
because that the function of devm_device_remove_group() has been
removed in kernel 6.6.7.

Verification:
 * An ISO image can be built with this commit applied to a repo
   project of StarlingX's kernel 6.6.7 branch.
 * The ISO image can be installed to a Dell XR11 server with a DORA
   card and a server with Intel QuickAssist Technology Hardware Version,
   and the system is successfully Ansible-bootstrapped.
 * The octeon_ep driver can be loaded manually with modprobe, and a
   PF interface is instantiated by the kernel and can transfer packets.
 * Virtual function (VF) interfaces can be brought up and transfer
   packets.
 * qat_service status/start/shutdown works well.
 * cpa_sample_code can run successfully.

Story: 2011056
Task: 49775

Change-Id: Icf19340396d99bde97533bfecf22472e01f6be34
Signed-off-by: Jiping Ma <jiping.ma2@windriver.com>
(cherry picked from commit 8bdd6fe0d0)
Signed-off-by: Jiping Ma <jiping.ma2@windriver.com>
This commit is contained in:
Jiping Ma 2024-04-08 03:05:10 -04:00
parent ba86fee885
commit 0cb2ed82dd
14 changed files with 282 additions and 127 deletions

View File

@ -17,8 +17,8 @@ ice
ice-rt
# Marvell kernel modules
#octeon-ep
#octeon-ep-rt
octeon-ep
octeon-ep-rt
#intel-igb_uio
igb-uio
@ -57,8 +57,8 @@ linux-perf
#mlnx-ofed-kernel-utils
#qat20
#qat2.0.l
#qat2.0.l-rt
qat2.0.l
qat2.0.l-rt
#rdma-core
ibacm

View File

@ -7,8 +7,8 @@ kernel-modules/intel-igb_uio
kernel-modules/intel-opae-fpga
kernel-modules/intel-qv
#kernel-modules/mlnx-ofa_kernel
#kernel-modules/qat
#kernel-modules/octeon_ep
kernel-modules/qat
kernel-modules/octeon_ep
userspace/broadcom/libbnxt_re
userspace/mellanox/rdma-core
userspace/mellanox/mstflint

View File

@ -7,5 +7,5 @@ kernel-modules/intel-igb_uio
kernel-modules/intel-opae-fpga
kernel-modules/intel-qv
#kernel-modules/mlnx-ofa_kernel
#kernel-modules/qat
#kernel-modules/octeon_ep
kernel-modules/qat
kernel-modules/octeon_ep

View File

@ -0,0 +1,71 @@
From 974bed2e482e5ff6bb4337a5bd8d5a288aa20af4 Mon Sep 17 00:00:00 2001
From: Bjorn Helgaas <bhelgaas@google.com>
Date: Tue, 7 Mar 2023 12:19:21 -0600
Subject: [PATCH 3/4] octeon_ep: Drop redundant
pci_enable_pcie_error_reporting()
pci_enable_pcie_error_reporting() enables the device to send ERR_*
Messages. Since f26e58bf6f54 ("PCI/AER: Enable error reporting when AER is
native"), the PCI core does this for all devices during enumeration, so the
driver doesn't need to do it itself.
Remove the redundant pci_enable_pcie_error_reporting() call from the
driver. Also remove the corresponding pci_disable_pcie_error_reporting()
from the driver .remove() path.
Note that this only controls ERR_* Messages from the device. An ERR_*
Message may cause the Root Port to generate an interrupt, depending on the
AER Root Error Command register managed by the AER service driver.
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Veerasenareddy Burru <vburru@marvell.com>
Cc: Abhijit Ayarekar <aayarekar@marvell.com>
Acked-by: Jesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit fe3f4c292da1908eda5d322e60b3c09ed9508288)
[jma: This commit was cherry-picked from
https://git.yoctoproject.org/linux-yocto/commit/?h=fe3f4c292da1]
Signed-off-by: Jiping Ma <jiping.ma2@windriver.com>
---
drivers/octeon_ep/octep_main.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/drivers/octeon_ep/octep_main.c b/drivers/octeon_ep/octep_main.c
index 675f7ab..43f388e 100644
--- a/drivers/octeon_ep/octep_main.c
+++ b/drivers/octeon_ep/octep_main.c
@@ -8,7 +8,6 @@
#include <linux/types.h>
#include <linux/module.h>
#include <linux/pci.h>
-#include <linux/aer.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/rtnetlink.h>
@@ -1744,7 +1743,6 @@ static int octep_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
goto err_pci_regions;
}
- pci_enable_pcie_error_reporting(pdev);
octep_enable_ptm(pdev);
pci_set_master(pdev);
@@ -1773,7 +1771,6 @@ static int octep_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
return 0;
err_alloc_netdev:
- pci_disable_pcie_error_reporting(pdev);
pci_release_mem_regions(pdev);
err_pci_regions:
err_dma_mask:
@@ -1817,7 +1814,6 @@ static void octep_remove(struct pci_dev *pdev)
free_resources:
pci_release_mem_regions(pdev);
free_netdev(oct->netdev);
- pci_disable_pcie_error_reporting(pdev);
pci_disable_device(pdev);
}
--
2.42.0

View File

@ -0,0 +1,56 @@
From b8e844961181909fcb44549ac47e3217276108a8 Mon Sep 17 00:00:00 2001
From: Jiping Ma <jiping.ma2@windriver.com>
Date: Sun, 10 Mar 2024 22:50:06 -0700
Subject: [PATCH 4/4] phc: convert .adjfreq to .adjfine
This commit accommodates commit 75ab70ec5cef ("ptp: remove the
.adjfreq interface function"), which removes .adjfreq interface
function. This commit was merged in the v6.4 development cycle
and causes compilation failures with the v6.6 kernel, which are
listed below.
References:
* https://git.yoctoproject.org/linux-yocto/commit/?h=2e77eded8ec3
* https://git.yoctoproject.org/linux-yocto/commit/?h=75ab70ec5cef
Resolved compilation errors:
drivers/phc/ep_phc.c:265:3: error: struct ptp_clock_info has no member \
named adjfreq
265 | .adjfreq = oct_ep_ptp_adjfreq,
| ^~~~~~~
drivers/phc/ep_phc.c:265:13: error: initialization of struct ptp_pin_desc\
from incompatible pointer type int (*)(struct ptp_clock_info *, s32) \
{int (*)(struct ptp_clock_info *, int)} [-Werror=incompatible-pointer-types]
265 | .adjfreq = oct_ep_ptp_adjfreq,
Signed-off-by: Jiping Ma <jiping.ma2@windriver.com>
---
drivers/phc/ep_phc.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/phc/ep_phc.c b/drivers/phc/ep_phc.c
index d381e36..cc4917f 100644
--- a/drivers/phc/ep_phc.c
+++ b/drivers/phc/ep_phc.c
@@ -238,7 +238,7 @@ static int oct_ep_ptp_enable(struct ptp_clock_info *ptp,
/* Nothing to do here, PTP hardware is enabled by EP */
return 0;
}
-static int oct_ep_ptp_adjfreq(struct ptp_clock_info *ptp, s32 ppb)
+static int oct_ep_ptp_adjfine(struct ptp_clock_info *ptp, long ppb)
{
return -ENOTSUPP;
}
@@ -262,7 +262,7 @@ static struct ptp_clock_info oct_ep_ptp_caps = {
.n_ext_ts = 0,
.n_pins = 0,
.pps = 0,
- .adjfreq = oct_ep_ptp_adjfreq,
+ .adjfine = oct_ep_ptp_adjfine,
.adjtime = oct_ep_ptp_adjtime,
.gettime64 = oct_ep_ptp_gettime_cn9xxx,
.settime64 = oct_ep_ptp_settime,
--
2.42.0

View File

@ -1,2 +1,4 @@
0001-octeon_ep-print-firmware-ready-status-only-once.patch
0002-phc-print-firmware-ready-status-only-once.patch
0003-octeon_ep-Drop-redundant-pci_enable_pcie_error_repor.patch
0004-phc-convert-.adjfreq-to-.adjfine.patch

View File

@ -1,3 +1,10 @@
qat2.0.l (1.1.40-00018) unstable; urgency=medium
* Upgrade to version 1.1.40-00018 from 1.0.20-00008
-- Jiping Ma <jiping.ma2@windriver.com> Thu, 28 Mar 2024 17:28:26 +0800
qat2.0.l (1.0.20-00008) unstable; urgency=medium
* Upgrade to version 2.0

View File

@ -1,7 +1,7 @@
From 84ab4e5497b83f04b19f9e101bdb18e562f5454d Mon Sep 17 00:00:00 2001
From 9413df478277bc8dd55fc17aee3c5423497e6e0d Mon Sep 17 00:00:00 2001
From: "M. Vefa Bicakci" <vefa.bicakci@windriver.com>
Date: Mon, 11 Dec 2023 10:33:55 -0500
Subject: [PATCH] Add placeholder version to shared library soname
Subject: [PATCH 1/3] Add placeholder version to shared library soname
This patch adds a placeholder version ("0") to the user-space shared
libraries built by the qat2.0.l package, so that Debian packaging tools
@ -75,7 +75,7 @@ Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/quickassist/build_system/build_files/OS/linux_2.6.mk b/quickassist/build_system/build_files/OS/linux_2.6.mk
index 651a0e3f043a..625382613287 100644
index 237084c..19432bd 100644
--- a/quickassist/build_system/build_files/OS/linux_2.6.mk
+++ b/quickassist/build_system/build_files/OS/linux_2.6.mk
@@ -61,7 +61,7 @@ EXTRA_CFLAGS+= -Wno-div-by-zero -Wfloat-equal -Wtraditional -Wundef -Wno-endif-
@ -87,3 +87,6 @@ index 651a0e3f043a..625382613287 100644
LIB_STATIC_FLAGS=
EXE_FLAGS?=
--
2.42.0

View File

@ -1,72 +0,0 @@
From 7065771c416d80bcd3cff7cb390602dc8e893b97 Mon Sep 17 00:00:00 2001
From: Alexander Sverdlin <alexander.sverdlin@nokia.com>
Date: Tue, 23 Jul 2019 07:24:01 +0000
Subject: [PATCH 1/3] crypto: qat - Silence smp_processor_id() warning
[ commit 1b82feb6c5e1996513d0fb0bbb475417088b4954 upstream ]
It seems that smp_processor_id() is only used for a best-effort
load-balancing, refer to qat_crypto_get_instance_node(). It's not feasible
to disable preemption for the duration of the crypto requests. Therefore,
just silence the warning. This commit is similar to e7a9b05ca4
("crypto: cavium - Fix smp_processor_id() warnings").
Silences the following splat:
BUG: using smp_processor_id() in preemptible [00000000] code: cryptomgr_test/2904
caller is qat_alg_ablkcipher_setkey+0x300/0x4a0 [intel_qat]
CPU: 1 PID: 2904 Comm: cryptomgr_test Tainted: P O 4.14.69 #1
...
Call Trace:
dump_stack+0x5f/0x86
check_preemption_disabled+0xd3/0xe0
qat_alg_ablkcipher_setkey+0x300/0x4a0 [intel_qat]
skcipher_setkey_ablkcipher+0x2b/0x40
__test_skcipher+0x1f3/0xb20
? cpumask_next_and+0x26/0x40
? find_busiest_group+0x10e/0x9d0
? preempt_count_add+0x49/0xa0
? try_module_get+0x61/0xf0
? crypto_mod_get+0x15/0x30
? __kmalloc+0x1df/0x1f0
? __crypto_alloc_tfm+0x116/0x180
? crypto_skcipher_init_tfm+0xa6/0x180
? crypto_create_tfm+0x4b/0xf0
test_skcipher+0x21/0xa0
alg_test_skcipher+0x3f/0xa0
alg_test.part.6+0x126/0x2a0
? finish_task_switch+0x21b/0x260
? __schedule+0x1e9/0x800
? __wake_up_common+0x8d/0x140
cryptomgr_test+0x40/0x50
kthread+0xff/0x130
? cryptomgr_notify+0x540/0x540
? kthread_create_on_node+0x70/0x70
ret_from_fork+0x24/0x50
Fixes: ed8ccaef52 ("crypto: qat - Add support for SRIOV")
Cc: stable@vger.kernel.org
Signed-off-by: Alexander Sverdlin <alexander.sverdlin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: zhao.shuai <zhaos@neusoft.com>
[zp: Adapted the patch for context changes.]
Signed-off-by: Peng Zhang <Peng.Zhang2@windriver.com>
---
quickassist/qat/drivers/crypto/qat/qat_common/adf_common_drv.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/quickassist/qat/drivers/crypto/qat/qat_common/adf_common_drv.h b/quickassist/qat/drivers/crypto/qat/qat_common/adf_common_drv.h
index abab8db..16cdc58 100644
--- a/quickassist/qat/drivers/crypto/qat/qat_common/adf_common_drv.h
+++ b/quickassist/qat/drivers/crypto/qat/qat_common/adf_common_drv.h
@@ -74,7 +74,7 @@ struct service_hndl {
static inline int get_current_node(void)
{
- return topology_physical_package_id(smp_processor_id());
+ return topology_physical_package_id(raw_smp_processor_id());
}
int adf_service_register(struct service_hndl *service);
--
2.25.1

View File

@ -1,37 +0,0 @@
From a41966a2056b6bc873b7314457c4c965f1b7d59b Mon Sep 17 00:00:00 2001
From: Li Zhou <li.zhou@windriver.com>
Date: Mon, 28 Jun 2021 11:37:20 +0800
Subject: [PATCH 3/3] Add mcx16 cflag to solve linking issue
Linking error is reported as:
MODPOST .../quickassist/lookaside/access_layer/src/Module.symvers
ERROR: modpost: "__sync_bool_compare_and_swap_16"
[.../quickassist/lookaside/access_layer/src/qat_api.ko] undefined!
Add -mcx16 option to enables GCC to generate CMPXCHG16B instructions
in 64-bit code to implement compare-and-exchange operations on 16-byte
aligned 128-bit objects.
Signed-off-by: Li Zhou <li.zhou@windriver.com>
[zp: Adapted the patch for context changes.]
Signed-off-by: Peng Zhang <Peng.Zhang2@windriver.com>
---
quickassist/lookaside/access_layer/src/common/utils/Makefile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/quickassist/lookaside/access_layer/src/common/utils/Makefile b/quickassist/lookaside/access_layer/src/common/utils/Makefile
index f286394..a131a2e 100644
--- a/quickassist/lookaside/access_layer/src/common/utils/Makefile
+++ b/quickassist/lookaside/access_layer/src/common/utils/Makefile
@@ -77,6 +77,8 @@ ifdef ICP_DC_RETURN_COUNTERS_ON_ERROR
EXTRA_CFLAGS += -DICP_DC_RETURN_COUNTERS_ON_ERROR
endif
+EXTRA_CFLAGS+=-mcx16
+
#common includes between all supported OSes
INCLUDES+=-I$(LAC_DIR)/src/common/include\
-I$(LAC_DIR)/src/common/compression/include\
--
2.25.1

View File

@ -0,0 +1,65 @@
From 7e55a0aa8c148f34ec342ec6e26d19319d99eebf Mon Sep 17 00:00:00 2001
From: Jiping Ma <jiping.ma2@windriver.com>
Date: Wed, 13 Mar 2024 23:31:56 -0700
Subject: [PATCH 2/3] Remove redundant pci_enable_pcie_error_reporting()
This commit accommodates commit 7ec4b34be423 ("PCI/AER: Unexport
pci_enable_pcie_error_reporting()") merged in the v6.6 development cycle,
by applying changes similar to the ones found in commit ba153552c18d
("ice: Remove redundant pci_enable_pcie_error_reporting()") to resolve the
qat driver compilation failures encountered with the v6.6 kernel, which are
listed below.
References:
* https://git.yoctoproject.org/linux-yocto/commit/?h=7ec4b34be423
* https://git.yoctoproject.org/linux-yocto/commit/?h=ba153552c18d
Resolved compilation errors:
quickassist/qat/drivers/crypto/qat/qat_common/adf_aer.c:378:8: error: \
implicit declaration of function pci_enable_pcie_error_reporting \
[-Werror=implicit-function-declaration]
378 | ret = pci_enable_pcie_error_reporting(pdev);
Signed-off-by: Jiping Ma <jiping.ma2@windriver.com>
---
.../qat/drivers/crypto/qat/qat_common/adf_aer.c | 11 -----------
1 file changed, 11 deletions(-)
diff --git a/quickassist/qat/drivers/crypto/qat/qat_common/adf_aer.c b/quickassist/qat/drivers/crypto/qat/qat_common/adf_aer.c
index ba277e0..4c56074 100644
--- a/quickassist/qat/drivers/crypto/qat/qat_common/adf_aer.c
+++ b/quickassist/qat/drivers/crypto/qat/qat_common/adf_aer.c
@@ -370,19 +370,11 @@ void adf_exit_fatal_error_wq(void)
*/
int adf_enable_aer(struct adf_accel_dev *accel_dev, struct pci_driver *adf)
{
- int ret;
- struct pci_dev *pdev = accel_to_pci_dev(accel_dev);
-
#ifdef QAT_UIO
adf->err_handler = (struct pci_error_handlers *)(&adf_err_handler);
#else
adf->err_handler = &adf_err_handler;
#endif
- ret = pci_enable_pcie_error_reporting(pdev);
- if (ret)
- dev_warn(&pdev->dev,
- "QAT: Failed to enable AER, error code %d\n", ret);
-
return 0;
}
EXPORT_SYMBOL_GPL(adf_enable_aer);
@@ -399,9 +391,6 @@ EXPORT_SYMBOL_GPL(adf_enable_aer);
*/
void adf_disable_aer(struct adf_accel_dev *accel_dev)
{
- struct pci_dev *pdev = accel_to_pci_dev(accel_dev);
-
- pci_disable_pcie_error_reporting(pdev);
}
EXPORT_SYMBOL_GPL(adf_disable_aer);
--
2.42.0

View File

@ -0,0 +1,60 @@
From 9dbbbde60fe73b2f0b17062a6b775d2fae1317ff Mon Sep 17 00:00:00 2001
From: Jiping Ma <jiping.ma2@windriver.com>
Date: Thu, 28 Mar 2024 22:12:23 -0700
Subject: [PATCH 3/3] qat: rename invalidate_range notifier
This commit accommodates commit 1af5a8109904 ("mmu_notifiers: rename
invalidate_range notifier") which renames invalidate_range to
arch_invalidate_secondary_tlbs. This commit was merged in the v6.6
development cycle and causes compilation failures with the v6.6 kernel,
which are listed below.
References:
* https://git.yoctoproject.org/linux-yocto/commit/?h=1af5a8109904
Resolved compilation errors:
quickassist/qat/drivers/crypto/qat/qat_common/adf_pasid.c:184:3: error:\
const struct mmu_notifier_ops has no member named invalidate_range; did\
you mean invalidate_range_end?
184 | .invalidate_range = adf_dev_mm_invalidate_range,
| ^~~~~~~~~~~~~~~~
| invalidate_range_end
quickassist/qat/drivers/crypto/qat/qat_common/adf_pasid.c:184:22: \
error: initialization of int (*)(struct mmu_notifier *, struct \
mm_struct *, long unsigned int, long unsigned int) from \
incompatible pointer type void (*)(struct mmu_notifier *, \
struct mm_struct *, long unsigned int, long unsigned int)\
[-Werror=incompatible-pointer-types]
184 | .invalidate_range = adf_dev_mm_invalidate_range,
Signed-off-by: Jiping Ma <jiping.ma2@windriver.com>
---
quickassist/qat/drivers/crypto/qat/qat_common/adf_pasid.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/quickassist/qat/drivers/crypto/qat/qat_common/adf_pasid.c b/quickassist/qat/drivers/crypto/qat/qat_common/adf_pasid.c
index 1871663..64586d9 100644
--- a/quickassist/qat/drivers/crypto/qat/qat_common/adf_pasid.c
+++ b/quickassist/qat/drivers/crypto/qat/qat_common/adf_pasid.c
@@ -171,7 +171,7 @@ static void adf_dev_mm_release(struct mmu_notifier *mn,
}
}
-static void adf_dev_mm_invalidate_range(struct mmu_notifier *mn,
+static void adf_dev_mm_arch_invalidate_secondary_tlbs(struct mmu_notifier *mn,
struct mm_struct *mm,
unsigned long start,
unsigned long end)
@@ -181,7 +181,7 @@ static void adf_dev_mm_invalidate_range(struct mmu_notifier *mn,
static const struct mmu_notifier_ops adf_dev_mmu_notifier_ops = {
.release = adf_dev_mm_release,
- .invalidate_range = adf_dev_mm_invalidate_range,
+ .arch_invalidate_secondary_tlbs = adf_dev_mm_arch_invalidate_secondary_tlbs,
};
#if (KERNEL_VERSION(5, 7, 0) <= LINUX_VERSION_CODE)
--
2.42.0

View File

@ -1,3 +1,3 @@
0001-crypto-qat-Silence-smp_processor_id-warning.patch
0002-Add-mcx16-cflag-to-solve-linking-issue.patch
0003-Add-placeholder-version-to-shared-library-soname.patch
0001-Add-placeholder-version-to-shared-library-soname.patch
0002-Remove-redundant-pci_enable_pcie_error_reporting.patch
0003-qat-rename-invalidate_range-notifier.patch

View File

@ -1,10 +1,10 @@
---
debver: 1.0.20-00008
debver: 1.1.40-00018
debname: qat2.0.l
dl_path:
name: QAT2.0.L.1.0.20-00008.tar.gz
url: https://downloadmirror.intel.com/777529/QAT20.L.1.0.20-00008.tar.gz
sha256sum: 8b8a46897217a806eeca911fe7a5040183fb32897ad20bafedcc07d711436799
name: QAT20.L.1.1.40-00018.tar.gz
url: https://downloadmirror.intel.com/818498/QAT20.L.1.1.40-00018.tar.gz
sha256sum: 941cd7aed2081b36e84d157510b7c31a5121e9df7a676ecfbc92be71ae395884
revision:
dist: $STX_DIST
GITREVCOUNT: