starlingx/kernel
Code Issues Proposed changes
Files
0ae9f6bd537b726950712ac1ee0d5a6a4f6f82f5
kernel/kernel-modules/bnxt_en/centos/patches/0001-bnxt_re-Makefile-Adapt-to-mlnx-ofa_kernel-for-Starli.patch
M. Vefa Bicakci 0ae9f6bd53 Add Broadcom NetXtreme-E drivers 
This commit introduces Broadcom's bnxt_en and bnxt_re out-of-tree
drivers, version 220.0.13.0, to StarlingX, and disables the in-tree
bnxt_en driver. (For clarity, the main driver package file name is
"bcm_220.0.83.0.tar.gz".)

There is a need to build Broadcom's driver against the version of the
Infiniband/RDMA code base bundled with Mellanox OFED (OpenFabrics
Enterprise Distribution), because the in-tree kernel Infiniband/RDMA
modules are disabled, and Mellanox OFED's out-of-tree Infiniband/RDMA
modules replace the in-tree versions.

A custom spec file was used, despite the fact that the Broadcom device
driver package includes a source RPM file. This was necessary due to two
reasons:

- The source RPM file has whitespace between its field labels (such as
  Name, Version, Release, and so on) and the values of these fields.
  (Example: "Name   :  bnxt_en") This unfortunately breaks "build-pkgs
  --edit", which cannot handle the whitespace during the extraction of
  the source RPM.

- More importantly, the spec file included by Broadcom has the following
  lines in its %prep section, which does not use the %setup macro:

  %prep
  #%setup
  mkdir -p source
  tar -xzvf %{SOURCE0} -C source/
  mv -f source/netxtreme-%name-%{version}-220.0.13.0/* source/
  rm -rf source/netxtreme-%name-%{version}-220.0.13.0/
  cp ./source/%{bnxt_re_driver}/README.TXT
  ./source/%{bnxt_re_driver}/%{bnxt_re_driver}_README.TXT
  cp ./source/%{bnxt_en_driver}/README.TXT
  ./source/%{bnxt_en_driver}/%{bnxt_en_driver}_README.TXT
  mkdir -p obj

  This is problematic, because the "build-pkgs --edit" procedure
  extracts the source RPM multiple times into the same directory to
  ensure that the extraction is stable. However, the %prep section
  quoted above does not remove the 'source' directory before attempting
  to populate it. (Unfortunately, this issue is not patchable using a
  meta-patch, because the extraction occurs even before the application
  of meta-patches.) This in turn triggers the following errors:

  + mv -f source/netxtreme-bnxt_en-1.10.2-220.0.13.0/Makefile \
    source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_en \
    source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_en-1.10.2-220.0.13.0 \
    source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_re \
    source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_re-220.0.12.0 \
    source/
  mv: cannot move 'source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_en-1.10.2-220.0.13.0' \
    to 'source/bnxt_en-1.10.2-220.0.13.0': Directory not empty
  mv: cannot move 'source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_re-220.0.12.0' \
    to 'source/bnxt_re-220.0.12.0': Directory not empty
  error: Bad exit status from .../std/srpm_work/bnxt_en/var/tmp/rpm-tmp.cDTpOP (%prep)
      Bad exit status from .../std/srpm_work/bnxt_en/var/tmp/rpm-tmp.cDTpOP (%prep)

The changes made to the spec file are as follows:

- Reliance on Red Hat's kernel module packaging RPM spec macros (such as
  %kernel_module_package) is removed, because the aforementioned macros
  make use of the kmodtool helper program, which automatically generates
  an RPM post-installation and post-uninstallation sections making use
  of the weak-modules utility, which StarlingX does not use/support. It
  is not possible to disable this behaviour of the kmodtool script
  ("/usr/lib/rpm/redhat/kmodtool") with a command line option or an
  environment variable.

- The spec file is reorganized to build the kernel module against only a
  single kernel "flavour", as StarlingX does not use the kernel flavour
  concept.

- The %build process is modified so that instead of using the top-level
  makefile (which does not take certain variables as arguments), the
  bnxt_en- and bnxt_re-specific makefiles are called with make
  separately. This allows passing KDIR, KVER and OFED_VERSION make
  variables as command line arguments to make.

- The use of the devtoolset-8 software collection was added to align to
  the kernel's build toolchain.

- The spec file is modified to detect the OFED version and pass the
  version to the bnxt_re makefile as the OFED_VERSION environment
  variable. This is necessary to automatically adapt to the version of
  OFED installed in the build environment.

- The %post and %postun scriptlets are modified to make them similar to
  the ones used by i40e, which passes the target kernel's version to
  depmod. The original %post only had 'depmod -a', which can be
  problematic when installing the package for a kernel that is not
  currently running.

- Kernel module signing was added to %install. This shell snippet was
  adapted from i40e.

- Finally, retpolines were enabled by default, given that StarlingX
  enables retpolines in its kernel packages as well.

In terms of source code modifications, the following changes were made
to bnxt_en and bnxt_re:

- The bnxt_en and bnxt_re kernel modules are patched to make use of the
  irq_update_affinity_hint function instead of the irq_set_affinity_hint
  function, so that the irqaffinity= kernel command line option takes
  effect for the affinities of the IRQs managed by the two drivers.

  However, it should be noted the changes in bnxt_en are not truly
  necessary, because pre-existing conditional compilation directives
  leave out the modified code when targeting kernels >= v4.15 or kernels
  that have the "IRQ matrix allocator" feature. The changes are kept for
  completeness, nevertheless.

  For further details, please see the patch description.

- A compilation failure caused by the definition of a placeholder macro
  aimed to stand in for the bpf_warn_invalid_xdp_action function in the
  kernel tree is resolved by removing the macro definition.

Testing:
- An ISO image was successfully built with a monolithic incremental
  build procedure.

- Using a virtual machine in All-in-One simplex configuration, the
  bnxt_en and bnxt_re kernel modules were confirmed to be successfully
  loaded manually using, for example, "modprobe -v -a bnxt_en bnxt_re".
  Runtime tests for these drivers were unfortunately very limited,
  because the necessary hardware has not yet been acquired. We intend to
  resolve integration issues when hardware is acquired.

Story: 2009915
Task: 44762

Depends-On: https://review.opendev.org/c/starlingx/kernel/+/833463
Depends-On: https://review.opendev.org/c/starlingx/tools/+/833462

Change-Id: Ieb4147335410160a781382b0b6b01e2015468a3b
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
2022-03-23 10:21:10 -04:00

118 lines
4.5 KiB
Diff
Raw Blame History

From b4de4f0d87331a3df820ca78a7196b41a5ee74d5 Mon Sep 17 00:00:00 2001
From: "M. Vefa Bicakci" <vefa.bicakci@windriver.com>
Date: Fri, 4 Feb 2022 18:27:53 -0500
Subject: [PATCH] bnxt_re/Makefile: Adapt to mlnx-ofa_kernel for StarlingX
This patch adapts the bnxt_re kernel module's Makefile to
mlnx-ofa_kernel for StarlingX.
Here is a summary of the changes:
- $(shell uname -r) is replaced with $(KVER) to ensure that the target
kernel version is selected instead of the build host's kernel version.
- To target Mellanox's OFED distribution instead of compat-rdma, the
references to "/usr/src/compat-rdma" are replaced with
"/usr/src/ofa_kernel".
- Mellanox's OFED distribution has version numbers in the form 5.x, so
the Makefile version match patterns are adjusted.
- "AUTOCONF_H += -include $(LINUXSRC)/include/linux/kconfig.h" line is
added avoid the following kernel module build errors indicating that
IS_ENABLED is unintendedly undefined:
include/asm-generic/bitops/instrumented-non-atomic.h: In function \
'__instrument_read_write_bitop':
include/asm-generic/bitops/instrumented-non-atomic.h:63:6: error: \
implicit declaration of function 'IS_ENABLED' \
[-Werror=implicit-function-declaration]
if (IS_ENABLED(CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC)) {
^~~~~~~~~~
- The line that reads
"KBUILD_EXTRA_SYMBOLS := $(BNXT_PEER_MEM_INC)/Module.symvers"
is guarded to avoid the following build warning, which is encountered
as BNXT_PEER_MEM_INC is not defined by default:
WARNING: Symbol version dump "/Module.symvers" is missing.
- OFA_BUILD_PATH and OFA_KERNEL_PATH are adjusted to refer to the kernel
source code belonging to the target kernel version in
"/usr/src/kernels/", because StarlingX's build environment does not
populate /lib/modules/<kver>/ with symbolic links for kernel source
and build trees.
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
---
bnxt_re-220.0.12.0/Makefile | 30 ++++++++++++++++++------------
1 file changed, 18 insertions(+), 12 deletions(-)
diff --git a/bnxt_re-220.0.12.0/Makefile b/bnxt_re-220.0.12.0/Makefile
index 532fb8025465..d13a61379acb 100644
--- a/bnxt_re-220.0.12.0/Makefile
+++ b/bnxt_re-220.0.12.0/Makefile
@@ -32,7 +32,7 @@ ifeq ($(shell ls /lib/modules/$(KVER)/build > /dev/null 2>&1 && echo build),)
# SuSE source RPMs
_KVER=$(shell echo $(KVER) | cut -d "-" -f1,2)
_KFLA=$(shell echo $(KVER) | cut -d "-" -f3)
- _ARCH=$(shell file -b /lib/modules/$(shell uname -r)/build | cut -d "/" -f5)
+ _ARCH=$(shell file -b /lib/modules/$(KVER)/build | cut -d "/" -f5)
ifeq ($(_ARCH),)
_ARCH=$(__ARCH)
endif
@@ -90,34 +90,40 @@ ifeq ($(OFED_VERSION), )
endif
#find OFED version and compat-includes
-ofed_major=$(filter OFED-3.% OFED-4.%, $(OFED_VERSION))
+ofed_major=$(filter OFED-3.% OFED-4.% 5.%, $(OFED_VERSION))
ifneq ($(ofed_major), )
-exists=$(shell if [ -e /usr/src/compat-rdma$(OFED_VERSION) ];\
+exists=$(shell if [ -e /usr/src/ofa_kernel-$(OFED_VERSION) ];\
then echo y; fi)
ifeq ($(exists), )
-$(shell ln -s /usr/src/compat-rdma\
- /usr/src/compat-rdma$(OFED_VERSION))
+$(shell ln -s /usr/src/ofa_kernel\
+ /usr/src/ofa_kernel-$(OFED_VERSION))
endif
-OFA_BUILD_PATH=/usr/src/compat-rdma$(OFED_VERSION)
-OFA_KERNEL_PATH=/usr/src/compat-rdma$(OFED_VERSION)
+OFA_BUILD_PATH=/usr/src/ofa_kernel/default
+OFA_KERNEL_PATH=/usr/src/ofa_kernel-$(OFED_VERSION)/source
EXTRA_CFLAGS += -DOFED_3_x
-ofed_4_17_x=$(filter OFED-4.17%, $(ofed_major))
+ofed_4_17_x=$(filter OFED-4.17% 5.%, $(ofed_major))
ifneq ($(ofed_4_17_x), )
EXTRA_CFLAGS += -D__OFED_BUILD__
endif
EXTRA_CFLAGS += -include $(OFA_KERNEL_PATH)/include/linux/compat-2.6.h
AUTOCONF_H = -include $(shell /bin/ls -1 $(LINUX)/include/*/autoconf.h 2> /dev/null | head -1)
+AUTOCONF_H += -include $(LINUXSRC)/include/linux/kconfig.h
endif #end non 3.x OFED
+
+ifneq ($(BNXT_PEER_MEM_INC),)
KBUILD_EXTRA_SYMBOLS := $(BNXT_PEER_MEM_INC)/Module.symvers
+else
+KBUILD_EXTRA_SYMBOLS :=
+endif
ifeq (OFED-NATIVE, $(findstring OFED-NATIVE, $(OFED_VERSION)))
ifeq ($(KERNELRELEASE),)
-OFA_BUILD_PATH =/lib/modules/`uname -r`/build/
-OFA_KERNEL_PATH=/lib/modules/`uname -r`/source/
+OFA_BUILD_PATH=/usr/src/kernels/$(KVER)/
+OFA_KERNEL_PATH=/usr/src/kernels/$(KVER)/
else
-OFA_BUILD_PATH =/lib/modules/$(KERNELRELEASE)/build/
-OFA_KERNEL_PATH=/lib/modules/$(KERNELRELEASE)/source/
+OFA_BUILD_PATH=/usr/src/kernels/$(KERNELRELEASE)/
+OFA_KERNEL_PATH=/usr/src/kernels/$(KERNELRELEASE)/
endif
else
# Add OFED symbols only if external OFED is used
--
2.29.2
View Git Blame Copy Permalink