stx-10.0: Uprevision kernel to v6.6

We propose aligning the StarlingX kernel to the latest long-term support
(LTS) release, which is v6.6 as of this writing, to let StarlingX take
advantage of upstream improvements made to the Linux kernel since the
release of StarlingX's current kernel based on v5.10.

Note that this proposal is based on an earlier version published at:

Story: 2011000
Task: 49370
Change-Id: I460c92c5e2913ccf76d3e7526f8e3bb0cf64cb41
Co-developed-by: Li Zhou <>
Signed-off-by: Li Zhou <>
Co-developed-by: M. Vefa Bicakci <>
Signed-off-by: M. Vefa Bicakci <>
This commit is contained in:
M. Vefa Bicakci 2024-01-10 14:28:20 -05:00
parent a6a1b0179d
commit 31bf76b1f8
2 changed files with 378 additions and 8 deletions

View File

@ -1,8 +0,0 @@
.. placeholder:
This file is a placeholder and should be deleted when the first spec is moved
to this directory.

View File

@ -0,0 +1,378 @@
This work is licensed under a Creative Commons Attribution 3.0 Unported
StarlingX: Uprevision the kernel to v6.6
Storyboard: `2011000`_
We propose aligning the StarlingX kernel to the latest long-term support (LTS)
release, which is v6.6 as of this writing, to let StarlingX take advantage of
upstream improvements made to the Linux kernel since the release of StarlingX's
current kernel based on v5.10.
Problem description
StarlingX currently ships with kernel version 5.10, while the latest LTS
release available as of this writing is version 6.6 according to `'s
Active Kernel Releases page`_. We propose aligning the StarlingX kernel to
v6.6, as made available by the Yocto Project's `linux-yocto repository`_.
The intent of this uprevision is to let StarlingX take advantage of upstream
improvements to the kernel that have been made since the release of the current
StarlingX kernel, with the stability and extended maintenance of an LTS
Updated device drivers inherited from the v6.6 kernel will let StarlingX:
* improve support for hardware platforms currently supported by StarlingX, such
as Intel Sapphire Rapids, with respect to features such as EDAC (Error
Detection and Correction) and hardware performance monitor unit (PMU)
counters (for perf),
* support newer platforms and peripheral hardware, examples for the latter of
which include storage controllers managed by device drivers like mpt3sas,
mpi3mr and megaraid_sas, and
* take advantage of newer in-tree device drivers (such as ice, i40e and iavf)
which introduce features that have so far been provided by their out-of-tree
counterparts in StarlingX.
In addition, the v6.6 kernel also offers newer features that are expected to
introduce performance improvements, such as the Earliest Eligible Virtual
Deadline First (`EEVDF`_) scheduler and `memory folios`_.
The v6.6 kernel also offers bug fixes that may not have been backported to the
v5.10 kernel, which is expected to increase stability.
Use Cases
With the move to the v6.6 kernel, StarlingX end users and deployers will be
able to use StarlingX on newer platforms that require more recent device
drivers and (potentially) recent core kernel features.
We identify the following potential impacts to StarlingX's stakeholders with
this kernel uprevision:
* End users: The kernel uprevision is expected to be transparent to the current
use cases of end users, thanks to the kernel's stable user-space ABI.
* Deployers: There may be slight impacts to StarlingX deployers, due to
potential changes to the kernel's configuration interfaces such as the kernel
command line, ``sysctl`` options, and the ``procfs`` and ``sysfs`` file
systems, in case deployers are customizing the kernel's run-time settings.
* End users and deployers: Given that the kernel development community does not
maintain a `stable in-tree kernel API`_, the kernel uprevision will likely
require the recompilation and redeployment of kernel modules that may be in
use by end users and/or deployers for their downstream projects.
* Developers: Higher-level components in StarlingX are not expected to be
impacted, but as with the impacts affecting Deployers, the proposed kernel
uprevision may require modifications to the StarlingX installer, Ansible
playbooks and other system configuration tooling.
* Developers: StarlingX kernel maintainers' development workflows are expected
to be improved with the proposed kernel uprevision by migrating the kernel
packages' changelog files and the kernel configuration from patch files to
regular files in the StarlingX kernel packaging repository
Proposed change
This specification proposes:
* To uprevision the StarlingX kernel to Linux kernel LTS release 6.6 for the
StarlingX 10.0 release.
* To continue the use of the Yocto Project's `linux-yocto repository`_ as the
upstream source code repository and to base the StarlingX kernel's source
code on the `v6.6/standard/base branch`_ for the standard kernel and the
`v6.6/standard/preempt-rt/base branch`_ for the low-latency kernel.
* To utilize the ``debian`` directory in the `bullseye-backports`_ branch of
the Debian kernel packaging repository, which, despite targeting kernel v6.1,
was observed to build the v6.6 kernel with changes.
* To streamline the maintenance of the StarlingX kernel by simplifying changes
to the kernel configuration and changelog files.
As of this writing, the StarlingX kernel's changelog and configuration are
maintained as patch files that make the necessary changes to the changelog
and kernel configuration files acquired from `Debian's kernel packaging
repository`_. These patch files are stored in in the
``kernel/kernel-{rt,std}/debian/deb_patches`` directories of the
`starlingx/kernel`_ repository. (`Example patch file`_)
With this specification, we propose making the StarlingX kernel configuration
and changelog standalone files in the
``kernel/kernel-{rt,std}/debian/source`` directories of the
`starlingx/kernel`_ repository. Such a change avoids unnecessary churn in the
``deb_patches`` directory and makes it easier to update the StarlingX
kernel's changelog and configuration.
We have thought of a number of alternatives to this proposal:
1. Do not uprevision the StarlingX kernel (i.e., stay with the v5.10 kernel)
Given that the v5.10 kernel and the v6.6 kernel currently have the same
planned end-of-life (EOL) time frame (December, 2026) according to
`'s Active Kernel Releases page`_ as of this writing, not
uprevisioning the StarlingX kernel *could* be considered as an alternative.
The concern with this alternative is that the v5.10 kernel has started to
show its age with respect to hardware support, which will hold back the
StarlingX community from being able to deploy StarlingX on newer hardware
platforms or to make use of newer peripheral hardware. The StarlingX
community could potentially continue to backport hardware support from newer
kernel versions to the v5.10 kernel, but the cost for such maintenance would
be prohibitive.
In addition, with the increasing divergence of the mainline kernel from the
v5.10 kernel, it is possible that fixes for bugs affecting the StarlingX
community will not be backported to the v5.10 kernel by upstream
2. Opt for another LTS kernel releases (such as v5.15 or v6.1)
Other LTS kernel releases that could be considered for this proposal are
v5.15 and v6.1, both of which have the same EOL time frame (December, 2026)
as the v6.6 kernel release chosen for this proposal. Given the older code
bases that the v5.15 and v6.1 releases have, we do not see an advantage for
using them, other than to avoid an undesirable aspect of the v6.6 kernel,
which we are not aware of.
Furthermore, the v6.6 kernel's more recent code base will allow StarlingX to
reduce its dependency on out-of-tree device drivers, by making use of their
in-tree counterparts, which will be covered by another specification
3. Opt for an upstream other than the `linux-yocto repository`_
The final aspect that could be considered is the use of the ``linux-yocto``
project as the upstream kernel source code repository. It is possible, for
example, to use the `linux-stable team's releases`_ for the standard kernel
and the `linux-rt team's repositories`_ for the PREEMPT_RT kernel.
For this topic, we would like to refer the reader to the relevant subsection
of the specification proposal for the 5.10 kernel uprevision approved for
the StarlingX 6.0 release:
Data model impact
REST API impact
Security impact
Given the nature of a kernel uprevision, we would like to note that there could
be a difficult-to-qualify impact on security, as a newer kernel code base could
inadvertently include recently introduced bugs with security implications.
We would also like to note that staying with older kernel releases is not a
viable approach to avoid negative impacts on security, because security-related
bug-fixes may not always be backported to older kernel releases due to a
variety of reasons.
Updating to the most recent LTS kernel releases is generally recommended by the
kernel development community when security is considered:
Other end user impact
Performance Impact
Given the scope of a kernel uprevision activity, it is difficult to predict
whether there will be a significant negative performance impact on use cases.
We would also like to note that the v6.6 kernel incorporates a new scheduler
(EEVDF) and that there is a *possibility* that the new scheduler will impact
certain workloads.
To guard against performance regressions, there will be work items for carrying
out performance tests with ``cyclictest`` and a Kubernetes-based network
performance benchmark, and significant degradations in performance are expected
to be resolved during the implementation of this proposal.
Other deployer impact
Developer impact
(Please see the `Use Cases`_ section.)
Upgrade impact
Primary assignee:
* Li Zhou (lzhou2)
Other contributors:
* Jiping Ma (jma11)
Repos Impacted
The following is a preliminary list:
* starlingx/kernel
* starlingx/tools
Work Items
The following work items are expected to be carried out, with the understanding
that the story board will be updated as more work items are found to be
* Uprevision the StarlingX standard kernel (kernel-std) to v6.6.y.
* Uprevision the StarlingX PREEMPT_RT kernel (kernel-rt) to v6.6.y.
* Adapt/upgrade the out-of-tree kernel drivers utilized by StarlingX to work
with the v6.6 kernel. (Parts of this work item may be eliminated in case a
separate specification proposal on migration from out-of-tree to in-tree
device drivers is approved.)
* Performance testing will be required to guard against performance
* Regression testing will be required for some of the StarlingX kernel-related
bugs that had been fixed in the past, due to the contextual changes that have
been encountered while forward-porting the bug-fix patches from the v5.10
kernel to the v6.6 kernel.
* Finally, basic sanity tests will be required for the out-of-tree drivers that
need adaptations for the v6.6 kernel.
Interactions with other proposals
We would also like to note that the StarlingX kernel's packaging will need
further updates due to a forthcoming StarlingX specification proposal to update
the StarlingX base distribution from Debian 11 (``bullseye``) to Debian 12
(``bookworm``). With the approval of the latter proposal, the kernel packaging
updates are expected to involve:
* Changing the Debian kernel packaging repository base branch from
`bullseye-backports`_ to `bookworm-backports`_, and
* Migration from Debian 11's gcc-10-based build toolchain to Debian 12's
gcc-12-based build toolchain.
Finally, as a side effect of this kernel uprevision, it will additionally be
possible to migrate StarlingX to in-tree versions of certain device drivers,
which will be the subject of a separate StarlingX specification proposal.
Interactions with past work
This proposal builds on past work on removing the hard-coded ABI name (i.e.,
"5.10.0-6") from the kernel package names and from numerous locations in the
packaging, which allows the StarlingX community to avoid the need to change the
ABI name for every major kernel version change. The commits in question can be
seen at:
Given the nature of this uprevisioning activity, sanity and regression tests
will be carried out, which are expected to include:
* Verification of basic kernel features using test suites such as the LTP
(Linux Test Project) test suite.
* Deployment of StarlingX on multiple types of nodes (e.g., All-in-One
Simplex/Duplex, standard, distributed cloud/sub-cloud) on a variety of
hardware platforms, using the low-latency and the standard installation
* Performance tests (such as cyclictest and Kubernetes network performance
test) to ensure that the new kernel's performance is acceptable.
* Regression tests for bugs that had been fixed by patching the StarlingX
kernel, if the patches in question are affected by the changes between the
v5.10 and v6.6 kernels.
* Basic sanity tests for the out-of-tree drivers that needed adaptations for
the v6.6 kernel.
Documentation Impact
At a minimum, the documentation will need to be updated to refer to the new
kernel version.
Documentation pages with references to the kernel's interfaces (such as command
line arguments, ``sysctl`` options, ``dmesg`` output), if any, will need to be
updated for the new kernel as well.
.. list-table:: Revisions
:header-rows: 1
* - Release Name
- Description
* - stx-10.0
- Introduced
.. Links
.. _2011000:!/story/2011000
..'s Active Kernel Releases page:
.. _linux-yocto repository:
.. _stable in-tree kernel API:
.. _starlingx/kernel:
.. _EEVDF:
.. _memory folios:
.. _v6.6/standard/base branch:
.. _v6.6/standard/preempt-rt/base branch:
.. _Debian's kernel packaging repository:
.. _bullseye-backports:
.. _bookworm-backports:
.. _Example patch file:
.. _linux-stable team's releases:
.. _linux-rt team's repositories: