Relocated some packages to repo 'utilities'
List of relocated subdirectories: pm-qos-mgr worker-utils Story: 2006166 Task: 35687 Depends-On: I665dc7fabbfffc798ad57843eb74dca16e7647a3 Change-Id: I63df9a59a8a409ab4b700b76fd4d39acb6ab0ed7 Signed-off-by: Scott Little <scott.little@windriver.com> Depends-On: Ie6fc7b2a185168424cb6158e817b6e240af89d5e
This commit is contained in:
parent
fed963b207
commit
23a41191c1
@ -1,2 +0,0 @@
|
|||||||
SRC_DIR="src"
|
|
||||||
TIS_PATCH_VER=1
|
|
@ -1,61 +0,0 @@
|
|||||||
%define debug_package %{nil}
|
|
||||||
%global pypi_name pm_qos_mgr
|
|
||||||
|
|
||||||
Name: pm-qos-mgr
|
|
||||||
Version: 1.0
|
|
||||||
Release: %{tis_patch_ver}%{?_tis_dist}
|
|
||||||
Summary: PM QoS CPU wakeup latency manager for kubelet cpu-manager
|
|
||||||
License: Apache-2.0
|
|
||||||
Group: base
|
|
||||||
URL: unknown
|
|
||||||
Source0: %{name}-%{version}.tar.gz
|
|
||||||
|
|
||||||
BuildRequires: git
|
|
||||||
BuildRequires: python-pbr >= 2.0.0
|
|
||||||
BuildRequires: python-setuptools
|
|
||||||
BuildRequires: python2-pip
|
|
||||||
BuildRequires: systemd-devel
|
|
||||||
|
|
||||||
Requires: python-pbr >= 2.0.0
|
|
||||||
Requires: python-inotify
|
|
||||||
Requires: systemd
|
|
||||||
|
|
||||||
%description
|
|
||||||
A daemon that monitors kubelet cpu-manager static cpu assignments
|
|
||||||
and modifies PM QoS CPU wakeup latency.
|
|
||||||
|
|
||||||
%define pythonroot %{_libdir}/python2.7/site-packages
|
|
||||||
|
|
||||||
%prep
|
|
||||||
%autosetup -n %{name}-%{version} -S git
|
|
||||||
|
|
||||||
# Remove bundled egg-info
|
|
||||||
rm -rf *.egg-info
|
|
||||||
|
|
||||||
%build
|
|
||||||
export PBR_VERSION=%{version}
|
|
||||||
%{__python} setup.py build
|
|
||||||
|
|
||||||
%install
|
|
||||||
export PBR_VERSION=%{version}
|
|
||||||
%{__python} setup.py install --root=%{buildroot} \
|
|
||||||
--install-lib=%{pythonroot} \
|
|
||||||
--prefix=%{_prefix} \
|
|
||||||
--install-data=%{_datadir} \
|
|
||||||
--single-version-externally-managed
|
|
||||||
|
|
||||||
install -p -D -m 664 pm-qos-mgr.service %{buildroot}%{_unitdir}/pm-qos-mgr.service
|
|
||||||
|
|
||||||
%post
|
|
||||||
systemctl enable pm-qos-mgr.service
|
|
||||||
|
|
||||||
%clean
|
|
||||||
rm -rf $RPM_BUILD_ROOT
|
|
||||||
|
|
||||||
%files
|
|
||||||
%defattr(-,root,root,-)
|
|
||||||
%doc LICENSE
|
|
||||||
%{_bindir}/*
|
|
||||||
%{pythonroot}/%{pypi_name}/*
|
|
||||||
%{pythonroot}/%{pypi_name}-%{version}*.egg-info
|
|
||||||
%{_unitdir}/*
|
|
@ -1,202 +0,0 @@
|
|||||||
|
|
||||||
Apache License
|
|
||||||
Version 2.0, January 2004
|
|
||||||
http://www.apache.org/licenses/
|
|
||||||
|
|
||||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
||||||
|
|
||||||
1. Definitions.
|
|
||||||
|
|
||||||
"License" shall mean the terms and conditions for use, reproduction,
|
|
||||||
and distribution as defined by Sections 1 through 9 of this document.
|
|
||||||
|
|
||||||
"Licensor" shall mean the copyright owner or entity authorized by
|
|
||||||
the copyright owner that is granting the License.
|
|
||||||
|
|
||||||
"Legal Entity" shall mean the union of the acting entity and all
|
|
||||||
other entities that control, are controlled by, or are under common
|
|
||||||
control with that entity. For the purposes of this definition,
|
|
||||||
"control" means (i) the power, direct or indirect, to cause the
|
|
||||||
direction or management of such entity, whether by contract or
|
|
||||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
||||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
||||||
|
|
||||||
"You" (or "Your") shall mean an individual or Legal Entity
|
|
||||||
exercising permissions granted by this License.
|
|
||||||
|
|
||||||
"Source" form shall mean the preferred form for making modifications,
|
|
||||||
including but not limited to software source code, documentation
|
|
||||||
source, and configuration files.
|
|
||||||
|
|
||||||
"Object" form shall mean any form resulting from mechanical
|
|
||||||
transformation or translation of a Source form, including but
|
|
||||||
not limited to compiled object code, generated documentation,
|
|
||||||
and conversions to other media types.
|
|
||||||
|
|
||||||
"Work" shall mean the work of authorship, whether in Source or
|
|
||||||
Object form, made available under the License, as indicated by a
|
|
||||||
copyright notice that is included in or attached to the work
|
|
||||||
(an example is provided in the Appendix below).
|
|
||||||
|
|
||||||
"Derivative Works" shall mean any work, whether in Source or Object
|
|
||||||
form, that is based on (or derived from) the Work and for which the
|
|
||||||
editorial revisions, annotations, elaborations, or other modifications
|
|
||||||
represent, as a whole, an original work of authorship. For the purposes
|
|
||||||
of this License, Derivative Works shall not include works that remain
|
|
||||||
separable from, or merely link (or bind by name) to the interfaces of,
|
|
||||||
the Work and Derivative Works thereof.
|
|
||||||
|
|
||||||
"Contribution" shall mean any work of authorship, including
|
|
||||||
the original version of the Work and any modifications or additions
|
|
||||||
to that Work or Derivative Works thereof, that is intentionally
|
|
||||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
||||||
or by an individual or Legal Entity authorized to submit on behalf of
|
|
||||||
the copyright owner. For the purposes of this definition, "submitted"
|
|
||||||
means any form of electronic, verbal, or written communication sent
|
|
||||||
to the Licensor or its representatives, including but not limited to
|
|
||||||
communication on electronic mailing lists, source code control systems,
|
|
||||||
and issue tracking systems that are managed by, or on behalf of, the
|
|
||||||
Licensor for the purpose of discussing and improving the Work, but
|
|
||||||
excluding communication that is conspicuously marked or otherwise
|
|
||||||
designated in writing by the copyright owner as "Not a Contribution."
|
|
||||||
|
|
||||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
||||||
on behalf of whom a Contribution has been received by Licensor and
|
|
||||||
subsequently incorporated within the Work.
|
|
||||||
|
|
||||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
||||||
this License, each Contributor hereby grants to You a perpetual,
|
|
||||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
||||||
copyright license to reproduce, prepare Derivative Works of,
|
|
||||||
publicly display, publicly perform, sublicense, and distribute the
|
|
||||||
Work and such Derivative Works in Source or Object form.
|
|
||||||
|
|
||||||
3. Grant of Patent License. Subject to the terms and conditions of
|
|
||||||
this License, each Contributor hereby grants to You a perpetual,
|
|
||||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
||||||
(except as stated in this section) patent license to make, have made,
|
|
||||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
||||||
where such license applies only to those patent claims licensable
|
|
||||||
by such Contributor that are necessarily infringed by their
|
|
||||||
Contribution(s) alone or by combination of their Contribution(s)
|
|
||||||
with the Work to which such Contribution(s) was submitted. If You
|
|
||||||
institute patent litigation against any entity (including a
|
|
||||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
||||||
or a Contribution incorporated within the Work constitutes direct
|
|
||||||
or contributory patent infringement, then any patent licenses
|
|
||||||
granted to You under this License for that Work shall terminate
|
|
||||||
as of the date such litigation is filed.
|
|
||||||
|
|
||||||
4. Redistribution. You may reproduce and distribute copies of the
|
|
||||||
Work or Derivative Works thereof in any medium, with or without
|
|
||||||
modifications, and in Source or Object form, provided that You
|
|
||||||
meet the following conditions:
|
|
||||||
|
|
||||||
(a) You must give any other recipients of the Work or
|
|
||||||
Derivative Works a copy of this License; and
|
|
||||||
|
|
||||||
(b) You must cause any modified files to carry prominent notices
|
|
||||||
stating that You changed the files; and
|
|
||||||
|
|
||||||
(c) You must retain, in the Source form of any Derivative Works
|
|
||||||
that You distribute, all copyright, patent, trademark, and
|
|
||||||
attribution notices from the Source form of the Work,
|
|
||||||
excluding those notices that do not pertain to any part of
|
|
||||||
the Derivative Works; and
|
|
||||||
|
|
||||||
(d) If the Work includes a "NOTICE" text file as part of its
|
|
||||||
distribution, then any Derivative Works that You distribute must
|
|
||||||
include a readable copy of the attribution notices contained
|
|
||||||
within such NOTICE file, excluding those notices that do not
|
|
||||||
pertain to any part of the Derivative Works, in at least one
|
|
||||||
of the following places: within a NOTICE text file distributed
|
|
||||||
as part of the Derivative Works; within the Source form or
|
|
||||||
documentation, if provided along with the Derivative Works; or,
|
|
||||||
within a display generated by the Derivative Works, if and
|
|
||||||
wherever such third-party notices normally appear. The contents
|
|
||||||
of the NOTICE file are for informational purposes only and
|
|
||||||
do not modify the License. You may add Your own attribution
|
|
||||||
notices within Derivative Works that You distribute, alongside
|
|
||||||
or as an addendum to the NOTICE text from the Work, provided
|
|
||||||
that such additional attribution notices cannot be construed
|
|
||||||
as modifying the License.
|
|
||||||
|
|
||||||
You may add Your own copyright statement to Your modifications and
|
|
||||||
may provide additional or different license terms and conditions
|
|
||||||
for use, reproduction, or distribution of Your modifications, or
|
|
||||||
for any such Derivative Works as a whole, provided Your use,
|
|
||||||
reproduction, and distribution of the Work otherwise complies with
|
|
||||||
the conditions stated in this License.
|
|
||||||
|
|
||||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
||||||
any Contribution intentionally submitted for inclusion in the Work
|
|
||||||
by You to the Licensor shall be under the terms and conditions of
|
|
||||||
this License, without any additional terms or conditions.
|
|
||||||
Notwithstanding the above, nothing herein shall supersede or modify
|
|
||||||
the terms of any separate license agreement you may have executed
|
|
||||||
with Licensor regarding such Contributions.
|
|
||||||
|
|
||||||
6. Trademarks. This License does not grant permission to use the trade
|
|
||||||
names, trademarks, service marks, or product names of the Licensor,
|
|
||||||
except as required for reasonable and customary use in describing the
|
|
||||||
origin of the Work and reproducing the content of the NOTICE file.
|
|
||||||
|
|
||||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
||||||
agreed to in writing, Licensor provides the Work (and each
|
|
||||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
||||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
||||||
implied, including, without limitation, any warranties or conditions
|
|
||||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
||||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
||||||
appropriateness of using or redistributing the Work and assume any
|
|
||||||
risks associated with Your exercise of permissions under this License.
|
|
||||||
|
|
||||||
8. Limitation of Liability. In no event and under no legal theory,
|
|
||||||
whether in tort (including negligence), contract, or otherwise,
|
|
||||||
unless required by applicable law (such as deliberate and grossly
|
|
||||||
negligent acts) or agreed to in writing, shall any Contributor be
|
|
||||||
liable to You for damages, including any direct, indirect, special,
|
|
||||||
incidental, or consequential damages of any character arising as a
|
|
||||||
result of this License or out of the use or inability to use the
|
|
||||||
Work (including but not limited to damages for loss of goodwill,
|
|
||||||
work stoppage, computer failure or malfunction, or any and all
|
|
||||||
other commercial damages or losses), even if such Contributor
|
|
||||||
has been advised of the possibility of such damages.
|
|
||||||
|
|
||||||
9. Accepting Warranty or Additional Liability. While redistributing
|
|
||||||
the Work or Derivative Works thereof, You may choose to offer,
|
|
||||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
||||||
or other liability obligations and/or rights consistent with this
|
|
||||||
License. However, in accepting such obligations, You may act only
|
|
||||||
on Your own behalf and on Your sole responsibility, not on behalf
|
|
||||||
of any other Contributor, and only if You agree to indemnify,
|
|
||||||
defend, and hold each Contributor harmless for any liability
|
|
||||||
incurred by, or claims asserted against, such Contributor by reason
|
|
||||||
of your accepting any such warranty or additional liability.
|
|
||||||
|
|
||||||
END OF TERMS AND CONDITIONS
|
|
||||||
|
|
||||||
APPENDIX: How to apply the Apache License to your work.
|
|
||||||
|
|
||||||
To apply the Apache License to your work, attach the following
|
|
||||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
||||||
replaced with your own identifying information. (Don't include
|
|
||||||
the brackets!) The text should be enclosed in the appropriate
|
|
||||||
comment syntax for the file format. We also recommend that a
|
|
||||||
file or class name and description of purpose be included on the
|
|
||||||
same "printed page" as the copyright notice for easier
|
|
||||||
identification within third-party archives.
|
|
||||||
|
|
||||||
Copyright [yyyy] [name of copyright owner]
|
|
||||||
|
|
||||||
Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
you may not use this file except in compliance with the License.
|
|
||||||
You may obtain a copy of the License at
|
|
||||||
|
|
||||||
http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
|
|
||||||
Unless required by applicable law or agreed to in writing, software
|
|
||||||
distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
See the License for the specific language governing permissions and
|
|
||||||
limitations under the License.
|
|
@ -1,11 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=PM QoS CPU wakeup latency manager for kubelet cpu-manager
|
|
||||||
After=syslog.target network-online.target remote-fs.target sw-patch.service
|
|
||||||
Before=kubelet.service
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=simple
|
|
||||||
ExecStart=/usr/bin/pm-qos-mgr
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
@ -1,165 +0,0 @@
|
|||||||
#
|
|
||||||
# Copyright (c) 2019 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
# Purpose:
|
|
||||||
# This manager watches for changes in /var/lib/kubelet/cpu_manager_state file.
|
|
||||||
# This sets appropriate PM QoS resume latency constraints for CPUs
|
|
||||||
# when kubelet cpu-manager is configured with 'static' policy.
|
|
||||||
#
|
|
||||||
# This parses the cpu_manager_state file, deduces the cpu-manager policy,
|
|
||||||
# and CPU lists of Guaranteed pods versus the remaining Default CPUs.
|
|
||||||
# Guaranteed pods with exclusive CPUs get "low" cpu wakeup latency policy.
|
|
||||||
# Default CPUs get "high" cpu wakeup latency policy.
|
|
||||||
|
|
||||||
import itertools as it
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
import logging.handlers
|
|
||||||
import os
|
|
||||||
import pyinotify
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
|
|
||||||
# Global variables
|
|
||||||
statefile = '/var/lib/kubelet/cpu_manager_state'
|
|
||||||
pm_script = '/usr/bin/set-cpu-wakeup-latency.sh'
|
|
||||||
|
|
||||||
LOG = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
def configure_logging(logger, level=logging.DEBUG):
|
|
||||||
""" Configure logger streams and format. """
|
|
||||||
LOG.setLevel(level)
|
|
||||||
|
|
||||||
syslog_facility = logging.handlers.SysLogHandler.LOG_DAEMON
|
|
||||||
ch = logging.handlers.SysLogHandler(address='/dev/log',
|
|
||||||
facility=syslog_facility)
|
|
||||||
ch.setLevel(level)
|
|
||||||
formatter = logging.Formatter('%(module)s[%(process)d]: %(message)s')
|
|
||||||
ch.setFormatter(formatter)
|
|
||||||
LOG.addHandler(ch)
|
|
||||||
|
|
||||||
|
|
||||||
def format_range_set(items):
|
|
||||||
""" Generate pretty-printed value of ranges, such as 3-6,12-17. """
|
|
||||||
ranges = []
|
|
||||||
for k, iterable in it.groupby(enumerate(sorted(items)),
|
|
||||||
lambda x: x[1] - x[0]):
|
|
||||||
rng = list(iterable)
|
|
||||||
if len(rng) == 1:
|
|
||||||
s = str(rng[0][1])
|
|
||||||
else:
|
|
||||||
s = "%s-%s" % (rng[0][1], rng[-1][1])
|
|
||||||
ranges.append(s)
|
|
||||||
return ','.join(ranges)
|
|
||||||
|
|
||||||
|
|
||||||
def range_to_list(csv_range=None):
|
|
||||||
""" Convert a string of comma separate ranges into an expanded list
|
|
||||||
of integers. e.g., '1-3,8-9,15' is converted to [1,2,3,8,9,15].
|
|
||||||
"""
|
|
||||||
if not csv_range:
|
|
||||||
return []
|
|
||||||
ranges = [(lambda L: range(L[0], L[-1] + 1))(map(int, r.split('-')))
|
|
||||||
for r in csv_range.split(',')]
|
|
||||||
return [y for x in ranges for y in x]
|
|
||||||
|
|
||||||
|
|
||||||
class ProcessTransientFile(pyinotify.ProcessEvent):
|
|
||||||
def __init__(self, *args, **kw):
|
|
||||||
self.policy = None
|
|
||||||
self.cpusets = {'default': set(),
|
|
||||||
'guaranteed': set()}
|
|
||||||
self.update_pm_qos_cpu_latency()
|
|
||||||
|
|
||||||
def update_pm_qos_cpu_latency(self, event=None):
|
|
||||||
if self.policy is not None and self.policy != 'static':
|
|
||||||
return
|
|
||||||
if event is not None:
|
|
||||||
LOG.debug('%s, %s', event.pathname, event.maskname)
|
|
||||||
|
|
||||||
# Read JSON formatted state file dictionary
|
|
||||||
state = {}
|
|
||||||
try:
|
|
||||||
with open(statefile, 'r') as f:
|
|
||||||
state = json.load(f)
|
|
||||||
except Exception as e:
|
|
||||||
LOG.error('Could not load: %s, error: %s.', statefile, e)
|
|
||||||
return
|
|
||||||
|
|
||||||
self.policy = str(state['policyName'])
|
|
||||||
if self.policy != 'static':
|
|
||||||
return
|
|
||||||
|
|
||||||
# Determine default cpuset
|
|
||||||
if 'defaultCpuSet' not in state:
|
|
||||||
LOG.error('Missing defaultCpuSet.', statefile)
|
|
||||||
return
|
|
||||||
default_cpuranges = str(state['defaultCpuSet'])
|
|
||||||
default_cpuset = set(range_to_list(csv_range=default_cpuranges))
|
|
||||||
|
|
||||||
# Determine guaranteed cpuset
|
|
||||||
guaranteed_cpuset = set()
|
|
||||||
if 'entries' in state:
|
|
||||||
for pod, cpus in state['entries'].items():
|
|
||||||
cpulist = range_to_list(csv_range=cpus)
|
|
||||||
guaranteed_cpuset.update(cpulist)
|
|
||||||
guaranteed_cpuranges = format_range_set(guaranteed_cpuset)
|
|
||||||
|
|
||||||
# Update PM QoS resume latency if the set of cpus have changed
|
|
||||||
if default_cpuset != self.cpusets['default']:
|
|
||||||
self.cpusets['default'] = default_cpuset.copy()
|
|
||||||
if default_cpuset:
|
|
||||||
pm_policy = 'high'
|
|
||||||
LOG.info('Set PM policy: %s, CPUs: %s',
|
|
||||||
pm_policy, default_cpuranges)
|
|
||||||
command = [pm_script, pm_policy, default_cpuranges]
|
|
||||||
proc = subprocess.Popen(command, stdout=subprocess.PIPE)
|
|
||||||
output, errors = proc.communicate()
|
|
||||||
if errors:
|
|
||||||
LOG.error('Problem with command: %s, error: %s',
|
|
||||||
command, errors)
|
|
||||||
|
|
||||||
if guaranteed_cpuset != self.cpusets['guaranteed']:
|
|
||||||
self.cpusets['guaranteed'] = guaranteed_cpuset.copy()
|
|
||||||
if guaranteed_cpuset:
|
|
||||||
pm_policy = 'low'
|
|
||||||
LOG.info('Set PM policy: %s, CPUs: %s',
|
|
||||||
pm_policy, guaranteed_cpuranges)
|
|
||||||
command = [pm_script, pm_policy, guaranteed_cpuranges]
|
|
||||||
proc = subprocess.Popen(command, stdout=subprocess.PIPE)
|
|
||||||
output, errors = proc.communicate()
|
|
||||||
if errors:
|
|
||||||
LOG.error('Problem with command: %s, error: %s',
|
|
||||||
command, errors)
|
|
||||||
|
|
||||||
def process_IN_MOVED_TO(self, event):
|
|
||||||
""" Handler for watched IN_MOVED_TO events.
|
|
||||||
|
|
||||||
kubelet cpu-manager overwrites state-file by moving a temp file,
|
|
||||||
so this is the expected handler.
|
|
||||||
"""
|
|
||||||
self.update_pm_qos_cpu_latency(event)
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
""" A shell command for pm-qos-daemon. """
|
|
||||||
configure_logging(LOG, level=logging.INFO)
|
|
||||||
if os.geteuid() != 0:
|
|
||||||
LOG.error('Require sudo/root.')
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
LOG.info('Watching: %s', statefile)
|
|
||||||
watch_manager = pyinotify.WatchManager()
|
|
||||||
notifier = pyinotify.Notifier(watch_manager)
|
|
||||||
flags = pyinotify.IN_MOVED_TO
|
|
||||||
watch_manager.watch_transient_file(statefile, flags, ProcessTransientFile)
|
|
||||||
|
|
||||||
try:
|
|
||||||
notifier.loop()
|
|
||||||
except pyinotify.NotifierError as err:
|
|
||||||
LOG.error('Problem with notifier.loop(), error: %s', err)
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
@ -1,38 +0,0 @@
|
|||||||
[metadata]
|
|
||||||
license_files = LICENSE
|
|
||||||
name = pm_qos_mgr
|
|
||||||
summary = PM QoS CPU wakeup latency manager for kubelet cpu-manager
|
|
||||||
author = StarlingX
|
|
||||||
author-email = starlingx-discuss@lists.starlingx.io
|
|
||||||
home-page = http://www.starlingx.io/
|
|
||||||
classifier =
|
|
||||||
Environment :: OpenStack
|
|
||||||
Intended Audience :: Information Technology
|
|
||||||
Intended Audience :: System Administrators
|
|
||||||
License :: OSI Approved :: Apache Software License
|
|
||||||
Operating System :: POSIX :: Linux
|
|
||||||
Programming Language :: Python
|
|
||||||
Programming Language :: Python :: 2
|
|
||||||
Programming Language :: Python :: 2.7
|
|
||||||
Programming Language :: Python :: 3
|
|
||||||
Programming Language :: Python :: 3.5
|
|
||||||
|
|
||||||
[global]
|
|
||||||
setup-hooks =
|
|
||||||
pbr.hooks.setup_hook
|
|
||||||
|
|
||||||
[files]
|
|
||||||
packages =
|
|
||||||
pm_qos_mgr
|
|
||||||
|
|
||||||
[entry_points]
|
|
||||||
console_scripts =
|
|
||||||
pm-qos-mgr = pm_qos_mgr.pm_qos_mgr:main
|
|
||||||
|
|
||||||
[wheel]
|
|
||||||
universal = 1
|
|
||||||
|
|
||||||
[egg_info]
|
|
||||||
tag_build =
|
|
||||||
tag_date = 0
|
|
||||||
tag_svn_revision = 0
|
|
@ -1,12 +0,0 @@
|
|||||||
#
|
|
||||||
# Copyright (c) 2019 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
|
|
||||||
import setuptools
|
|
||||||
|
|
||||||
setuptools.setup(
|
|
||||||
setup_requires=['pbr>=2.0.0'],
|
|
||||||
pbr=True)
|
|
||||||
|
|
6
worker-utils/.gitignore
vendored
6
worker-utils/.gitignore
vendored
@ -1,6 +0,0 @@
|
|||||||
!.distro
|
|
||||||
.distro/centos7/rpmbuild/RPMS
|
|
||||||
.distro/centos7/rpmbuild/SRPMS
|
|
||||||
.distro/centos7/rpmbuild/BUILD
|
|
||||||
.distro/centos7/rpmbuild/BUILDROOT
|
|
||||||
.distro/centos7/rpmbuild/SOURCES/worker-utils*tar.gz
|
|
@ -1,3 +0,0 @@
|
|||||||
SRC_DIR="worker-utils"
|
|
||||||
COPY_LIST="$SRC_DIR/LICENSE"
|
|
||||||
TIS_PATCH_VER=4
|
|
@ -1,58 +0,0 @@
|
|||||||
Summary: Initial worker node resource reservation and misc. utilities
|
|
||||||
Name: worker-utils
|
|
||||||
Version: 1.0
|
|
||||||
Release: %{tis_patch_ver}%{?_tis_dist}
|
|
||||||
License: Apache-2.0
|
|
||||||
Group: base
|
|
||||||
Packager: Wind River <info@windriver.com>
|
|
||||||
URL: unknown
|
|
||||||
Source0: %{name}-%{version}.tar.gz
|
|
||||||
Source1: LICENSE
|
|
||||||
|
|
||||||
BuildRequires: systemd-devel
|
|
||||||
Requires: systemd
|
|
||||||
Requires: python
|
|
||||||
Requires: /bin/systemctl
|
|
||||||
Requires: perl
|
|
||||||
|
|
||||||
%description
|
|
||||||
Initial worker node resource reservation and misc. utilities
|
|
||||||
|
|
||||||
%define local_bindir /usr/bin/
|
|
||||||
%define local_etc_initd /etc/init.d/
|
|
||||||
%define local_etc_platform /etc/platform/
|
|
||||||
%define local_etc_goenabledd /etc/goenabled.d/
|
|
||||||
|
|
||||||
%define debug_package %{nil}
|
|
||||||
|
|
||||||
%prep
|
|
||||||
%setup
|
|
||||||
|
|
||||||
%build
|
|
||||||
make
|
|
||||||
|
|
||||||
%install
|
|
||||||
make install BINDIR=%{buildroot}%{local_bindir} \
|
|
||||||
INITDDIR=%{buildroot}%{local_etc_initd} \
|
|
||||||
GOENABLEDDIR=%{buildroot}%{local_etc_goenabledd} \
|
|
||||||
PLATFORMCONFDIR=%{buildroot}%{local_etc_platform} \
|
|
||||||
SYSTEMDDIR=%{buildroot}%{_unitdir}
|
|
||||||
|
|
||||||
%post
|
|
||||||
/bin/systemctl enable affine-platform.sh.service >/dev/null 2>&1
|
|
||||||
/bin/systemctl enable affine-tasks.service >/dev/null 2>&1
|
|
||||||
|
|
||||||
%clean
|
|
||||||
rm -rf $RPM_BUILD_ROOT
|
|
||||||
|
|
||||||
%files
|
|
||||||
|
|
||||||
%defattr(-,root,root,-)
|
|
||||||
|
|
||||||
%{local_bindir}/*
|
|
||||||
%{local_etc_initd}/*
|
|
||||||
%{local_etc_goenabledd}/*
|
|
||||||
%config(noreplace) %{local_etc_platform}/worker_reserved.conf
|
|
||||||
|
|
||||||
%{_unitdir}/affine-platform.sh.service
|
|
||||||
%{_unitdir}/affine-tasks.service
|
|
@ -1,202 +0,0 @@
|
|||||||
|
|
||||||
Apache License
|
|
||||||
Version 2.0, January 2004
|
|
||||||
http://www.apache.org/licenses/
|
|
||||||
|
|
||||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
||||||
|
|
||||||
1. Definitions.
|
|
||||||
|
|
||||||
"License" shall mean the terms and conditions for use, reproduction,
|
|
||||||
and distribution as defined by Sections 1 through 9 of this document.
|
|
||||||
|
|
||||||
"Licensor" shall mean the copyright owner or entity authorized by
|
|
||||||
the copyright owner that is granting the License.
|
|
||||||
|
|
||||||
"Legal Entity" shall mean the union of the acting entity and all
|
|
||||||
other entities that control, are controlled by, or are under common
|
|
||||||
control with that entity. For the purposes of this definition,
|
|
||||||
"control" means (i) the power, direct or indirect, to cause the
|
|
||||||
direction or management of such entity, whether by contract or
|
|
||||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
||||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
||||||
|
|
||||||
"You" (or "Your") shall mean an individual or Legal Entity
|
|
||||||
exercising permissions granted by this License.
|
|
||||||
|
|
||||||
"Source" form shall mean the preferred form for making modifications,
|
|
||||||
including but not limited to software source code, documentation
|
|
||||||
source, and configuration files.
|
|
||||||
|
|
||||||
"Object" form shall mean any form resulting from mechanical
|
|
||||||
transformation or translation of a Source form, including but
|
|
||||||
not limited to compiled object code, generated documentation,
|
|
||||||
and conversions to other media types.
|
|
||||||
|
|
||||||
"Work" shall mean the work of authorship, whether in Source or
|
|
||||||
Object form, made available under the License, as indicated by a
|
|
||||||
copyright notice that is included in or attached to the work
|
|
||||||
(an example is provided in the Appendix below).
|
|
||||||
|
|
||||||
"Derivative Works" shall mean any work, whether in Source or Object
|
|
||||||
form, that is based on (or derived from) the Work and for which the
|
|
||||||
editorial revisions, annotations, elaborations, or other modifications
|
|
||||||
represent, as a whole, an original work of authorship. For the purposes
|
|
||||||
of this License, Derivative Works shall not include works that remain
|
|
||||||
separable from, or merely link (or bind by name) to the interfaces of,
|
|
||||||
the Work and Derivative Works thereof.
|
|
||||||
|
|
||||||
"Contribution" shall mean any work of authorship, including
|
|
||||||
the original version of the Work and any modifications or additions
|
|
||||||
to that Work or Derivative Works thereof, that is intentionally
|
|
||||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
||||||
or by an individual or Legal Entity authorized to submit on behalf of
|
|
||||||
the copyright owner. For the purposes of this definition, "submitted"
|
|
||||||
means any form of electronic, verbal, or written communication sent
|
|
||||||
to the Licensor or its representatives, including but not limited to
|
|
||||||
communication on electronic mailing lists, source code control systems,
|
|
||||||
and issue tracking systems that are managed by, or on behalf of, the
|
|
||||||
Licensor for the purpose of discussing and improving the Work, but
|
|
||||||
excluding communication that is conspicuously marked or otherwise
|
|
||||||
designated in writing by the copyright owner as "Not a Contribution."
|
|
||||||
|
|
||||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
||||||
on behalf of whom a Contribution has been received by Licensor and
|
|
||||||
subsequently incorporated within the Work.
|
|
||||||
|
|
||||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
||||||
this License, each Contributor hereby grants to You a perpetual,
|
|
||||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
||||||
copyright license to reproduce, prepare Derivative Works of,
|
|
||||||
publicly display, publicly perform, sublicense, and distribute the
|
|
||||||
Work and such Derivative Works in Source or Object form.
|
|
||||||
|
|
||||||
3. Grant of Patent License. Subject to the terms and conditions of
|
|
||||||
this License, each Contributor hereby grants to You a perpetual,
|
|
||||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
||||||
(except as stated in this section) patent license to make, have made,
|
|
||||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
||||||
where such license applies only to those patent claims licensable
|
|
||||||
by such Contributor that are necessarily infringed by their
|
|
||||||
Contribution(s) alone or by combination of their Contribution(s)
|
|
||||||
with the Work to which such Contribution(s) was submitted. If You
|
|
||||||
institute patent litigation against any entity (including a
|
|
||||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
||||||
or a Contribution incorporated within the Work constitutes direct
|
|
||||||
or contributory patent infringement, then any patent licenses
|
|
||||||
granted to You under this License for that Work shall terminate
|
|
||||||
as of the date such litigation is filed.
|
|
||||||
|
|
||||||
4. Redistribution. You may reproduce and distribute copies of the
|
|
||||||
Work or Derivative Works thereof in any medium, with or without
|
|
||||||
modifications, and in Source or Object form, provided that You
|
|
||||||
meet the following conditions:
|
|
||||||
|
|
||||||
(a) You must give any other recipients of the Work or
|
|
||||||
Derivative Works a copy of this License; and
|
|
||||||
|
|
||||||
(b) You must cause any modified files to carry prominent notices
|
|
||||||
stating that You changed the files; and
|
|
||||||
|
|
||||||
(c) You must retain, in the Source form of any Derivative Works
|
|
||||||
that You distribute, all copyright, patent, trademark, and
|
|
||||||
attribution notices from the Source form of the Work,
|
|
||||||
excluding those notices that do not pertain to any part of
|
|
||||||
the Derivative Works; and
|
|
||||||
|
|
||||||
(d) If the Work includes a "NOTICE" text file as part of its
|
|
||||||
distribution, then any Derivative Works that You distribute must
|
|
||||||
include a readable copy of the attribution notices contained
|
|
||||||
within such NOTICE file, excluding those notices that do not
|
|
||||||
pertain to any part of the Derivative Works, in at least one
|
|
||||||
of the following places: within a NOTICE text file distributed
|
|
||||||
as part of the Derivative Works; within the Source form or
|
|
||||||
documentation, if provided along with the Derivative Works; or,
|
|
||||||
within a display generated by the Derivative Works, if and
|
|
||||||
wherever such third-party notices normally appear. The contents
|
|
||||||
of the NOTICE file are for informational purposes only and
|
|
||||||
do not modify the License. You may add Your own attribution
|
|
||||||
notices within Derivative Works that You distribute, alongside
|
|
||||||
or as an addendum to the NOTICE text from the Work, provided
|
|
||||||
that such additional attribution notices cannot be construed
|
|
||||||
as modifying the License.
|
|
||||||
|
|
||||||
You may add Your own copyright statement to Your modifications and
|
|
||||||
may provide additional or different license terms and conditions
|
|
||||||
for use, reproduction, or distribution of Your modifications, or
|
|
||||||
for any such Derivative Works as a whole, provided Your use,
|
|
||||||
reproduction, and distribution of the Work otherwise complies with
|
|
||||||
the conditions stated in this License.
|
|
||||||
|
|
||||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
||||||
any Contribution intentionally submitted for inclusion in the Work
|
|
||||||
by You to the Licensor shall be under the terms and conditions of
|
|
||||||
this License, without any additional terms or conditions.
|
|
||||||
Notwithstanding the above, nothing herein shall supersede or modify
|
|
||||||
the terms of any separate license agreement you may have executed
|
|
||||||
with Licensor regarding such Contributions.
|
|
||||||
|
|
||||||
6. Trademarks. This License does not grant permission to use the trade
|
|
||||||
names, trademarks, service marks, or product names of the Licensor,
|
|
||||||
except as required for reasonable and customary use in describing the
|
|
||||||
origin of the Work and reproducing the content of the NOTICE file.
|
|
||||||
|
|
||||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
||||||
agreed to in writing, Licensor provides the Work (and each
|
|
||||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
||||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
||||||
implied, including, without limitation, any warranties or conditions
|
|
||||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
||||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
||||||
appropriateness of using or redistributing the Work and assume any
|
|
||||||
risks associated with Your exercise of permissions under this License.
|
|
||||||
|
|
||||||
8. Limitation of Liability. In no event and under no legal theory,
|
|
||||||
whether in tort (including negligence), contract, or otherwise,
|
|
||||||
unless required by applicable law (such as deliberate and grossly
|
|
||||||
negligent acts) or agreed to in writing, shall any Contributor be
|
|
||||||
liable to You for damages, including any direct, indirect, special,
|
|
||||||
incidental, or consequential damages of any character arising as a
|
|
||||||
result of this License or out of the use or inability to use the
|
|
||||||
Work (including but not limited to damages for loss of goodwill,
|
|
||||||
work stoppage, computer failure or malfunction, or any and all
|
|
||||||
other commercial damages or losses), even if such Contributor
|
|
||||||
has been advised of the possibility of such damages.
|
|
||||||
|
|
||||||
9. Accepting Warranty or Additional Liability. While redistributing
|
|
||||||
the Work or Derivative Works thereof, You may choose to offer,
|
|
||||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
||||||
or other liability obligations and/or rights consistent with this
|
|
||||||
License. However, in accepting such obligations, You may act only
|
|
||||||
on Your own behalf and on Your sole responsibility, not on behalf
|
|
||||||
of any other Contributor, and only if You agree to indemnify,
|
|
||||||
defend, and hold each Contributor harmless for any liability
|
|
||||||
incurred by, or claims asserted against, such Contributor by reason
|
|
||||||
of your accepting any such warranty or additional liability.
|
|
||||||
|
|
||||||
END OF TERMS AND CONDITIONS
|
|
||||||
|
|
||||||
APPENDIX: How to apply the Apache License to your work.
|
|
||||||
|
|
||||||
To apply the Apache License to your work, attach the following
|
|
||||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
||||||
replaced with your own identifying information. (Don't include
|
|
||||||
the brackets!) The text should be enclosed in the appropriate
|
|
||||||
comment syntax for the file format. We also recommend that a
|
|
||||||
file or class name and description of purpose be included on the
|
|
||||||
same "printed page" as the copyright notice for easier
|
|
||||||
identification within third-party archives.
|
|
||||||
|
|
||||||
Copyright [yyyy] [name of copyright owner]
|
|
||||||
|
|
||||||
Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
you may not use this file except in compliance with the License.
|
|
||||||
You may obtain a copy of the License at
|
|
||||||
|
|
||||||
http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
|
|
||||||
Unless required by applicable law or agreed to in writing, software
|
|
||||||
distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
See the License for the specific language governing permissions and
|
|
||||||
limitations under the License.
|
|
@ -1,33 +0,0 @@
|
|||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
|
|
||||||
BINDIR ?= /usr/bin
|
|
||||||
INITDDIR ?= /etc/init.d/
|
|
||||||
GOENABLEDDIR ?= /etc/goenabled.d/
|
|
||||||
PLATFORMCONFDIR ?= /etc/platform
|
|
||||||
SYSTEMDDIR ?= /usr/lib/systemd/system/
|
|
||||||
|
|
||||||
all:
|
|
||||||
python -m compileall topology.py
|
|
||||||
|
|
||||||
install:
|
|
||||||
install -d -m 755 $(BINDIR)
|
|
||||||
install -d -m 755 $(INITDDIR)
|
|
||||||
install -d -m 755 $(GOENABLEDDIR)
|
|
||||||
install -d -m 755 $(PLATFORMCONFDIR)
|
|
||||||
install -d -m 755 $(SYSTEMDDIR)
|
|
||||||
install -p -D -m 755 affine-platform.sh $(INITDDIR)/affine-platform.sh
|
|
||||||
install -p -D -m 755 affine-tasks.sh $(INITDDIR)/affine-tasks.sh
|
|
||||||
install -p -D -m 755 cpumap_functions.sh $(INITDDIR)/cpumap_functions.sh
|
|
||||||
install -p -D -m 755 task_affinity_functions.sh $(INITDDIR)/task_affinity_functions.sh
|
|
||||||
install -p -D -m 755 ps-sched.sh $(BINDIR)/ps-sched.sh
|
|
||||||
install -p -D -m 755 topology.py $(BINDIR)/topology.py
|
|
||||||
install -p -D -m 755 topology.pyc $(BINDIR)/topology.pyc
|
|
||||||
install -p -D -m 755 affine-interrupts.sh $(BINDIR)/affine-interrupts.sh
|
|
||||||
install -p -D -m 755 set-cpu-wakeup-latency.sh $(BINDIR)/set-cpu-wakeup-latency.sh
|
|
||||||
install -p -D -m 755 topology $(BINDIR)/topology
|
|
||||||
install -p -D -m 644 worker_reserved.conf $(PLATFORMCONFDIR)/worker_reserved.conf
|
|
||||||
install -p -D -m 755 worker-goenabled.sh $(GOENABLEDDIR)/worker-goenabled.sh
|
|
||||||
install -p -D -m 664 affine-platform.sh.service $(SYSTEMDDIR)/affine-platform.sh.service
|
|
||||||
install -p -D -m 664 affine-tasks.service $(SYSTEMDDIR)/affine-tasks.service
|
|
@ -1,59 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
################################################################################
|
|
||||||
# Copyright (c) 2015-2016 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
#
|
|
||||||
# Purpose:
|
|
||||||
# Affine the interface IRQ to specified cpulist.
|
|
||||||
#
|
|
||||||
# Usage: /usr/bin/affine-interrupts.sh interface cpulist
|
|
||||||
#
|
|
||||||
# Define minimal path
|
|
||||||
PATH=/bin:/usr/bin:/usr/local/bin
|
|
||||||
|
|
||||||
# logger setup
|
|
||||||
WHOAMI=`basename $0`
|
|
||||||
LOG_FACILITY=user
|
|
||||||
LOG_PRIORITY=info
|
|
||||||
TMPLOG=/tmp/${WHOAMI}.log
|
|
||||||
|
|
||||||
# LOG() - generates log and puts in temporary file
|
|
||||||
function LOG {
|
|
||||||
logger -t "${0##*/}[$$]" -p ${LOG_FACILITY}.${LOG_PRIORITY} "$@"
|
|
||||||
echo "${0##*/}[$$]" "$@" >> ${TMPLOG}
|
|
||||||
}
|
|
||||||
function INFO {
|
|
||||||
MSG="INFO"
|
|
||||||
LOG "${MSG} $@"
|
|
||||||
}
|
|
||||||
function ERROR {
|
|
||||||
MSG="ERROR"
|
|
||||||
LOG "${MSG} $@"
|
|
||||||
}
|
|
||||||
|
|
||||||
if [ "$#" -ne 2 ]; then
|
|
||||||
ERROR "Interface name and cpulist are required"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
interface=$1
|
|
||||||
cpulist=$2
|
|
||||||
|
|
||||||
# Find PCI device matching interface, keep last matching device name
|
|
||||||
dev=$(find /sys/devices -name "${interface}" | \
|
|
||||||
perl -ne 'print $1 if /([[:xdigit:]]{4}:[[:xdigit:]]{2}:[[:xdigit:]]{2}\.[[:xdigit:]])\/[[:alpha:]]/;')
|
|
||||||
|
|
||||||
# Obtain all IRQs for this device
|
|
||||||
irq=$(cat /sys/bus/pci/devices/${dev}/irq 2>/dev/null)
|
|
||||||
msi_irqs=$(ls /sys/bus/pci/devices/${dev}/msi_irqs 2>/dev/null | xargs)
|
|
||||||
|
|
||||||
INFO $LINENO "affine ${interface} (dev:${dev} irq:${irq} msi_irqs:${msi_irqs}) with cpus (${cpulist})"
|
|
||||||
|
|
||||||
for i in $(echo "${irq} ${msi_irqs}"); do echo $i; done | \
|
|
||||||
xargs --no-run-if-empty -i{} \
|
|
||||||
/bin/bash -c "[[ -e /proc/irq/{} ]] && echo ${cpulist} > /proc/irq/{}/smp_affinity_list" 2>/dev/null
|
|
||||||
|
|
||||||
exit 0
|
|
@ -1,163 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
################################################################################
|
|
||||||
# Copyright (c) 2013 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
#
|
|
||||||
### BEGIN INIT INFO
|
|
||||||
# Provides: affine-platform
|
|
||||||
# Required-Start:
|
|
||||||
# Required-Stop:
|
|
||||||
# Default-Start: 2 3 4 5
|
|
||||||
# Default-Stop: 0 1 6
|
|
||||||
# Short-Description: Affine platform
|
|
||||||
### END INIT INFO
|
|
||||||
|
|
||||||
# Define minimal path
|
|
||||||
PATH=/bin:/usr/bin:/usr/local/bin
|
|
||||||
|
|
||||||
LOG_FUNCTIONS=${LOG_FUNCTIONS:-"/etc/init.d/log_functions.sh"}
|
|
||||||
CPUMAP_FUNCTIONS=${CPUMAP_FUNCTIONS:-"/etc/init.d/cpumap_functions.sh"}
|
|
||||||
TASK_AFFINITY_FUNCTIONS=${TASK_AFFINITY_FUNCTIONS:-"/etc/init.d/task_affinity_functions.sh"}
|
|
||||||
source /etc/init.d/functions
|
|
||||||
[[ -e ${LOG_FUNCTIONS} ]] && source ${LOG_FUNCTIONS}
|
|
||||||
[[ -e ${CPUMAP_FUNCTIONS} ]] && source ${CPUMAP_FUNCTIONS}
|
|
||||||
[[ -e ${TASK_AFFINITY_FUNCTIONS} ]] && source ${TASK_AFFINITY_FUNCTIONS}
|
|
||||||
linkname=$(readlink -n -f $0)
|
|
||||||
scriptname=$(basename $linkname)
|
|
||||||
|
|
||||||
# Enable debug logs
|
|
||||||
LOG_DEBUG=1
|
|
||||||
|
|
||||||
. /etc/platform/platform.conf
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Affine all running tasks to the CPULIST provided in the first parameter.
|
|
||||||
################################################################################
|
|
||||||
function affine_tasks {
|
|
||||||
local CPULIST=$1
|
|
||||||
local PIDLIST
|
|
||||||
local RET=0
|
|
||||||
|
|
||||||
# Get number of logical cpus
|
|
||||||
N_CPUS=$(cat /proc/cpuinfo 2>/dev/null | \
|
|
||||||
awk '/^[pP]rocessor/ { n +=1 } END { print (n>0) ? n : 1}')
|
|
||||||
|
|
||||||
# Calculate platform cores cpumap
|
|
||||||
PLATFORM_COREMASK=$(cpulist_to_cpumap ${CPULIST} ${N_CPUS})
|
|
||||||
|
|
||||||
# Set default IRQ affinity
|
|
||||||
echo ${PLATFORM_COREMASK} > /proc/irq/default_smp_affinity
|
|
||||||
|
|
||||||
# Affine all PCI/MSI interrupts to platform cores; this overrides
|
|
||||||
# irqaffinity boot arg, since that does not handle IRQs for PCI devices
|
|
||||||
# on numa nodes that do not intersect with platform cores.
|
|
||||||
PCIDEVS=/sys/bus/pci/devices
|
|
||||||
declare -a irqs=()
|
|
||||||
irqs+=($(cat ${PCIDEVS}/*/irq 2>/dev/null | xargs))
|
|
||||||
irqs+=($(ls ${PCIDEVS}/*/msi_irqs 2>/dev/null | grep -E '^[0-9]+$' | xargs))
|
|
||||||
# flatten list of irqs, removing duplicates
|
|
||||||
irqs=($(echo ${irqs[@]} | tr ' ' '\n' | sort -nu))
|
|
||||||
log_debug "Affining all PCI/MSI irqs(${irqs[@]}) with cpus (${CPULIST})"
|
|
||||||
for i in ${irqs[@]}; do
|
|
||||||
/bin/bash -c "[[ -e /proc/irq/${i} ]] && echo ${CPULIST} > /proc/irq/${i}/smp_affinity_list" 2>/dev/null
|
|
||||||
done
|
|
||||||
if [[ "$subfunction" == *"worker,lowlatency" ]]; then
|
|
||||||
# Affine work queues to platform cores
|
|
||||||
echo ${PLATFORM_COREMASK} > /sys/devices/virtual/workqueue/cpumask
|
|
||||||
echo ${PLATFORM_COREMASK} > /sys/bus/workqueue/devices/writeback/cpumask
|
|
||||||
|
|
||||||
# On low latency compute reassign the per cpu threads rcuc, ksoftirq,
|
|
||||||
# ktimersoftd to FIFO along with the specified priority
|
|
||||||
PIDLIST=$( ps -e -p 2 |grep rcuc | awk '{ print $1; }')
|
|
||||||
for PID in ${PIDLIST[@]}; do
|
|
||||||
chrt -p -f 4 ${PID} 2>/dev/null
|
|
||||||
done
|
|
||||||
|
|
||||||
PIDLIST=$( ps -e -p 2 |grep ksoftirq | awk '{ print $1; }')
|
|
||||||
for PID in ${PIDLIST[@]}; do
|
|
||||||
chrt -p -f 2 ${PID} 2>/dev/null
|
|
||||||
done
|
|
||||||
|
|
||||||
PIDLIST=$( ps -e -p 2 |grep ktimersoftd | awk '{ print $1; }')
|
|
||||||
for PID in ${PIDLIST[@]}; do
|
|
||||||
chrt -p -f 3 ${PID} 2>/dev/null
|
|
||||||
done
|
|
||||||
fi
|
|
||||||
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Start Action
|
|
||||||
################################################################################
|
|
||||||
function start {
|
|
||||||
local RET=0
|
|
||||||
|
|
||||||
echo -n "Starting ${scriptname}: "
|
|
||||||
|
|
||||||
## Check whether we are root (need root for taskset)
|
|
||||||
if [ $UID -ne 0 ]; then
|
|
||||||
log_error "require root or sudo"
|
|
||||||
RET=1
|
|
||||||
return ${RET}
|
|
||||||
fi
|
|
||||||
|
|
||||||
## Define platform cpulist to be thread siblings of core 0
|
|
||||||
PLATFORM_CPULIST=$(get_platform_cpu_list)
|
|
||||||
|
|
||||||
# Affine all tasks to platform cpulist
|
|
||||||
affine_tasks ${PLATFORM_CPULIST}
|
|
||||||
RET=$?
|
|
||||||
if [ ${RET} -ne 0 ]; then
|
|
||||||
log_error "Failed to affine tasks ${PLATFORM_CPULIST}, rc=${RET}"
|
|
||||||
return ${RET}
|
|
||||||
fi
|
|
||||||
|
|
||||||
print_status ${RET}
|
|
||||||
return ${RET}
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Stop Action - don't do anything
|
|
||||||
################################################################################
|
|
||||||
function stop {
|
|
||||||
local RET=0
|
|
||||||
echo -n "Stopping ${scriptname}: "
|
|
||||||
print_status ${RET}
|
|
||||||
return ${RET}
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Restart Action
|
|
||||||
################################################################################
|
|
||||||
function restart {
|
|
||||||
stop
|
|
||||||
start
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Main Entry
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
case "$1" in
|
|
||||||
start)
|
|
||||||
start
|
|
||||||
;;
|
|
||||||
stop)
|
|
||||||
stop
|
|
||||||
;;
|
|
||||||
restart|reload)
|
|
||||||
restart
|
|
||||||
;;
|
|
||||||
status)
|
|
||||||
echo -n "OK"
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
echo $"Usage: $0 {start|stop|restart|reload|status}"
|
|
||||||
exit 1
|
|
||||||
esac
|
|
||||||
|
|
||||||
exit $?
|
|
@ -1,14 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=Titanium Cloud Affine Platform
|
|
||||||
After=syslog.service network.service dbus.service sw-patch.service
|
|
||||||
Before=workerconfig.service
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=oneshot
|
|
||||||
RemainAfterExit=yes
|
|
||||||
ExecStart=/etc/init.d/affine-platform.sh start
|
|
||||||
ExecStop=/etc/init.d/affine-platform.sh stop
|
|
||||||
ExecReload=/etc/init.d/affine-platform.sh restart
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
@ -1,11 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=StarlingX Affine Tasks
|
|
||||||
After=syslog.service network.service dbus.service sw-patch.service affine-platform.sh.service
|
|
||||||
Before=workerconfig.service
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=simple
|
|
||||||
ExecStart=/etc/init.d/affine-tasks.sh start
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
@ -1,536 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
#
|
|
||||||
# Copyright (c) 2019 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
|
|
||||||
#
|
|
||||||
# chkconfig: 2345 80 80
|
|
||||||
#
|
|
||||||
|
|
||||||
### BEGIN INIT INFO
|
|
||||||
# Provides: affine-tasks
|
|
||||||
# Required-Start:
|
|
||||||
# Required-Stop:
|
|
||||||
# Default-Start: 2 3 4 5
|
|
||||||
# Default-Stop: 0 1 6
|
|
||||||
# Short-Description: reaffine tasks on AIO
|
|
||||||
# Description: This script will dynamically reaffine tasks
|
|
||||||
# and k8s-infra cgroup cpuset on AIO nodes only. This accomodates
|
|
||||||
# CPU intensive phases of work. Tasks are initially allowed to float
|
|
||||||
# across all cores. Once system is at steady-state, this will ensure
|
|
||||||
# that K8S pods are constrained to platform cores and do not run on
|
|
||||||
# cores with VMs/containers.
|
|
||||||
### END INIT INFO
|
|
||||||
#
|
|
||||||
# Background:
|
|
||||||
# There is significant parallel CPU intensive activity:
|
|
||||||
# - during stx-application apply before critical openstack pods are running,
|
|
||||||
# e.g., to download docker images, and start all pods.
|
|
||||||
# - during init and pod recovery after reboot or DOR.
|
|
||||||
#
|
|
||||||
# This enables use of all cpus during CPU intensive phase, otherwise the
|
|
||||||
# startup processing time is considerably longer and we easily hit timeout.
|
|
||||||
#
|
|
||||||
# This script waits forever for sufficient platform readiness criteria
|
|
||||||
# (e.g., system critical pods are recovered, nova-compute is running,
|
|
||||||
# cinder-volume is running, openstack pods are running), and we have waited
|
|
||||||
# a short stabilization period before reaffining to the platform cpus.
|
|
||||||
#
|
|
||||||
# NOTE: child cgroup cpuset and nodeset must be a subset of the parent
|
|
||||||
# cgroup's attributes. This requires traversing the tree hierachy in
|
|
||||||
# specific order when dynamically modifying these attributes.
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
# Define minimal path
|
|
||||||
PATH=/bin:/usr/bin:/usr/sbin:/usr/local/bin
|
|
||||||
|
|
||||||
CPUMAP_FUNCTIONS=${CPUMAP_FUNCTIONS:-"/etc/init.d/cpumap_functions.sh"}
|
|
||||||
[[ -e ${CPUMAP_FUNCTIONS} ]] && source ${CPUMAP_FUNCTIONS}
|
|
||||||
|
|
||||||
# Bring in platform definitions
|
|
||||||
. /etc/platform/platform.conf
|
|
||||||
|
|
||||||
# Environment for kubectl
|
|
||||||
export KUBECONFIG=/etc/kubernetes/admin.conf
|
|
||||||
|
|
||||||
# Global parameters
|
|
||||||
CGDIR_K8S=/sys/fs/cgroup/cpuset/k8s-infra
|
|
||||||
INIT_INTERVAL_SECONDS=10
|
|
||||||
CHECK_INTERVAL_SECONDS=30
|
|
||||||
PRINT_INTERVAL_SECONDS=300
|
|
||||||
STABILIZATION_SECONDS=150
|
|
||||||
|
|
||||||
# Define pidfile
|
|
||||||
LNAME=$(readlink -n -f $0)
|
|
||||||
NAME=$(basename $LNAME)
|
|
||||||
PIDFILE=/var/run/${NAME}.pid
|
|
||||||
|
|
||||||
# Define number of logical cpus
|
|
||||||
LOGICAL_CPUS=$(getconf _NPROCESSORS_ONLN)
|
|
||||||
|
|
||||||
# Define the memory nodeset and cpuset that span all online cpus and nodes
|
|
||||||
ONLINE_NODES=$(/bin/cat /sys/devices/system/node/online)
|
|
||||||
ONLINE_CPUS=$(/bin/cat /sys/devices/system/cpu/online)
|
|
||||||
ONLINE_MASK=$(cpulist_to_cpumap ${ONLINE_CPUS} ${LOGICAL_CPUS} | \
|
|
||||||
awk '{print tolower($0)}')
|
|
||||||
|
|
||||||
ISOL_CPUS=$(/bin/cat /sys/devices/system/cpu/isolated)
|
|
||||||
if [ ! -z "${ISOL_CPUS}" ]; then
|
|
||||||
ISOL_CPUMAP=$(cpulist_to_cpumap ${ISOL_CPUS} ${LOGICAL_CPUS})
|
|
||||||
NONISOL_CPUMAP=$(invert_cpumap ${ISOL_CPUMAP} ${LOGICAL_CPUS})
|
|
||||||
NONISOL_CPUS=$(cpumap_to_cpulist ${NONISOL_CPUMAP} ${LOGICAL_CPUS})
|
|
||||||
NONISOL_MASK=$(cpulist_to_cpumap ${NONISOL_CPUS} ${LOGICAL_CPUS} | \
|
|
||||||
awk '{print tolower($0)}')
|
|
||||||
else
|
|
||||||
ISOL_CPUMAP='0'
|
|
||||||
NONISOL_CPUS=${ONLINE_CPUS}
|
|
||||||
NONISOL_MASK=${ONLINE_MASK}
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Define platform memory nodeset and cpuset
|
|
||||||
PLATFORM_NODES=$(cat /sys/devices/system/node/online)
|
|
||||||
PLATFORM_CPUS=$(platform_expanded_cpu_list)
|
|
||||||
|
|
||||||
# Global variables
|
|
||||||
NOT_READY_REASON=""
|
|
||||||
STABLE=0
|
|
||||||
|
|
||||||
# Log info message to /var/log/daemon.log
|
|
||||||
function LOG {
|
|
||||||
logger -p daemon.info -t "${NAME}($$): " "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
# Log error message to /var/log/daemon.log
|
|
||||||
function ERROR {
|
|
||||||
logger -s -p daemon.error -t "${NAME}($$): " "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
# Update cgroup k8s-infra cpuset and nodeset to span all non-isolated cpus.
|
|
||||||
function update_cgroup_cpuset_k8s_infra_all {
|
|
||||||
# Set all cgroup cpuset and nodeset in tree hierarchy order.
|
|
||||||
# This will always work, no matter the previous cpuset state.
|
|
||||||
find ${CGDIR_K8S} -type d | \
|
|
||||||
while read d; do
|
|
||||||
/bin/echo ${ONLINE_NODES} > ${d}/cpuset.mems 2>/dev/null
|
|
||||||
/bin/echo ${NONISOL_CPUS} > ${d}/cpuset.cpus 2>/dev/null
|
|
||||||
done
|
|
||||||
LOG "Update ${CGDIR_K8S}," \
|
|
||||||
"ONLINE_NODES=${ONLINE_NODES}, NONISOL_CPUS=${NONISOL_CPUS}"
|
|
||||||
}
|
|
||||||
|
|
||||||
# Update cgroup k8s-infra to span platform cpuset and nodeset.
|
|
||||||
function update_cgroup_cpuset_k8s_infra_platform {
|
|
||||||
# Clear any existing cpuset settings. This ensures that the
|
|
||||||
# subsequent shrink to platform cpuset will always work.
|
|
||||||
update_cgroup_cpuset_k8s_infra_all
|
|
||||||
|
|
||||||
# Set all cgroup cpuset and nodeset in depth-first order.
|
|
||||||
# NOTE: this only works if we are shrinking the cpuset.
|
|
||||||
find ${CGDIR_K8S} -depth -type d | \
|
|
||||||
while read d; do
|
|
||||||
/bin/echo ${PLATFORM_NODES} > ${d}/cpuset.mems 2>/dev/null
|
|
||||||
/bin/echo ${PLATFORM_CPUS} > ${d}/cpuset.cpus 2>/dev/null
|
|
||||||
done
|
|
||||||
LOG "Update ${CGDIR_K8S}," \
|
|
||||||
"PLATFORM_NODES=${PLATFORM_NODES}, PLATFORM_CPUS=${PLATFORM_CPUS}"
|
|
||||||
}
|
|
||||||
|
|
||||||
# Check criteria for K8s platform ready on this node.
|
|
||||||
# i.e., k8s-infra is configured, kubelet is running
|
|
||||||
function is_k8s_platform_ready {
|
|
||||||
local PASS=0
|
|
||||||
local FAIL=1
|
|
||||||
|
|
||||||
# Global variable
|
|
||||||
NOT_READY_REASON=""
|
|
||||||
|
|
||||||
# Check that cgroup cpuset k8s-infra has been configured
|
|
||||||
if [ ! -e ${CGDIR_K8S} ]; then
|
|
||||||
NOT_READY_REASON="k8s-infra not configured"
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Check that kubelet is running and stable
|
|
||||||
if systemctl is-active kubelet --quiet; then
|
|
||||||
PID=$(systemctl show kubelet.service -p MainPID | \
|
|
||||||
awk -vFS='=' '{print $2}')
|
|
||||||
if [ ${PID} -eq 0 ]; then
|
|
||||||
NOT_READY_REASON="kubelet not running"
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
up=$(ps -p ${PID} -o etimes= 2>/dev/null | awk '{print $1}')
|
|
||||||
if ! { [ -n "${up}" -a ${up} -ge 30 ]; }
|
|
||||||
then
|
|
||||||
NOT_READY_REASON="kubelet not yet stable"
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
else
|
|
||||||
NOT_READY_REASON="kubelet not running"
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
|
|
||||||
LOG "kubelet is ready"
|
|
||||||
return ${PASS}
|
|
||||||
}
|
|
||||||
|
|
||||||
# Determine whether this node has 'static' cpu manager policy.
|
|
||||||
# NOTE: This check assumes that kubelet is already running locally.
|
|
||||||
function is_static_cpu_manager_policy {
|
|
||||||
local PASS=0
|
|
||||||
local FAIL=1
|
|
||||||
|
|
||||||
state=$(cat /var/lib/kubelet/cpu_manager_state 2>/dev/null)
|
|
||||||
if [[ $state =~ \"policyName\":.?\"static\" ]]; then
|
|
||||||
return ${PASS}
|
|
||||||
else
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# Check criteria for K8s platform steady-state ready on this node.
|
|
||||||
# i.e., kube-system pods have recovered, kube application apply
|
|
||||||
# has completed, nova-compute is running, cinder-volume is running.
|
|
||||||
# NOTE: This function depends on kubectl commands, so is only
|
|
||||||
# usable on controllers.
|
|
||||||
function is_k8s_platform_steady_state_ready {
|
|
||||||
local PASS=0
|
|
||||||
local FAIL=1
|
|
||||||
local this_node=${HOSTNAME}
|
|
||||||
|
|
||||||
# Global variable
|
|
||||||
NOT_READY_REASON=""
|
|
||||||
|
|
||||||
# Check that kube-system pods have recovered on this node
|
|
||||||
npods=$(kubectl get pods --namespace kube-system --no-headers \
|
|
||||||
--field-selector spec.nodeName=${this_node} 2>/dev/null | \
|
|
||||||
awk '
|
|
||||||
BEGIN { n=0; }
|
|
||||||
!/Completed|Running/ { n+=1 }
|
|
||||||
END { printf "%d\n", n; }
|
|
||||||
')
|
|
||||||
if [ ${npods} -gt 0 ]; then
|
|
||||||
NOT_READY_REASON="${npods} kube-system pods not recovered"
|
|
||||||
STABLE=0
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Wait for a few critical openstack pods to be running if this is
|
|
||||||
# an openstack-compute-node. This is not an exhaustive list.
|
|
||||||
# Make sure that all openstack pods on this node are running.
|
|
||||||
labels=$(kubectl get node ${this_node} \
|
|
||||||
--no-headers --show-labels 2>/dev/null | awk '{print $NF}')
|
|
||||||
if [[ $labels =~ openstack-compute-node=enabled ]]; then
|
|
||||||
# nova-compute is one of the last charts to recover after reboot
|
|
||||||
PODS=( $(kubectl get pods --namespace openstack --no-headers \
|
|
||||||
--selector application=nova,component=compute \
|
|
||||||
--field-selector \
|
|
||||||
spec.nodeName=${this_node},status.phase=Running 2>/dev/null) )
|
|
||||||
if [ ${#PODS[@]} -eq 0 ]; then
|
|
||||||
NOT_READY_REASON="nova-compute pod not running"
|
|
||||||
STABLE=0
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
|
|
||||||
# cinder-volume is one of the last charts to recover after reboot
|
|
||||||
PODS=( $(kubectl get pods --namespace openstack --no-headers \
|
|
||||||
--selector application=cinder,component=volume \
|
|
||||||
--field-selector \
|
|
||||||
spec.nodeName=${this_node},status.phase=Running 2>/dev/null) )
|
|
||||||
if [ ${#PODS[@]} -eq 0 ]; then
|
|
||||||
NOT_READY_REASON="cinder-volume pod not running"
|
|
||||||
STABLE=0
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Check that all openstack pods on this node have recovered
|
|
||||||
npods=$(kubectl get pods --namespace openstack --no-headers \
|
|
||||||
--field-selector spec.nodeName=${this_node} 2>/dev/null | \
|
|
||||||
awk '
|
|
||||||
BEGIN { n=0; }
|
|
||||||
!/Completed|Running/ { n+=1 }
|
|
||||||
END { printf "%d\n", n; }
|
|
||||||
')
|
|
||||||
if [ ${npods} -gt 0 ]; then
|
|
||||||
NOT_READY_REASON="${npods} openstack pods not recovered"
|
|
||||||
STABLE=0
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Evaluate elapsed time since check criteria pass
|
|
||||||
if [ ${STABLE} -eq 0 ]; then
|
|
||||||
STABLE=${SECONDS}
|
|
||||||
fi
|
|
||||||
dt=$(( ${SECONDS} - ${STABLE} ))
|
|
||||||
if [ ${dt} -lt ${STABILIZATION_SECONDS} ]; then
|
|
||||||
NOT_READY_REASON="stabilization wait"
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
|
|
||||||
LOG "K8S is ready"
|
|
||||||
return ${PASS}
|
|
||||||
}
|
|
||||||
|
|
||||||
# Get number of DRBD resources started.
|
|
||||||
# Returns 0 if DRBD not ready.
|
|
||||||
function number_drbd_resources_started {
|
|
||||||
local started
|
|
||||||
|
|
||||||
# Number of started DRBD resources
|
|
||||||
started=$(cat /proc/drbd 2>/dev/null | \
|
|
||||||
awk '/cs:/ { n+=1; } END {printf "%d\n", n}')
|
|
||||||
echo "${started}"
|
|
||||||
}
|
|
||||||
|
|
||||||
# Check criteria for all drbd resources started.
|
|
||||||
# i.e., see running DRBD worker threads for each configured resource.
|
|
||||||
function all_drbd_resources_started {
|
|
||||||
local PASS=0
|
|
||||||
local FAIL=1
|
|
||||||
local -i started=0
|
|
||||||
local -i resources=0
|
|
||||||
|
|
||||||
# Global variable
|
|
||||||
NOT_READY_REASON=""
|
|
||||||
|
|
||||||
# Number of started DRBD resources
|
|
||||||
started=$(number_drbd_resources_started)
|
|
||||||
if [ ${started} -eq 0 ]; then
|
|
||||||
NOT_READY_REASON="no drbd resources started"
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Number of expected DRBD resources
|
|
||||||
resources=$(drbdadm sh-resources | \
|
|
||||||
awk -vFS='[[:space:]]' 'END {print NF}')
|
|
||||||
if [ ${started} -ne ${resources} ]; then
|
|
||||||
NOT_READY_REASON="${started} of ${resources} drbd resources started"
|
|
||||||
return ${FAIL}
|
|
||||||
fi
|
|
||||||
|
|
||||||
return ${PASS}
|
|
||||||
}
|
|
||||||
|
|
||||||
function affine_drbd_tasks {
|
|
||||||
local CPUS=$1
|
|
||||||
local pidlist
|
|
||||||
|
|
||||||
LOG "Affine drbd tasks, CPUS=${CPUS}"
|
|
||||||
|
|
||||||
# Affine drbd_r_* threads to all cores. The DRBD receiver threads are
|
|
||||||
# particularly CPU intensive. Leave the other DRBD threads alone.
|
|
||||||
pidlist=$(pgrep drbd_r_)
|
|
||||||
for pid in ${pidlist[@]}; do
|
|
||||||
taskset --pid --cpu-list ${CPUS} ${pid} > /dev/null 2>&1
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
# Return list of reaffineable pids. This includes all processes, but excludes
|
|
||||||
# kernel threads, vSwitch, and anything in K8S or qemu/kvm.
|
|
||||||
function reaffineable_pids {
|
|
||||||
local pids_excl
|
|
||||||
local pidlist
|
|
||||||
|
|
||||||
pids_excl=$(ps -eL -o pid=,comm= | \
|
|
||||||
awk -vORS=',' '/eal-intr-thread|kthreadd/ {print $1}' | \
|
|
||||||
sed 's/,$/\n/')
|
|
||||||
pidlist=$(ps --ppid ${pids_excl} -p ${pids_excl} --deselect \
|
|
||||||
-o pid=,cgroup= | \
|
|
||||||
awk '!/k8s-infra|machine.slice/ {print $1; }')
|
|
||||||
echo "${pidlist[@]}"
|
|
||||||
}
|
|
||||||
|
|
||||||
function affine_tasks_to_all_cores {
|
|
||||||
local pidlist
|
|
||||||
local count=0
|
|
||||||
|
|
||||||
LOG "Affine all tasks, CPUS: ${NONISOL_CPUS};" \
|
|
||||||
"online=${ONLINE_CPUS} (0x${ONLINE_MASK})," \
|
|
||||||
"isol=${ISOL_CPUS}, nonisol=${NONISOL_CPUS} (0x${NONISOL_MASK})"
|
|
||||||
|
|
||||||
pidlist=( $(reaffineable_pids) )
|
|
||||||
for pid in ${pidlist[@]}; do
|
|
||||||
count=$((${count} + 1))
|
|
||||||
taskset --all-tasks --pid --cpu-list \
|
|
||||||
${NONISOL_CPUS} ${pid} > /dev/null 2>&1
|
|
||||||
done
|
|
||||||
|
|
||||||
LOG "Affined ${count} processes to all cores."
|
|
||||||
}
|
|
||||||
|
|
||||||
function affine_tasks_to_platform_cores {
|
|
||||||
local pidlist
|
|
||||||
local count=0
|
|
||||||
|
|
||||||
LOG "Affine all tasks, PLATFORM_CPUS=${PLATFORM_CPUS}"
|
|
||||||
|
|
||||||
pidlist=( $(reaffineable_pids) )
|
|
||||||
for pid in ${pidlist[@]}; do
|
|
||||||
pid_mask=$(taskset -p $pid 2> /dev/null | awk '{print $6}')
|
|
||||||
if [ "${pid_mask}" == "${NONISOL_MASK}" ]; then
|
|
||||||
count=$((${count} + 1))
|
|
||||||
taskset --all-tasks --pid --cpu-list \
|
|
||||||
${PLATFORM_CPUS} ${pid} > /dev/null 2>&1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
# Reaffine vSwitch tasks that span multiple cpus to platform cpus
|
|
||||||
pidlist=$(ps -eL -o pid=,comm= | awk '/eal-intr-thread/ {print $1}')
|
|
||||||
for pid in ${pidlist[@]}; do
|
|
||||||
count=$((${count} + 1))
|
|
||||||
grep Cpus_allowed_list /proc/${pid}/task/*/status 2>/dev/null | \
|
|
||||||
sed 's#/# #g' | awk '/,|-/ {print $4}' | \
|
|
||||||
xargs --no-run-if-empty -i{} \
|
|
||||||
taskset --pid --cpu-list ${PLATFORM_CPUS} {} > /dev/null 2>&1
|
|
||||||
done
|
|
||||||
|
|
||||||
# Reaffine drbd_r_* threads to platform cpus
|
|
||||||
affine_drbd_tasks ${PLATFORM_CPUS}
|
|
||||||
|
|
||||||
LOG "Affined ${count} processes to platform cores."
|
|
||||||
}
|
|
||||||
|
|
||||||
function start {
|
|
||||||
# Ensure this only runs on AIO
|
|
||||||
if ! { [[ "$nodetype" = "controller" ]] && [[ $subfunction = *worker* ]]; }
|
|
||||||
then
|
|
||||||
LOG "Not AIO, nothing to do."
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Abort if another instantiation is already running
|
|
||||||
if [ -e ${PIDFILE} ]; then
|
|
||||||
PID=$(cat ${PIDFILE})
|
|
||||||
if [ -n "${PID}" -a -e /proc/${PID} ]; then
|
|
||||||
ERROR "Aborting, ${PID} already running: ${PIDFILE}."
|
|
||||||
exit 1
|
|
||||||
else
|
|
||||||
OUT=$(rm -v -f ${PIDFILE})
|
|
||||||
LOG "${OUT}"
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
LOG "Starting."
|
|
||||||
|
|
||||||
# Create pidfile to indicate the script is running
|
|
||||||
echo $$ > ${PIDFILE}
|
|
||||||
|
|
||||||
# Affine all tasks to float on all cores
|
|
||||||
affine_tasks_to_all_cores
|
|
||||||
|
|
||||||
# Wait for kubelet to be running
|
|
||||||
t0=${SECONDS}
|
|
||||||
until is_k8s_platform_ready; do
|
|
||||||
dt=$(( ${SECONDS} - ${t0} ))
|
|
||||||
if [ ${dt} -ge ${PRINT_INTERVAL_SECONDS} ]; then
|
|
||||||
t0=${SECONDS}
|
|
||||||
LOG "Recovery wait, elapsed ${SECONDS} seconds." \
|
|
||||||
"Reason: ${NOT_READY_REASON}"
|
|
||||||
fi
|
|
||||||
sleep ${INIT_INTERVAL_SECONDS}
|
|
||||||
done
|
|
||||||
|
|
||||||
# Update K8S cpuset so that pods float on all cpus
|
|
||||||
# NOTE: dynamic cpuset changes incompatible with static policy
|
|
||||||
if ! is_static_cpu_manager_policy; then
|
|
||||||
update_cgroup_cpuset_k8s_infra_all
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Wait for all DRBD resources to have started. Affine DRBD tasks
|
|
||||||
# to float on all cores as we find them.
|
|
||||||
until all_drbd_resources_started; do
|
|
||||||
started=$(number_drbd_resources_started)
|
|
||||||
if [ ${started} -gt 0 ]; then
|
|
||||||
affine_drbd_tasks ${NONISOL_CPUS}
|
|
||||||
fi
|
|
||||||
dt=$(( ${SECONDS} - ${t0} ))
|
|
||||||
if [ ${dt} -ge ${PRINT_INTERVAL_SECONDS} ]; then
|
|
||||||
t0=${SECONDS}
|
|
||||||
LOG "Recovery wait, elapsed ${SECONDS} seconds." \
|
|
||||||
"Reason: ${NOT_READY_REASON}"
|
|
||||||
fi
|
|
||||||
sleep ${INIT_INTERVAL_SECONDS}
|
|
||||||
done
|
|
||||||
affine_drbd_tasks ${NONISOL_CPUS}
|
|
||||||
|
|
||||||
# Wait until K8s pods have recovered and nova-compute is running
|
|
||||||
t0=${SECONDS}
|
|
||||||
until is_k8s_platform_steady_state_ready; do
|
|
||||||
dt=$(( ${SECONDS} - ${t0} ))
|
|
||||||
if [ ${dt} -ge ${PRINT_INTERVAL_SECONDS} ]; then
|
|
||||||
t0=${SECONDS}
|
|
||||||
LOG "Recovery wait, elapsed ${SECONDS} seconds." \
|
|
||||||
"Reason: ${NOT_READY_REASON}"
|
|
||||||
fi
|
|
||||||
sleep ${CHECK_INTERVAL_SECONDS}
|
|
||||||
done
|
|
||||||
|
|
||||||
# Update K8S cpuset to platform cores
|
|
||||||
if ! is_static_cpu_manager_policy; then
|
|
||||||
update_cgroup_cpuset_k8s_infra_platform
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Affine all floating tasks back to platform cores
|
|
||||||
affine_tasks_to_platform_cores
|
|
||||||
|
|
||||||
# Remove pidfile after successful completion
|
|
||||||
rm -f ${PIDFILE}
|
|
||||||
|
|
||||||
LOG "Complete."
|
|
||||||
}
|
|
||||||
|
|
||||||
function stop {
|
|
||||||
LOG "Stopping."
|
|
||||||
|
|
||||||
# Forcibly stop any running instantiation
|
|
||||||
if [ -e ${PIDFILE} ]; then
|
|
||||||
PID=$(cat ${PIDFILE})
|
|
||||||
if [ -n "${PID}" -a -e /proc/${PID} ]; then
|
|
||||||
LOG "Stopping ${PID}: ${PIDFILE}."
|
|
||||||
kill -9 ${PID}
|
|
||||||
timeout 20 tail --pid=${PID} -f /dev/null
|
|
||||||
fi
|
|
||||||
OUT=$(rm -v -f ${PIDFILE})
|
|
||||||
LOG "${OUT}"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
function status {
|
|
||||||
:
|
|
||||||
}
|
|
||||||
|
|
||||||
function reset {
|
|
||||||
:
|
|
||||||
}
|
|
||||||
|
|
||||||
if [ ${UID} -ne 0 ]; then
|
|
||||||
ERROR "Need sudo/root permission."
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
case "$1" in
|
|
||||||
start)
|
|
||||||
start
|
|
||||||
;;
|
|
||||||
stop)
|
|
||||||
stop
|
|
||||||
;;
|
|
||||||
restart|force-reload|reload)
|
|
||||||
stop
|
|
||||||
start
|
|
||||||
;;
|
|
||||||
status)
|
|
||||||
status
|
|
||||||
;;
|
|
||||||
reset)
|
|
||||||
reset
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
echo "Usage: $0 {start|stop|force-reload|restart|reload|status|reset}"
|
|
||||||
exit 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
exit 0
|
|
@ -1,403 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
################################################################################
|
|
||||||
# Copyright (c) 2013-2018 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
#
|
|
||||||
### BEGIN INIT INFO
|
|
||||||
# Provides: cpumap_functions
|
|
||||||
# Required-Start:
|
|
||||||
# Required-Stop:
|
|
||||||
# Default-Start: 2 3 4 5
|
|
||||||
# Default-Stop: 0 1 6
|
|
||||||
# Short-Description: cpumap_functions
|
|
||||||
### END INIT INFO
|
|
||||||
|
|
||||||
source /etc/platform/platform.conf
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Utility function to expand a sequence of numbers (e.g., 0-7,16-23)
|
|
||||||
################################################################################
|
|
||||||
function expand_sequence {
|
|
||||||
SEQUENCE=(${1//,/ })
|
|
||||||
DELIMITER=${2:-","}
|
|
||||||
|
|
||||||
LIST=
|
|
||||||
for entry in ${SEQUENCE[@]}; do
|
|
||||||
range=(${entry/-/ })
|
|
||||||
a=${range[0]}
|
|
||||||
b=${range[1]:-${range[0]}}
|
|
||||||
|
|
||||||
for i in $(seq $a $b); do
|
|
||||||
LIST="${LIST}${DELIMITER}${i}"
|
|
||||||
done
|
|
||||||
done
|
|
||||||
echo ${LIST:1}
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Append a string to comma separated list string
|
|
||||||
################################################################################
|
|
||||||
function append_list {
|
|
||||||
local PUSH=${1-}
|
|
||||||
local LIST=${2-}
|
|
||||||
if [ -z "${LIST}" ]; then
|
|
||||||
LIST=${PUSH}
|
|
||||||
else
|
|
||||||
LIST="${LIST},${PUSH}"
|
|
||||||
fi
|
|
||||||
echo ${LIST}
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Condense a sequence of numbers to a list of ranges (e.g, 7-12,15-16)
|
|
||||||
################################################################################
|
|
||||||
function condense_sequence {
|
|
||||||
local arr=( $(printf '%s\n' "$@" | sort -n) )
|
|
||||||
local first
|
|
||||||
local last
|
|
||||||
local cpulist=""
|
|
||||||
for ((i=0; i < ${#arr[@]}; i++)); do
|
|
||||||
num=${arr[$i]}
|
|
||||||
if [[ -z $first ]]; then
|
|
||||||
first=$num
|
|
||||||
last=$num
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
if [[ num -ne $((last + 1)) ]]; then
|
|
||||||
if [[ first -eq last ]]; then
|
|
||||||
cpulist=$(append_list ${first} ${cpulist})
|
|
||||||
else
|
|
||||||
cpulist=$(append_list "${first}-${last}" ${cpulist})
|
|
||||||
fi
|
|
||||||
first=$num
|
|
||||||
last=$num
|
|
||||||
else
|
|
||||||
: $((last++))
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
if [[ first -eq last ]]; then
|
|
||||||
cpulist=$(append_list ${first} ${cpulist})
|
|
||||||
else
|
|
||||||
cpulist=$(append_list "${first}-${last}" ${cpulist})
|
|
||||||
fi
|
|
||||||
echo "$cpulist"
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Converts a CPULIST (e.g., 0-7,16-23) to a CPUMAP (e.g., 0x00FF00FF). The
|
|
||||||
# CPU map is returned as a string representation of a large hexidecimal
|
|
||||||
# number but without the leading "0x" characters.
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
function cpulist_to_cpumap {
|
|
||||||
local CPULIST=$1
|
|
||||||
local NR_CPUS=$2
|
|
||||||
local CPUMAP=0
|
|
||||||
local CPUID=0
|
|
||||||
if [ -z "${NR_CPUS}" ] || [ ${NR_CPUS} -eq 0 ]; then
|
|
||||||
echo 0
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
for CPUID in $(expand_sequence $CPULIST " "); do
|
|
||||||
if [ "${CPUID}" -lt "${NR_CPUS}" ]; then
|
|
||||||
CPUMAP=$(echo "${CPUMAP} + (2^${CPUID})" | bc -l)
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
echo "obase=16;ibase=10;${CPUMAP}" | bc -l
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Converts a CPUMAP (e.g., 0x00FF00FF) to a CPULIST (e.g., 0-7,16-23). The
|
|
||||||
# CPUMAP is expected in hexidecimal (base=10) form without the leading "0x"
|
|
||||||
# characters.
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
function cpumap_to_cpulist {
|
|
||||||
local CPUMAP
|
|
||||||
CPUMAP=$(echo "obase=10;ibase=16;$1" | bc -l)
|
|
||||||
local NR_CPUS=$2
|
|
||||||
local list=()
|
|
||||||
local cpulist=""
|
|
||||||
for((i=0; i < NR_CPUS; i++))
|
|
||||||
do
|
|
||||||
## Since 'bc' does not support any bitwise operators this expression:
|
|
||||||
## if (CPUMAP & (1 << CPUID))
|
|
||||||
## has to be rewritten like this:
|
|
||||||
## if (CPUMAP % (2**(CPUID+1)) > ((2**(CPUID)) - 1))
|
|
||||||
##
|
|
||||||
ISSET=$(echo "scale=0; (${CPUMAP} % 2^(${i}+1)) > (2^${i})-1" | bc -l)
|
|
||||||
if [ "${ISSET}" -ne 0 ]; then
|
|
||||||
list+=($i)
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
cpulist=$(condense_sequence ${list[@]} )
|
|
||||||
echo "$cpulist"
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Bitwise NOT of a hexidecimal representation of a CPULIST. The value is
|
|
||||||
# returned as a hexidecimal value but without the leading "0x" characters
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
function invert_cpumap {
|
|
||||||
local CPUMAP
|
|
||||||
CPUMAP=$(echo "obase=10;ibase=16;$1" | bc -l)
|
|
||||||
local NR_CPUS=$2
|
|
||||||
local INVERSE_CPUMAP=0
|
|
||||||
|
|
||||||
for CPUID in $(seq 0 $((NR_CPUS - 1))); do
|
|
||||||
## See comment in previous function
|
|
||||||
ISSET=$(echo "scale=0; (${CPUMAP} % 2^(${CPUID}+1)) > (2^${CPUID})-1" | bc -l)
|
|
||||||
if [ "${ISSET}" -eq 1 ]; then
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
|
|
||||||
INVERSE_CPUMAP=$(echo "${INVERSE_CPUMAP} + (2^${CPUID})" | bc -l)
|
|
||||||
done
|
|
||||||
|
|
||||||
echo "obase=16;ibase=10;${INVERSE_CPUMAP}" | bc -l
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Builds the complement representation of a CPULIST
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
function invert_cpulist {
|
|
||||||
local CPULIST=$1
|
|
||||||
local NR_CPUS=$2
|
|
||||||
local CPUMAP
|
|
||||||
CPUMAP=$(cpulist_to_cpumap ${CPULIST} ${NR_CPUS})
|
|
||||||
cpumap_to_cpulist $(invert_cpumap ${CPUMAP} ${NR_CPUS}) ${NR_CPUS}
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# in_list() - check whether item is contained in list
|
|
||||||
# param: item
|
|
||||||
# param: list (i.e. 0-3,8-11)
|
|
||||||
# returns: 0 - item is contained in list;
|
|
||||||
# 1 - item is not contained in list
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
function in_list {
|
|
||||||
local item="${1-}"
|
|
||||||
local list="${2-}"
|
|
||||||
|
|
||||||
# expand list format 0-3,8-11 to a full sequence {0..3} {8..11}
|
|
||||||
local exp_list
|
|
||||||
exp_list=$(echo ${list} | \
|
|
||||||
sed -e 's#,# #g' -e 's#\([0-9]*\)-\([0-9]*\)#{\1\.\.\2}#g')
|
|
||||||
|
|
||||||
local e
|
|
||||||
for e in $(eval echo ${exp_list}); do
|
|
||||||
[[ "$e" == "$item" ]] && return 0
|
|
||||||
done
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# any_in_list() - check if any item of sublist is contained in list
|
|
||||||
# param: sublist
|
|
||||||
# param: list
|
|
||||||
# returns: 0 - an item of sublist is contained in list;
|
|
||||||
# 1 - no sublist items contained in list
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
function any_in_list {
|
|
||||||
local sublist="$1"
|
|
||||||
local list="$2"
|
|
||||||
local e
|
|
||||||
local exp_list
|
|
||||||
|
|
||||||
# expand list format 0-3,8-11 to a full sequence {0..3} {8..11}
|
|
||||||
exp_list=$(echo ${list} | \
|
|
||||||
sed -e 's#,# #g' -e 's#\([0-9]*\)-\([0-9]*\)#{\1\.\.\2}#g')
|
|
||||||
declare -A a_list
|
|
||||||
for e in $(eval echo ${exp_list}); do
|
|
||||||
a_list[$e]=1
|
|
||||||
done
|
|
||||||
|
|
||||||
# expand list format 0-3,8-11 to a full sequence {0..3} {8..11}
|
|
||||||
exp_list=$(echo ${sublist} | \
|
|
||||||
sed -e 's#,# #g' -e 's#\([0-9]*\)-\([0-9]*\)#{\1\.\.\2}#g')
|
|
||||||
declare -A a_sublist
|
|
||||||
for e in $(eval echo ${exp_list}); do
|
|
||||||
a_sublist[$e]=1
|
|
||||||
done
|
|
||||||
|
|
||||||
# Check if any element of sublist is in list
|
|
||||||
for e in "${!a_sublist[@]}"; do
|
|
||||||
if [[ "${a_list[$e]}" == 1 ]]; then
|
|
||||||
return 0 # matches
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
return 1 # no match
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Return list of CPUs reserved for platform
|
|
||||||
################################################################################
|
|
||||||
function get_platform_cpu_list {
|
|
||||||
## Define platform cpulist based on engineering a number of cores and
|
|
||||||
## whether this is a combo or not, and include SMT siblings.
|
|
||||||
if [[ $subfunction = *worker* ]]; then
|
|
||||||
RESERVE_CONF="/etc/platform/worker_reserved.conf"
|
|
||||||
[[ -e ${RESERVE_CONF} ]] && source ${RESERVE_CONF}
|
|
||||||
if [ -n "$PLATFORM_CPU_LIST" ];then
|
|
||||||
echo "$PLATFORM_CPU_LIST"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
local PLATFORM_SOCKET=0
|
|
||||||
local PLATFORM_START=0
|
|
||||||
local PLATFORM_CORES=1
|
|
||||||
if [ "$nodetype" = "controller" ]; then
|
|
||||||
PLATFORM_CORES=$(($PLATFORM_CORES+1))
|
|
||||||
fi
|
|
||||||
local PLATFORM_CPULIST
|
|
||||||
PLATFORM_CPULIST=$(topology_to_cpulist ${PLATFORM_SOCKET} ${PLATFORM_START} ${PLATFORM_CORES})
|
|
||||||
echo ${PLATFORM_CPULIST}
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Return number of CPUs reserved for platform
|
|
||||||
################################################################################
|
|
||||||
function get_platform_cpus {
|
|
||||||
local PLATFORM_CPULIST
|
|
||||||
PLATFORM_CPULIST=($(platform_expanded_cpu_list | \
|
|
||||||
perl -pe 's/(\d+)-(\d+)/join(",",$1..$2)/eg'| \
|
|
||||||
sed 's/,/ /g'))
|
|
||||||
echo ${#PLATFORM_CPULIST[@]}
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Return list of CPUs reserved for vswitch
|
|
||||||
################################################################################
|
|
||||||
function get_vswitch_cpu_list {
|
|
||||||
## Define default avp cpulist based on engineered number of platform cores,
|
|
||||||
## engineered avp cores, and include SMT siblings.
|
|
||||||
if [[ $subfunction = *worker* ]]; then
|
|
||||||
VSWITCH_CONF="/etc/vswitch/vswitch.conf"
|
|
||||||
[[ -e ${VSWITCH_CONF} ]] && source ${VSWITCH_CONF}
|
|
||||||
if [ -n "$VSWITCH_CPU_LIST" ];then
|
|
||||||
echo "$VSWITCH_CPU_LIST"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
local N_CORES_IN_PKG
|
|
||||||
N_CORES_IN_PKG=$(cat /proc/cpuinfo 2>/dev/null | \
|
|
||||||
awk '/^cpu cores/ {n = $4} END { print (n>0) ? n : 1 }')
|
|
||||||
# engineer platform cores
|
|
||||||
local PLATFORM_CORES=1
|
|
||||||
if [ "$nodetype" = "controller" ]; then
|
|
||||||
PLATFORM_CORES=$(($PLATFORM_CORES+1))
|
|
||||||
fi
|
|
||||||
|
|
||||||
# engineer AVP cores
|
|
||||||
local AVP_SOCKET=0
|
|
||||||
local AVP_START=${PLATFORM_CORES}
|
|
||||||
local AVP_CORES=1
|
|
||||||
if [ ${N_CORES_IN_PKG} -gt 4 ]; then
|
|
||||||
AVP_CORES=$(($AVP_CORES+1))
|
|
||||||
fi
|
|
||||||
local AVP_CPULIST
|
|
||||||
AVP_CPULIST=$(topology_to_cpulist ${AVP_SOCKET} ${AVP_START} ${AVP_CORES})
|
|
||||||
echo ${AVP_CPULIST}
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# vswitch_expanded_cpu_list() - compute the vswitch cpu list, including it's siblings
|
|
||||||
################################################################################
|
|
||||||
function vswitch_expanded_cpu_list {
|
|
||||||
list=$(get_vswitch_cpu_list)
|
|
||||||
|
|
||||||
# Expand vswitch cpulist
|
|
||||||
vswitch_cpulist=$(expand_sequence ${list} " ")
|
|
||||||
|
|
||||||
cpulist=""
|
|
||||||
for e in $vswitch_cpulist; do
|
|
||||||
# claim hyperthread siblings if SMT enabled
|
|
||||||
SIBLINGS_CPULIST=$(cat /sys/devices/system/cpu/cpu${e}/topology/thread_siblings_list 2>/dev/null)
|
|
||||||
siblings_cpulist=$(expand_sequence ${SIBLINGS_CPULIST} " ")
|
|
||||||
for s in $siblings_cpulist; do
|
|
||||||
in_list ${s} ${cpulist}
|
|
||||||
if [ $? -eq 1 ]; then
|
|
||||||
cpulist=$(append_list ${s} ${cpulist})
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
done
|
|
||||||
|
|
||||||
echo "$cpulist"
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# platform_expanded_cpu_list() - compute the platform cpu list, including it's siblings
|
|
||||||
################################################################################
|
|
||||||
function platform_expanded_cpu_list {
|
|
||||||
list=$(get_platform_cpu_list)
|
|
||||||
|
|
||||||
# Expand platform cpulist
|
|
||||||
platform_cpulist=$(expand_sequence ${list} " ")
|
|
||||||
|
|
||||||
cpulist=""
|
|
||||||
for e in $platform_cpulist; do
|
|
||||||
# claim hyperthread siblings if SMT enabled
|
|
||||||
SIBLINGS_CPULIST=$(cat /sys/devices/system/cpu/cpu${e}/topology/thread_siblings_list 2>/dev/null)
|
|
||||||
siblings_cpulist=$(expand_sequence ${SIBLINGS_CPULIST} " ")
|
|
||||||
for s in $siblings_cpulist; do
|
|
||||||
in_list ${s} ${cpulist}
|
|
||||||
if [ $? -eq 1 ]; then
|
|
||||||
cpulist=$(append_list ${s} ${cpulist})
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
done
|
|
||||||
|
|
||||||
echo "$cpulist"
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Return list of CPUs based on cpu topology. Select the socket, starting core
|
|
||||||
# within the socket, select number of cores, and SMT siblings.
|
|
||||||
################################################################################
|
|
||||||
function topology_to_cpulist {
|
|
||||||
local SOCKET=$1
|
|
||||||
local CORE_START=$2
|
|
||||||
local NUM_CORES=$3
|
|
||||||
local CPULIST
|
|
||||||
CPULIST=$(cat /proc/cpuinfo 2>/dev/null | perl -sne \
|
|
||||||
'BEGIN { %T = {}; %H = {}; $L = $P = $C = $S = 0; }
|
|
||||||
{
|
|
||||||
if (/processor\s+:\s+(\d+)/) { $L = $1; }
|
|
||||||
if (/physical id\s+:\s+(\d+)/) { $P = $1; }
|
|
||||||
if (/core id\s+:\s+(\d+)/) {
|
|
||||||
$C = $1;
|
|
||||||
$T{$P}{$C}++;
|
|
||||||
$S = $T{$P}{$C};
|
|
||||||
$H{$P}{$C}{$S} = $L;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
END {
|
|
||||||
@cores = sort { $a <=> $b } keys $T{$socket};
|
|
||||||
@sel_cores = splice @cores, $core_start, $num_cores;
|
|
||||||
@lcpus = ();
|
|
||||||
for $C (@sel_cores) {
|
|
||||||
for $S (sort {$a <=> $b } keys %{ $H{$socket}{$C} }) {
|
|
||||||
push @lcpus, $H{$socket}{$C}{$S};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
printf "%s\n", join(",", @lcpus);
|
|
||||||
}' -- -socket=${SOCKET} -core_start=${CORE_START} -num_cores=${NUM_CORES})
|
|
||||||
echo ${CPULIST}
|
|
||||||
}
|
|
@ -1,241 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
|
|
||||||
#
|
|
||||||
# Copyright (c) 2015-2016 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
|
|
||||||
source /etc/init.d/cpumap_functions.sh
|
|
||||||
|
|
||||||
export NR_CPUS_LIST=("4" "8" "16" "32" "64" "128")
|
|
||||||
if [ ! -z ${1} ]; then
|
|
||||||
NR_CPUS_LIST=(${1//,/ })
|
|
||||||
fi
|
|
||||||
|
|
||||||
function test_cpumap_to_cpulist {
|
|
||||||
local NR_CPUS=$1
|
|
||||||
declare -A CPULISTS
|
|
||||||
|
|
||||||
if [ ${NR_CPUS} -ge 4 ]; then
|
|
||||||
CPULISTS["0"]=""
|
|
||||||
CPULISTS["1"]="0"
|
|
||||||
CPULISTS["2"]="1"
|
|
||||||
CPULISTS["3"]="0-1"
|
|
||||||
CPULISTS["5"]="0,2"
|
|
||||||
CPULISTS["7"]="0-2"
|
|
||||||
CPULISTS["F"]="0-3"
|
|
||||||
CPULISTS["9"]="0,3"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 8 ]; then
|
|
||||||
CPULISTS["00"]=""
|
|
||||||
CPULISTS["11"]="0,4"
|
|
||||||
CPULISTS["FF"]="0-7"
|
|
||||||
CPULISTS["81"]="0,7"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 16 ]; then
|
|
||||||
CPULISTS["0000"]=""
|
|
||||||
CPULISTS["1111"]="0,4,8,12"
|
|
||||||
CPULISTS["FFF"]="0-11"
|
|
||||||
CPULISTS["F0F"]="0-3,8-11"
|
|
||||||
CPULISTS["F0F0"]="4-7,12-15"
|
|
||||||
CPULISTS["FFFF"]="0-15"
|
|
||||||
CPULISTS["FFFE"]="1-15"
|
|
||||||
CPULISTS["8001"]="0,15"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 32 ]; then
|
|
||||||
CPULISTS["00000000"]=""
|
|
||||||
CPULISTS["11111111"]="0,4,8,12,16,20,24,28"
|
|
||||||
CPULISTS["0F0F0F0F"]="0-3,8-11,16-19,24-27"
|
|
||||||
CPULISTS["F0F0F0F0"]="4-7,12-15,20-23,28-31"
|
|
||||||
CPULISTS["FFFFFFFF"]="0-31"
|
|
||||||
CPULISTS["FFFFFFFE"]="1-31"
|
|
||||||
CPULISTS["80000001"]="0,31"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 64 ]; then
|
|
||||||
CPULISTS["0000000000000000"]=""
|
|
||||||
CPULISTS["1111111111111111"]="0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60"
|
|
||||||
CPULISTS["0F0F0F0F0F0F0F0F"]="0-3,8-11,16-19,24-27,32-35,40-43,48-51,56-59"
|
|
||||||
CPULISTS["F0F0F0F0F0F0F0F0"]="4-7,12-15,20-23,28-31,36-39,44-47,52-55,60-63"
|
|
||||||
CPULISTS["FFFFFFFFFFFFFFFF"]="0-63"
|
|
||||||
CPULISTS["FFFFFFFFFFFFFFFE"]="1-63"
|
|
||||||
CPULISTS["8000000000000001"]="0,63"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 128 ]; then
|
|
||||||
CPULISTS["00000000000000000000000000000000"]=""
|
|
||||||
CPULISTS["11111111111111111111111111111111"]="0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60,64,68,72,76,80,84,88,92,96,100,104,108,112,116,120,124"
|
|
||||||
CPULISTS["0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F"]="0-3,8-11,16-19,24-27,32-35,40-43,48-51,56-59,64-67,72-75,80-83,88-91,96-99,104-107,112-115,120-123"
|
|
||||||
CPULISTS["F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0"]="4-7,12-15,20-23,28-31,36-39,44-47,52-55,60-63,68-71,76-79,84-87,92-95,100-103,108-111,116-119,124-127"
|
|
||||||
CPULISTS["FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF"]="0-127"
|
|
||||||
CPULISTS["FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE"]="1-127"
|
|
||||||
CPULISTS["80000000000000000000000000000001"]="0,127"
|
|
||||||
fi
|
|
||||||
|
|
||||||
for CPUMAP in ${!CPULISTS[@]}; do
|
|
||||||
EXPECTED=${CPULISTS[${CPUMAP}]}
|
|
||||||
CPULIST=$(cpumap_to_cpulist ${CPUMAP} ${NR_CPUS})
|
|
||||||
if [ "${CPULIST}" != "${EXPECTED}" ]; then
|
|
||||||
printf "\n"
|
|
||||||
echo "error: (cpumap_to_list ${CPUMAP} ${NR_CPUS}) returned \"${CPULIST}\" instead of \"${EXPECTED}\""
|
|
||||||
fi
|
|
||||||
printf "."
|
|
||||||
done
|
|
||||||
|
|
||||||
printf "\n"
|
|
||||||
}
|
|
||||||
|
|
||||||
function test_cpulist_to_cpumap {
|
|
||||||
local NR_CPUS=$1
|
|
||||||
declare -A CPUMAPS
|
|
||||||
|
|
||||||
if [ ${NR_CPUS} -ge 4 ]; then
|
|
||||||
CPUMAPS[" "]="0"
|
|
||||||
CPUMAPS["0"]="1"
|
|
||||||
CPUMAPS["1"]="2"
|
|
||||||
CPUMAPS["0-1"]="3"
|
|
||||||
CPUMAPS["0,2"]="5"
|
|
||||||
CPUMAPS["0-2"]="7"
|
|
||||||
CPUMAPS["0-3"]="F"
|
|
||||||
CPUMAPS["0,3"]="9"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 8 ]; then
|
|
||||||
CPUMAPS["0,4"]="11"
|
|
||||||
CPUMAPS["0-7"]="FF"
|
|
||||||
CPUMAPS["0,7"]="81"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 16 ]; then
|
|
||||||
CPUMAPS["0,4,8,12"]="1111"
|
|
||||||
CPUMAPS["0-11"]="FFF"
|
|
||||||
CPUMAPS["0-3,8-11"]="F0F"
|
|
||||||
CPUMAPS["4-7,12-15"]="F0F0"
|
|
||||||
CPUMAPS["0-15"]="FFFF"
|
|
||||||
CPUMAPS["1-15"]="FFFE"
|
|
||||||
CPUMAPS["0,15"]="8001"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 32 ]; then
|
|
||||||
CPUMAPS["0,4,8,12,16,20,24,28"]="11111111"
|
|
||||||
CPUMAPS["0-3,8-11,16-19,24-27"]="F0F0F0F"
|
|
||||||
CPUMAPS["4-7,12-15,20-23,28-31"]="F0F0F0F0"
|
|
||||||
CPUMAPS["0-31"]="FFFFFFFF"
|
|
||||||
CPUMAPS["1-31"]="FFFFFFFE"
|
|
||||||
CPUMAPS["0,31"]="80000001"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 64 ]; then
|
|
||||||
CPUMAPS["0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60"]="1111111111111111"
|
|
||||||
CPUMAPS["0-3,8-11,16-19,24-27,32-35,40-43,48-51,56-59"]="F0F0F0F0F0F0F0F"
|
|
||||||
CPUMAPS["4-7,12-15,20-23,28-31,36-39,44-47,52-55,60-63"]="F0F0F0F0F0F0F0F0"
|
|
||||||
CPUMAPS["0-63"]="FFFFFFFFFFFFFFFF"
|
|
||||||
CPUMAPS["1-63"]="FFFFFFFFFFFFFFFE"
|
|
||||||
CPUMAPS["0,63"]="8000000000000001"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 128 ]; then
|
|
||||||
CPUMAPS["0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60,64,68,72,76,80,84,88,92,96,100,104,108,112,116,120,124"]="11111111111111111111111111111111"
|
|
||||||
CPUMAPS["0-3,8-11,16-19,24-27,32-35,40-43,48-51,56-59,64-67,72-75,80-83,88-91,96-99,104-107,112-115,120-123"]="F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F"
|
|
||||||
CPUMAPS["4-7,12-15,20-23,28-31,36-39,44-47,52-55,60-63,68-71,76-79,84-87,92-95,100-103,108-111,116-119,124-127"]="F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0"
|
|
||||||
CPUMAPS["0-127"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF"
|
|
||||||
CPUMAPS["1-127"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE"
|
|
||||||
CPUMAPS["0,127"]="80000000000000000000000000000001"
|
|
||||||
fi
|
|
||||||
|
|
||||||
for CPULIST in ${!CPUMAPS[@]}; do
|
|
||||||
EXPECTED=${CPUMAPS[${CPULIST}]}
|
|
||||||
CPUMAP=$(cpulist_to_cpumap ${CPULIST} ${NR_CPUS})
|
|
||||||
if [ "${CPUMAP}" != "${EXPECTED}" ]; then
|
|
||||||
printf "\n"
|
|
||||||
echo "error: (cpulist_to_cpumap ${CPULIST} ${NR_CPUS}) returned \"${CPUMAP}\" instead of \"${EXPECTED}\""
|
|
||||||
fi
|
|
||||||
printf "."
|
|
||||||
done
|
|
||||||
|
|
||||||
printf "\n"
|
|
||||||
}
|
|
||||||
|
|
||||||
function test_invert_cpumap {
|
|
||||||
local NR_CPUS=$1
|
|
||||||
declare -A INVERSES
|
|
||||||
|
|
||||||
if [ $((${NR_CPUS} % 4)) -ne 0 ]; then
|
|
||||||
echo "test_invert_cpumap skipping NR_CPUS=${NR_CPUS}; not a multiple of 4"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ ${NR_CPUS} -ge 4 ]; then
|
|
||||||
INVERSES["0"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF"
|
|
||||||
INVERSES["1"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE"
|
|
||||||
INVERSES["2"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFD"
|
|
||||||
INVERSES["3"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFC"
|
|
||||||
INVERSES["5"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFA"
|
|
||||||
INVERSES["7"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF8"
|
|
||||||
INVERSES["F"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0"
|
|
||||||
INVERSES["9"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF6"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 8 ]; then
|
|
||||||
INVERSES["11"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEE"
|
|
||||||
INVERSES["FF"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00"
|
|
||||||
INVERSES["F0"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0F"
|
|
||||||
INVERSES["81"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF7E"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 16 ]; then
|
|
||||||
INVERSES["1111"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFEEEE"
|
|
||||||
INVERSES["FFF"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFF000"
|
|
||||||
INVERSES["F0F"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFF0F0"
|
|
||||||
INVERSES["F0F0"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFF0F0F"
|
|
||||||
INVERSES["0F0F"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFFF0F0"
|
|
||||||
INVERSES["FFFF"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFF0000"
|
|
||||||
INVERSES["FFFE"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFF0001"
|
|
||||||
INVERSES["8001"]="FFFFFFFFFFFFFFFFFFFFFFFFFFFF7FFE"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 32 ]; then
|
|
||||||
INVERSES["11111111"]="FFFFFFFFFFFFFFFFFFFFFFFFEEEEEEEE"
|
|
||||||
INVERSES["0F0F0F0F"]="FFFFFFFFFFFFFFFFFFFFFFFFF0F0F0F0"
|
|
||||||
INVERSES["F0F0F0F0"]="FFFFFFFFFFFFFFFFFFFFFFFF0F0F0F0F"
|
|
||||||
INVERSES["FFFFFFFF"]="FFFFFFFFFFFFFFFFFFFFFFFF00000000"
|
|
||||||
INVERSES["FFFFFFFE"]="FFFFFFFFFFFFFFFFFFFFFFFF00000001"
|
|
||||||
INVERSES["80000001"]="FFFFFFFFFFFFFFFFFFFFFFFF7FFFFFFE"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 64 ]; then
|
|
||||||
INVERSES["1111111111111111"]="FFFFFFFFFFFFFFFFEEEEEEEEEEEEEEEE"
|
|
||||||
INVERSES["0F0F0F0F0F0F0F0F"]="FFFFFFFFFFFFFFFFF0F0F0F0F0F0F0F0"
|
|
||||||
INVERSES["F0F0F0F0F0F0F0F0"]="FFFFFFFFFFFFFFFF0F0F0F0F0F0F0F0F"
|
|
||||||
INVERSES["FFFFFFFFFFFFFFFF"]="FFFFFFFFFFFFFFFF0000000000000000"
|
|
||||||
INVERSES["FFFFFFFFFFFFFFFE"]="FFFFFFFFFFFFFFFF0000000000000001"
|
|
||||||
INVERSES["8000000000000001"]="FFFFFFFFFFFFFFFF7FFFFFFFFFFFFFFE"
|
|
||||||
fi
|
|
||||||
if [ ${NR_CPUS} -ge 128 ]; then
|
|
||||||
INVERSES["11111111111111111111111111111111"]="EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE"
|
|
||||||
INVERSES["0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F"]="F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0"
|
|
||||||
INVERSES["F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0"]="0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F"
|
|
||||||
INVERSES["FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF"]="00000000000000000000000000000000"
|
|
||||||
INVERSES["FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE"]="00000000000000000000000000000001"
|
|
||||||
INVERSES["80000000000000000000000000000001"]="7FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE"
|
|
||||||
fi
|
|
||||||
|
|
||||||
for CPUMAP in ${!INVERSES[@]}; do
|
|
||||||
EXPECTED=${INVERSES[${CPUMAP}]}
|
|
||||||
if [ ${NR_CPUS} -lt 128 ]; then
|
|
||||||
EXPECTED=$(echo ${EXPECTED} | cut --complement -c1-$((32-((${NR_CPUS}+3)/4))))
|
|
||||||
fi
|
|
||||||
EXPECTED=$(echo ${EXPECTED} | sed -e "s/^0*//")
|
|
||||||
if [ -z ${EXPECTED} ]; then
|
|
||||||
EXPECTED="0"
|
|
||||||
fi
|
|
||||||
INVERSE=$(invert_cpumap ${CPUMAP} ${NR_CPUS})
|
|
||||||
if [ "${INVERSE}" != "${EXPECTED}" ]; then
|
|
||||||
printf "\n"
|
|
||||||
echo "error: (invert_cpumap ${CPUMAP} ${NR_CPUS}) returned \"${INVERSE}\" instead of \"${EXPECTED}\""
|
|
||||||
fi
|
|
||||||
printf "."
|
|
||||||
done
|
|
||||||
|
|
||||||
printf "\n"
|
|
||||||
}
|
|
||||||
|
|
||||||
for NR_CPUS in ${NR_CPUS_LIST[@]}; do
|
|
||||||
echo "NR_CPUS=${NR_CPUS}"
|
|
||||||
test_cpumap_to_cpulist ${NR_CPUS}
|
|
||||||
test_cpulist_to_cpumap ${NR_CPUS}
|
|
||||||
test_invert_cpumap ${NR_CPUS}
|
|
||||||
echo ""
|
|
||||||
done
|
|
||||||
|
|
||||||
exit 0
|
|
@ -1,26 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
################################################################################
|
|
||||||
# Copyright (c) 2013 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
#
|
|
||||||
# ps-sched.sh -- gives detailed task listing with scheduling attributes
|
|
||||||
# -- this is cpu and scheduling intensive version (shell/taskset based)
|
|
||||||
# (note: does not print fields 'group' or 'timeslice')
|
|
||||||
|
|
||||||
printf "%6s %6s %6s %1c %2s %4s %6s %4s %-24s %2s %-16s %s\n" "PID" "TID" "PPID" "S" "PO" "NICE" "RTPRIO" "PR" "AFFINITY" "P" "COMM" "COMMAND"
|
|
||||||
ps -eL -o pid=,lwp=,ppid=,state=,class=,nice=,rtprio=,priority=,psr=,comm=,command= | \
|
|
||||||
while read pid tid ppid state policy nice rtprio priority psr comm command; do
|
|
||||||
bitmask=$(taskset -p $tid 2>/dev/null)
|
|
||||||
aff=${bitmask##*: }
|
|
||||||
if [ -z "${aff}" ]; then
|
|
||||||
aff="0x0"
|
|
||||||
else
|
|
||||||
aff="0x${aff}"
|
|
||||||
fi
|
|
||||||
printf "%6d %6d %6d %1c %2s %4s %6s %4d %-24s %2d %-16s %s\n" $pid $tid $ppid $state $policy $nice $rtprio $priority $aff $psr $comm "$command"
|
|
||||||
done
|
|
||||||
|
|
||||||
exit 0
|
|
@ -1,89 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
|
|
||||||
#
|
|
||||||
# Copyright (c) 2017 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
# Purpose: set PM QoS resume latency constraints for CPUs.
|
|
||||||
# Usage: /usr/bin/set-cpu-wakeup-latency.sh policy cpulist
|
|
||||||
# policy may be either "low" or "high" to set appropriate latency.
|
|
||||||
# "low" means HALT (C1) is the deepest C-state we allow the CPU to enter.
|
|
||||||
# "high" means we allow the CPU to sleep as deeply as possible.
|
|
||||||
# cpulist is for specifying a numerical list of processors.
|
|
||||||
# It may contain multiple items, separated by comma, and ranges.
|
|
||||||
# For example, 0,5,7,9-11.
|
|
||||||
|
|
||||||
# Define minimal path
|
|
||||||
PATH=/bin:/usr/bin:/usr/local/bin
|
|
||||||
|
|
||||||
LOG_FUNCTIONS=${LOG_FUNCTIONS:-"/etc/init.d/log_functions.sh"}
|
|
||||||
CPUMAP_FUNCTIONS=${CPUMAP_FUNCTIONS:-"/etc/init.d/cpumap_functions.sh"}
|
|
||||||
[[ -e ${LOG_FUNCTIONS} ]] && source ${LOG_FUNCTIONS}
|
|
||||||
[[ -e ${CPUMAP_FUNCTIONS} ]] && source ${CPUMAP_FUNCTIONS}
|
|
||||||
|
|
||||||
if [ $UID -ne 0 ]; then
|
|
||||||
log_error "$0 requires root or sudo privileges"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ "$#" -ne 2 ]; then
|
|
||||||
log_error "$0 requires policy and cpulist parameters"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
POLICY=$1
|
|
||||||
CPU_LIST=$2
|
|
||||||
NUMBER_OF_CPUS=$(getconf _NPROCESSORS_CONF 2>/dev/null)
|
|
||||||
STATUS=1
|
|
||||||
|
|
||||||
for CPU_NUM in $(expand_sequence "$CPU_LIST" " "); do
|
|
||||||
# Check that we are not setting PM QoS policy for non-existing CPU
|
|
||||||
if [ "$CPU_NUM" -lt "0" ] || [ "$CPU_NUM" -ge "$NUMBER_OF_CPUS" ]; then
|
|
||||||
log_error "CPU number ${CPU_NUM} is invalid, available CPUs are 0-${NUMBER_OF_CPUS-1}"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Obtain CPU wakeup latencies for all C-states available starting from operating state to deepest sleep
|
|
||||||
declare -a LIMITS=()
|
|
||||||
LIMITS+=($(cat /sys/devices/system/cpu/cpu${CPU_NUM}/cpuidle/state*/latency 2>/dev/null | xargs | sort))
|
|
||||||
if [ ${#LIMITS[@]} -eq 0 ]; then
|
|
||||||
log_debug "Failed to get PM QoS latency limits for CPU ${CPU_NUM}"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Select appropriate CPU wakeup latency based on "low" or "high" policy
|
|
||||||
case "${POLICY}" in
|
|
||||||
"low")
|
|
||||||
# Get first sleep state for "low" policy
|
|
||||||
if [ ${#LIMITS[@]} -eq 0 ]; then
|
|
||||||
LATENCY=1
|
|
||||||
else
|
|
||||||
LATENCY=${LIMITS[1]}
|
|
||||||
fi
|
|
||||||
;;
|
|
||||||
"high")
|
|
||||||
# Get deepest sleep state for "high" policy
|
|
||||||
if [ ${#LIMITS[@]} -eq 0 ]; then
|
|
||||||
LATENCY=1000
|
|
||||||
else
|
|
||||||
LATENCY=${LIMITS[${#LIMITS[@]}-1]}
|
|
||||||
fi
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
log_error "Policy is invalid, can be either low or high"
|
|
||||||
exit 1
|
|
||||||
esac
|
|
||||||
|
|
||||||
# Set the latency for paricular CPU
|
|
||||||
echo ${LATENCY} > /sys/devices/system/cpu/cpu${CPU_NUM}/power/pm_qos_resume_latency_us 2>/dev/null
|
|
||||||
RET_VAL=$?
|
|
||||||
if [ ${RET_VAL} -ne 0 ]; then
|
|
||||||
log_error "Failed to set PM QoS latency for CPU ${CPU_NUM}, rc=${RET_VAL}"
|
|
||||||
continue
|
|
||||||
else
|
|
||||||
log_debug "Succesfully set PM QoS latency for CPU ${CPU_NUM}, rc=${RET_VAL}"
|
|
||||||
STATUS=0
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
exit ${STATUS}
|
|
@ -1,233 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
################################################################################
|
|
||||||
# Copyright (c) 2017 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
#
|
|
||||||
### BEGIN INIT INFO
|
|
||||||
# Provides: task_affinity_functions
|
|
||||||
# Required-Start:
|
|
||||||
# Required-Stop:
|
|
||||||
# Default-Start: 2 3 4 5
|
|
||||||
# Default-Stop: 0 1 6
|
|
||||||
# Short-Description: task_affinity_functions
|
|
||||||
### END INIT INFO
|
|
||||||
|
|
||||||
# Define minimal path
|
|
||||||
PATH=/bin:/usr/bin:/usr/local/bin
|
|
||||||
|
|
||||||
. /etc/platform/platform.conf
|
|
||||||
LOG_FUNCTIONS=${LOG_FUNCTIONS:-"/etc/init.d/log_functions.sh"}
|
|
||||||
CPUMAP_FUNCTIONS=${CPUMAP_FUNCTIONS:-"/etc/init.d/cpumap_functions.sh"}
|
|
||||||
[[ -e ${LOG_FUNCTIONS} ]] && source ${LOG_FUNCTIONS}
|
|
||||||
[[ -e ${CPUMAP_FUNCTIONS} ]] && source ${CPUMAP_FUNCTIONS}
|
|
||||||
|
|
||||||
# Enable debug logs and tag them
|
|
||||||
LOG_DEBUG=1
|
|
||||||
TAG="TASKAFFINITY:"
|
|
||||||
|
|
||||||
TASK_AFFINING_INCOMPLETE="/etc/platform/.task_affining_incomplete"
|
|
||||||
N_CPUS=$(getconf _NPROCESSORS_ONLN)
|
|
||||||
FULLSET_CPUS="0-"$((N_CPUS-1))
|
|
||||||
FULLSET_MASK=$(cpulist_to_cpumap ${FULLSET_CPUS} ${N_CPUS})
|
|
||||||
PLATFORM_CPUS=$(platform_expanded_cpu_list)
|
|
||||||
PLATFORM_CPULIST=$(platform_expanded_cpu_list| \
|
|
||||||
perl -pe 's/(\d+)-(\d+)/join(",",$1..$2)/eg'| \
|
|
||||||
sed 's/,/ /g')
|
|
||||||
VSWITCH_CPULIST=$(get_vswitch_cpu_list| \
|
|
||||||
perl -pe 's/(\d+)-(\d+)/join(",",$1..$2)/eg'| \
|
|
||||||
sed 's/,/ /g')
|
|
||||||
if [[ $vswitch_type =~ none ]]; then
|
|
||||||
VSWITCH_CPULIST=""
|
|
||||||
fi
|
|
||||||
|
|
||||||
IDLE_MARK=95.0
|
|
||||||
KERNEL=$(uname -a)
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Check if a given core is one of the platform cores
|
|
||||||
################################################################################
|
|
||||||
function is_platform_core {
|
|
||||||
local core=$1
|
|
||||||
for CPU in ${PLATFORM_CPULIST}; do
|
|
||||||
if [ $core -eq $CPU ]; then
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# Check if a given core is one of the vswitch cores
|
|
||||||
################################################################################
|
|
||||||
function is_vswitch_core {
|
|
||||||
local core=$1
|
|
||||||
for CPU in ${VSWITCH_CPULIST}; do
|
|
||||||
if [ $core -eq $CPU ]; then
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
# Return list of reaffineable pids. This includes all processes, but excludes
|
|
||||||
# kernel threads, vSwitch, and anything in K8S or qemu/kvm.
|
|
||||||
function reaffineable_pids {
|
|
||||||
local pids_excl
|
|
||||||
local pidlist
|
|
||||||
|
|
||||||
pids_excl=$(ps -eL -o pid=,comm= | \
|
|
||||||
awk -vORS=',' '/eal-intr-thread|kthreadd/ {print $1}' | \
|
|
||||||
sed 's/,$/\n/')
|
|
||||||
pidlist=$(ps --ppid ${pids_excl} -p ${pids_excl} --deselect \
|
|
||||||
-o pid=,cgroup= | \
|
|
||||||
awk '!/k8s-infra|machine.slice/ {print $1; }')
|
|
||||||
echo "${pidlist[@]}"
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# The following function can be called by any platform service that needs to
|
|
||||||
# temporarily make use of idle VM cores to run a short-duration, service
|
|
||||||
# critical and cpu intensive operation in AIO. For instance, sm can levearage
|
|
||||||
# the idle cores to speed up swact activity.
|
|
||||||
#
|
|
||||||
# At the end of the operation, regarless of the result, the service must be
|
|
||||||
# calling function affine_tasks_to_platform_cores to re-affine platform tasks
|
|
||||||
# back to their assigned core(s).
|
|
||||||
#
|
|
||||||
# Kernel, vswitch and VM related tasks are untouched.
|
|
||||||
################################################################################
|
|
||||||
function affine_tasks_to_idle_cores {
|
|
||||||
local cpulist
|
|
||||||
local cpuocc_list
|
|
||||||
local vswitch_pid
|
|
||||||
local pidlist
|
|
||||||
local idle_cpulist
|
|
||||||
local platform_cpus
|
|
||||||
local rc=0
|
|
||||||
local cpu=0
|
|
||||||
|
|
||||||
if [ -f ${TASK_AFFINING_INCOMPLETE} ]; then
|
|
||||||
read cpulist < ${TASK_AFFINING_INCOMPLETE}
|
|
||||||
log_debug "${TAG} Tasks have already been affined to CPU ($cpulist)."
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [[ "${KERNEL}" == *" RT "* ]]; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Compile a list of cpus with idle percentage greater than 95% in the last
|
|
||||||
# 5 seconds.
|
|
||||||
cpuocc_list=($(sar -P ALL 1 5|grep Average|awk '{if(NR>2)print $8}'))
|
|
||||||
|
|
||||||
for idle_value in ${cpuocc_list[@]}; do
|
|
||||||
is_vswitch_core $cpu
|
|
||||||
if [ $? -eq 1 ]; then
|
|
||||||
cpu=$(($cpu+1))
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
|
|
||||||
is_platform_core $cpu
|
|
||||||
if [ $? -eq 1 ]; then
|
|
||||||
# Platform core is added to the idle list by default
|
|
||||||
idle_cpulist=$idle_cpulist$cpu","
|
|
||||||
else
|
|
||||||
# Non platform core is added to the idle list if it is more
|
|
||||||
# than 95% idle
|
|
||||||
if [[ $(echo "$idle_value > ${IDLE_MARK}"|bc) -eq 1 ]]; then
|
|
||||||
idle_cpulist=$idle_cpulist$cpu","
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
cpu=$(($cpu+1))
|
|
||||||
done
|
|
||||||
|
|
||||||
idle_cpulist=$(echo $idle_cpulist|sed 's/.$//')
|
|
||||||
|
|
||||||
log_debug "${TAG} Affining all tasks to idle CPU ($idle_cpulist)"
|
|
||||||
pidlist=( $(reaffineable_pids) )
|
|
||||||
for pid in ${pidlist[@]}; do
|
|
||||||
taskset --all-tasks --pid --cpu-list \
|
|
||||||
${idle_cpulist} ${pid} > /dev/null 2>&1
|
|
||||||
done
|
|
||||||
|
|
||||||
# Save the cpu list to the temp file which will be read and removed when
|
|
||||||
# tasks are reaffined to the platform cores later on.
|
|
||||||
echo $idle_cpulist > ${TASK_AFFINING_INCOMPLETE}
|
|
||||||
return $rc
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# The following function is called by sm at the end of swact sequence
|
|
||||||
# to re-affine management tasks back to the platform cores.
|
|
||||||
################################################################################
|
|
||||||
function affine_tasks_to_platform_cores {
|
|
||||||
local cpulist
|
|
||||||
local pidlist
|
|
||||||
local rc=0
|
|
||||||
local count=0
|
|
||||||
|
|
||||||
if [ ! -f ${TASK_AFFINING_INCOMPLETE} ]; then
|
|
||||||
dbg_str="${TAG} Either tasks have never been affined to all/idle"
|
|
||||||
dbg_str="${TAG} cores or they have already been reaffined to"
|
|
||||||
dbg_str="${TAG} platform cores."
|
|
||||||
log_debug "$dbg_str"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
read cpulist < ${TASK_AFFINING_INCOMPLETE}
|
|
||||||
|
|
||||||
log_debug "${TAG} Reaffining tasks to platform cores (${PLATFORM_CPUS})..."
|
|
||||||
pidlist=( $(reaffineable_pids) )
|
|
||||||
for pid in ${pidlist[@]}; do
|
|
||||||
taskset --all-tasks --pid --cpu-list \
|
|
||||||
${PLATFORM_CPUS} ${pid} > /dev/null 2>&1
|
|
||||||
done
|
|
||||||
|
|
||||||
# Reaffine vSwitch tasks that span multiple cpus to platform cpus
|
|
||||||
pidlist=$(ps -eL -o pid=,comm= | awk '/eal-intr-thread/ {print $1}')
|
|
||||||
for pid in ${pidlist[@]}; do
|
|
||||||
grep Cpus_allowed_list /proc/${pid}/task/*/status 2>/dev/null | \
|
|
||||||
sed 's#/# #g' | awk '/,|-/ {print $4}' | \
|
|
||||||
xargs --no-run-if-empty -i{} \
|
|
||||||
taskset --pid --cpu-list ${PLATFORM_CPUS} {} > /dev/null 2>&1
|
|
||||||
done
|
|
||||||
|
|
||||||
rm -rf ${TASK_AFFINING_INCOMPLETE}
|
|
||||||
return $rc
|
|
||||||
}
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
# The following function can be leveraged by cron tasks
|
|
||||||
################################################################################
|
|
||||||
function get_most_idle_core {
|
|
||||||
local cpuocc_list
|
|
||||||
local cpu=0
|
|
||||||
local most_idle_value=${IDLE_MARK}
|
|
||||||
local most_idle_cpu=0
|
|
||||||
|
|
||||||
if [[ "${KERNEL}" == *" RT "* ]]; then
|
|
||||||
echo $cpu
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
cpuocc_list=($(sar -P ALL 1 5|grep Average|awk '{if(NR>2)print $8}'))
|
|
||||||
|
|
||||||
for idle_value in ${cpuocc_list[@]}; do
|
|
||||||
is_vswitch_core $cpu
|
|
||||||
if [ $? -eq 1 ]; then
|
|
||||||
cpu=$(($cpu+1))
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ $(echo "$idle_value > $most_idle_value"|bc) -eq 1 ]; then
|
|
||||||
most_idle_value=$idle_value
|
|
||||||
most_idle_cpu=$cpu
|
|
||||||
fi
|
|
||||||
cpu=$(($cpu+1))
|
|
||||||
done
|
|
||||||
|
|
||||||
echo $most_idle_cpu
|
|
||||||
}
|
|
@ -1,8 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
#
|
|
||||||
# Copyright (c) 2013-2014 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
|
|
||||||
python /usr/bin/topology.pyc
|
|
@ -1,242 +0,0 @@
|
|||||||
#!/usr/bin/env python
|
|
||||||
################################################################################
|
|
||||||
# Copyright (c) 2013 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
#
|
|
||||||
# topology.py -- gives a summary of logical cpu enumeration,
|
|
||||||
# sockets, cores per package, threads per core,
|
|
||||||
# total memory, and numa nodes
|
|
||||||
|
|
||||||
from __future__ import print_function
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
import re
|
|
||||||
|
|
||||||
class Topology(object):
|
|
||||||
""" Build up topology information.
|
|
||||||
(i.e. logical cpu topology, NUMA nodes, memory)
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
self.num_cpus = 0
|
|
||||||
self.num_nodes = 0
|
|
||||||
self.num_sockets = 0
|
|
||||||
self.num_cores_per_pkg = 0
|
|
||||||
self.num_threads_per_core = 0
|
|
||||||
|
|
||||||
self.topology = {}
|
|
||||||
self.topology_idx = {}
|
|
||||||
self.total_memory_MiB = 0
|
|
||||||
self.total_memory_nodes_MiB = []
|
|
||||||
|
|
||||||
self._get_cpu_topology()
|
|
||||||
self._get_total_memory_MiB()
|
|
||||||
self._get_total_memory_nodes_MiB()
|
|
||||||
|
|
||||||
def _get_cpu_topology(self):
|
|
||||||
'''Enumerate logical cpu topology based on parsing /proc/cpuinfo
|
|
||||||
as function of socket_id, core_id, and thread_id. This updates
|
|
||||||
topology and reverse index topology_idx mapping.
|
|
||||||
|
|
||||||
:param self
|
|
||||||
:updates self.num_cpus - number of logical cpus
|
|
||||||
:updates self.num_nodes - number of sockets; maps to number of numa nodes
|
|
||||||
:updates self.topology[socket_id][core_id][thread_id] = cpu
|
|
||||||
:updates self.topology_idx[cpu] = {'s': socket_id, 'c': core_id, 't': thread_id}
|
|
||||||
:returns None
|
|
||||||
'''
|
|
||||||
|
|
||||||
self.num_cpus = 0
|
|
||||||
self.num_nodes = 0
|
|
||||||
self.num_sockets = 0
|
|
||||||
self.num_cores = 0
|
|
||||||
self.num_threads = 0
|
|
||||||
self.topology = {}
|
|
||||||
self.topology_idx = {}
|
|
||||||
|
|
||||||
Thread_cnt = {}
|
|
||||||
cpu = socket_id = core_id = thread_id = -1
|
|
||||||
re_processor = re.compile(r'^[Pp]rocessor\s+:\s+(\d+)')
|
|
||||||
re_socket = re.compile(r'^physical id\s+:\s+(\d+)')
|
|
||||||
re_core = re.compile(r'^core id\s+:\s+(\d+)')
|
|
||||||
|
|
||||||
with open('/proc/cpuinfo', 'r') as infile:
|
|
||||||
for line in infile:
|
|
||||||
|
|
||||||
match = re_processor.search(line)
|
|
||||||
if match:
|
|
||||||
cpu = int(match.group(1))
|
|
||||||
socket_id = -1; core_id = -1; thread_id = -1
|
|
||||||
self.num_cpus += 1
|
|
||||||
continue
|
|
||||||
|
|
||||||
match = re_socket.search(line)
|
|
||||||
if match:
|
|
||||||
socket_id = int(match.group(1))
|
|
||||||
continue
|
|
||||||
|
|
||||||
match = re_core.search(line)
|
|
||||||
if match:
|
|
||||||
core_id = int(match.group(1))
|
|
||||||
|
|
||||||
if socket_id not in Thread_cnt:
|
|
||||||
Thread_cnt[socket_id] = {}
|
|
||||||
if core_id not in Thread_cnt[socket_id]:
|
|
||||||
Thread_cnt[socket_id][core_id] = 0
|
|
||||||
else:
|
|
||||||
Thread_cnt[socket_id][core_id] += 1
|
|
||||||
thread_id = Thread_cnt[socket_id][core_id]
|
|
||||||
|
|
||||||
if socket_id not in self.topology:
|
|
||||||
self.topology[socket_id] = {}
|
|
||||||
if core_id not in self.topology[socket_id]:
|
|
||||||
self.topology[socket_id][core_id] = {}
|
|
||||||
|
|
||||||
self.topology[socket_id][core_id][thread_id] = cpu
|
|
||||||
self.topology_idx[cpu] = {'s': socket_id, 'c': core_id, 't': thread_id}
|
|
||||||
continue
|
|
||||||
self.num_nodes = len(self.topology.keys())
|
|
||||||
|
|
||||||
# In the case topology not detected, hard-code structures
|
|
||||||
if self.num_nodes == 0:
|
|
||||||
n_sockets, n_cores, n_threads = (1, self.num_cpus, 1)
|
|
||||||
self.topology = {}
|
|
||||||
for socket_id in range(n_sockets):
|
|
||||||
self.topology[socket_id] = {}
|
|
||||||
for core_id in range(n_cores):
|
|
||||||
self.topology[socket_id][core_id] = {}
|
|
||||||
for thread_id in range(n_threads):
|
|
||||||
self.topology[socket_id][core_id][thread_id] = 0
|
|
||||||
# Define Thread-Socket-Core order for logical cpu enumeration
|
|
||||||
self.topology_idx = {}
|
|
||||||
cpu = 0
|
|
||||||
for thread_id in range(n_threads):
|
|
||||||
for socket_id in range(n_sockets):
|
|
||||||
for core_id in range(n_cores):
|
|
||||||
self.topology[socket_id][core_id][thread_id] = cpu
|
|
||||||
self.topology_idx[cpu] = {'s': socket_id, 'c': core_id, 't': thread_id}
|
|
||||||
cpu += 1
|
|
||||||
self.num_nodes = len(self.topology.keys())
|
|
||||||
|
|
||||||
self.num_sockets = len(self.topology.keys())
|
|
||||||
self.num_cores_per_pkg = len(self.topology[0].keys())
|
|
||||||
self.num_threads_per_core = len(self.topology[0][0].keys())
|
|
||||||
|
|
||||||
return None
|
|
||||||
|
|
||||||
def _get_total_memory_MiB(self):
|
|
||||||
"""Get the total memory for VMs (MiB).
|
|
||||||
|
|
||||||
:updates: total memory for VMs (MiB)
|
|
||||||
|
|
||||||
"""
|
|
||||||
|
|
||||||
self.total_memory_MiB = 0
|
|
||||||
|
|
||||||
# Total memory
|
|
||||||
try:
|
|
||||||
m = open('/proc/meminfo').read().split()
|
|
||||||
idx_Total = m.index('MemTotal:') + 1
|
|
||||||
self.total_memory_MiB = int(m[idx_Total]) / 1024
|
|
||||||
except IOError:
|
|
||||||
# silently ignore IO errors (eg. file missing)
|
|
||||||
pass
|
|
||||||
return None
|
|
||||||
|
|
||||||
def _get_total_memory_nodes_MiB(self):
|
|
||||||
"""Get the total memory per numa node for VMs (MiB).
|
|
||||||
|
|
||||||
:updates: total memory per numa node for VMs (MiB)
|
|
||||||
|
|
||||||
"""
|
|
||||||
|
|
||||||
self.total_memory_nodes_MiB = []
|
|
||||||
|
|
||||||
# Memory of each numa node (MiB)
|
|
||||||
for node in range(self.num_nodes):
|
|
||||||
Total_MiB = 0
|
|
||||||
|
|
||||||
meminfo = "/sys/devices/system/node/node%d/meminfo" % node
|
|
||||||
try:
|
|
||||||
m = open(meminfo).read().split()
|
|
||||||
idx_Total = m.index('MemTotal:') + 1
|
|
||||||
Total_MiB = int(m[idx_Total]) / 1024
|
|
||||||
except IOError:
|
|
||||||
# silently ignore IO errors (eg. file missing)
|
|
||||||
pass
|
|
||||||
|
|
||||||
self.total_memory_nodes_MiB.append(Total_MiB)
|
|
||||||
return None
|
|
||||||
|
|
||||||
def _print_cpu_topology(self):
|
|
||||||
'''Print logical cpu topology enumeration as function of:
|
|
||||||
socket_id, core_id, and thread_id.
|
|
||||||
|
|
||||||
:param self
|
|
||||||
:returns None
|
|
||||||
'''
|
|
||||||
|
|
||||||
cpu_list = self.topology_idx.keys()
|
|
||||||
cpu_list.sort()
|
|
||||||
total_memory_GiB = self.total_memory_MiB/1024.0
|
|
||||||
|
|
||||||
print('TOPOLOGY:')
|
|
||||||
print('%16s : %5d' % ('logical cpus', self.num_cpus))
|
|
||||||
print('%16s : %5d' % ('sockets', self.num_sockets))
|
|
||||||
print('%16s : %5d' % ('cores_per_pkg', self.num_cores_per_pkg))
|
|
||||||
print('%16s : %5d' % ('threads_per_core', self.num_threads_per_core))
|
|
||||||
print('%16s : %5d' % ('numa_nodes', self.num_nodes))
|
|
||||||
print('%16s : %5.2f %s' % ('total_memory', total_memory_GiB, 'GiB'))
|
|
||||||
print('%16s :' % ('memory_per_node'), end=' ')
|
|
||||||
for node in range(self.num_nodes):
|
|
||||||
node_memory_GiB = self.total_memory_nodes_MiB[node]/1024.0
|
|
||||||
print('%5.2f' % (node_memory_GiB), end=' ')
|
|
||||||
print('%s' % ('GiB'))
|
|
||||||
print('')
|
|
||||||
|
|
||||||
print('LOGICAL CPU TOPOLOGY:')
|
|
||||||
print("%9s :" % 'cpu_id', end=' ')
|
|
||||||
for cpu in cpu_list:
|
|
||||||
print("%3d" % cpu, end=' ')
|
|
||||||
print('')
|
|
||||||
print("%9s :" % 'socket_id', end=' ')
|
|
||||||
for cpu in cpu_list:
|
|
||||||
socket_id = self.topology_idx[cpu]['s']
|
|
||||||
print("%3d" % socket_id, end=' ')
|
|
||||||
print('')
|
|
||||||
print("%9s :" % 'core_id', end=' ')
|
|
||||||
for cpu in cpu_list:
|
|
||||||
core_id = self.topology_idx[cpu]['c']
|
|
||||||
print("%3d" % core_id, end=' ')
|
|
||||||
print('')
|
|
||||||
print("%9s :" % 'thread_id', end=' ')
|
|
||||||
for cpu in cpu_list:
|
|
||||||
thread_id = self.topology_idx[cpu]['t']
|
|
||||||
print("%3d" % thread_id, end=' ')
|
|
||||||
print('')
|
|
||||||
print('')
|
|
||||||
|
|
||||||
print('CORE TOPOLOGY:')
|
|
||||||
print("%6s %9s %7s %9s %s" % ('cpu_id', 'socket_id', 'core_id', 'thread_id', 'affinity'))
|
|
||||||
for cpu in cpu_list:
|
|
||||||
affinity = 1<<cpu
|
|
||||||
socket_id = self.topology_idx[cpu]['s']
|
|
||||||
core_id = self.topology_idx[cpu]['c']
|
|
||||||
thread_id = self.topology_idx[cpu]['t']
|
|
||||||
print("%6d %9d %7d %9d 0x%x" \
|
|
||||||
% (cpu, socket_id, core_id, thread_id, affinity))
|
|
||||||
|
|
||||||
return None
|
|
||||||
|
|
||||||
#-------------------------------------------------------------------------------
|
|
||||||
''' Main Program
|
|
||||||
'''
|
|
||||||
|
|
||||||
# Get logical cpu topology
|
|
||||||
topology = Topology()
|
|
||||||
topology._print_cpu_topology()
|
|
||||||
|
|
||||||
sys.exit(0)
|
|
@ -1,24 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
#
|
|
||||||
# Copyright (c) 2014,2016 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
|
|
||||||
#
|
|
||||||
# worker "goenabled" check.
|
|
||||||
#
|
|
||||||
# If a problem was detected during configuration of worker
|
|
||||||
# resources then the board is not allowed to enable.
|
|
||||||
#
|
|
||||||
WORKER_GOENABLED="/var/run/worker_goenabled"
|
|
||||||
|
|
||||||
source "/etc/init.d/log_functions.sh"
|
|
||||||
source "/usr/bin/tsconfig"
|
|
||||||
|
|
||||||
if [ -e ${VOLATILE_WORKER_CONFIG_COMPLETE} -a ! -f ${WORKER_GOENABLED} ]; then
|
|
||||||
log_error "Worker manifest CPU configuration check failed. Failing goenabled check."
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
exit 0
|
|
@ -1,55 +0,0 @@
|
|||||||
################################################################################
|
|
||||||
# Copyright (c) 2018 Wind River Systems, Inc.
|
|
||||||
#
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
# WORKER Node configuration parameters for reserved memory and physical cores
|
|
||||||
# used by Base software and VSWITCH. These are resources that libvirt cannot use.
|
|
||||||
#
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
#
|
|
||||||
# List of logical CPU instances available in the system. This value is used
|
|
||||||
# for auditing purposes so that the current configuration can be checked for
|
|
||||||
# validity against the actual number of logical CPU instances in the system.
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
WORKER_CPU_LIST="0-1"
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
#
|
|
||||||
# List of Base software resources reserved per numa node. Each array element
|
|
||||||
# consists of a 3-tuple formatted as: <node>:<memory>:<cores>.
|
|
||||||
#
|
|
||||||
# Example: To reserve 1500MB and 1 core on NUMA node0, and 1500MB and 1 core
|
|
||||||
# on NUMA node1, the variable must be specified as follows.
|
|
||||||
# WORKER_BASE_MEMORY=("node0:1500MB:1" "node1:1500MB:1")
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
WORKER_BASE_RESERVED=("node0:8000MB:1" "node1:2000MB:0" "node2:2000MB:0" "node3:2000MB:0")
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
#
|
|
||||||
# List of HugeTLB memory descriptors to configure. Each array element
|
|
||||||
# consists of a 3-tuple descriptor formatted as: <node>:<pgsize>:<pgcount>.
|
|
||||||
# The NUMA node specified must exist and the HugeTLB pagesize must be a valid
|
|
||||||
# value such as 2048kB or 1048576kB.
|
|
||||||
#
|
|
||||||
# For example, to request 256 x 2MB HugeTLB pages on NUMA node0 and node1 the
|
|
||||||
# variable must be specified as follows.
|
|
||||||
# WORKER_VSWITCH_MEMORY=("node0:2048kB:256" "node1:2048kB:256")
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
WORKER_VSWITCH_MEMORY=("node0:1048576kB:1" "node1:1048576kB:1" "node2:1048576kB:1" "node3:1048576kB:1")
|
|
||||||
|
|
||||||
################################################################################
|
|
||||||
#
|
|
||||||
# List of VSWITCH physical cores reserved for VSWITCH applications.
|
|
||||||
#
|
|
||||||
# Example: To reserve 2 cores on NUMA node0, and 2 cores on NUMA node1, the
|
|
||||||
# variable must be specified as follows.
|
|
||||||
# WORKER_VSWITCH_CORES=("node0:2" "node1:2")
|
|
||||||
#
|
|
||||||
################################################################################
|
|
||||||
WORKER_VSWITCH_CORES=("node0:2" "node1:0" "node2:0" "node3:0")
|
|
Loading…
Reference in New Issue
Block a user