865 Commits

Author SHA1 Message Date
Zuul
4e4c503625 Merge "Define base tox-functional jobs" 2019-06-24 22:21:14 +00:00
Zuul
6cea1d1e4c Merge "speclint: Add script for linting RPM specfiles" 2019-06-24 16:11:17 +00:00
Saul Wold
39968d9909 speclint: Add script for linting RPM specfiles
This script along with the macros file will be used to initially
check the specfiles being added as part of the MultiOS work. It
will be executed via zuul and tox to scan the openSUSE specfiles,
later we can start add the CentOS specfiles also.

The macros.openstack-singlespec is orignally from
openstack/rpm-packaging customized with tis_patch_ver

Change-Id: I22bf778388e6dcc3ca42d5fd6ec16b30200d6b75
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2019-06-18 19:11:54 -07:00
Stefan Dinescu
8decb63402 Revert "Container Openstackclient wrapper"
This reverts commit 20ca6a6167805e5c78e1ca40777f4a22bfa47fab.

Change-Id: Iadd239d62352e46c6a921233af7da7d5c44b66fc
Signed-off-by: Stefan Dinescu <stefan.dinescu@windriver.com>
2019-06-18 20:35:27 +03:00
Zuul
e94582dc37 Merge "Align the keystone uids with upstream" 2019-06-17 20:03:23 +00:00
Tao Liu
7ef6d4e330 Align the keystone uids with upstream
Update the keystone user uid and gid to align it with
upstream. This is required by distributed cloud system
where the keystone container mounts the platform fernet
key repo.

Change-Id: Ide8935d8f6baf5796cde57e2ccc2ae8d9cf1b8ea
Story: 2004766
Task: 34152
Signed-off-by: Tao Liu <tao.liu@windriver.com>
2019-06-17 12:07:53 -04:00
Saul Wold
83c6575d51 integ: Convert wrsroot -> sysadmin
This also changes the group wrs_protected to sys_protected
to de-brand the user and group names.

Depends-On: I887464a20fc17d66529caea03be2b445156f9426
Change-Id: Ic2ea06d3ac15c31854a604af5f4cecf9094fcaea
Story: 2004716
Task: 28748
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2019-06-14 15:09:09 -07:00
Zuul
6ccb588bf8 Merge "Container Openstackclient wrapper" 2019-06-13 18:08:55 +00:00
Zuul
4941be06cc Merge "Upgrade std kernel patch to CentOS7.6 3.10.0-957.12.2" 2019-06-12 01:20:14 +00:00
Zuul
dd59b2fa65 Merge "Upgrade rt kernel patch to CentOS7.6 3.10.0-957.12.2" 2019-06-12 01:15:48 +00:00
zhiguo.zhang
30788066d6 Upgrade std kernel patch to CentOS7.6 3.10.0-957.12.2
New set of CVEs was reported against Intel CPUs: CVE-2018-12126,
CVE-2018-12127, CVE-2018-12130 and CVE-2019-11091.
For these CVEs there are RH and CentOS updates available.

CVE-2018-12126:
Microarchitectural Store Buffer Data Sampling (MSBDS):
Store buffers on some microprocessors utilizing speculative
execution may allow an authenticated user to potentially
enable information disclosure via a side channel with local access.
A list of impacted products can be found here:
https://www.intel.com/content/dam/www/public/us/en/documents/
corporate-information/SA00233-microcode-update-guidance_05132019.pdf

CVE-2018-12127:
Microarchitectural Load Port Data Sampling (MLPDS):
Load ports on some microprocessors utilizing speculative execution
may allow an authenticated user to potentially enable information
disclosure via a side channel with local access. A list of impacted
products can be found here:
https://www.intel.com/content/dam/www/public/us/en/documents/
corporate-information/SA00233-microcode-update-guidance_05132019.pdf

CVE-2018-12130:
Microarchitectural Fill Buffer Data Sampling (MFBDS):
Fill buffers on some microprocessors utilizing speculative execution
may allow an authenticated user to potentially enable information
disclosure via a side channel with local access. A list of impacted
products can be found here:
https://www.intel.com/content/dam/www/public/us/en/documents/
corporate-information/SA00233-microcode-update-guidance_05132019.pdf

CVE-2019-11091:
Microarchitectural Data Sampling Uncacheable Memory(MDSUM):
Uncacheable memory on some microprocessors utilizing speculative
execution may allow an authenticated user to potentially enable
information disclosure via a side channel with local access.
A list of impacted products can be found here:
https://www.intel.com/content/dam/www/public/us/en/documents/
corporate-information/SA00233-microcode-update-guidance_05132019.pdf

These are from the http://cve.mitre.org website.
These are the MDS security CVEs.

The patch is modified as follows:
1.Delete the 929-931 line of the arch/x86/kernel/cpu/cacheinfo.c file,
  because starlingx's Porting-Cacheinfo-from-Kernel-4.10.17.patch
  removes the ici_cpuid4_info structure.

2.The build-logic-and-sources-for-TiC.patch version number
  has been modified.

3.In addition to the modifications in the files in 1 and 2,
  other patches only modify the line number.

Closes-Bug: 1830487
Depends-On: https://review.opendev.org/663071
Change-Id: I4cad783311ed4a6c60b4f69bdad75d773d0cd23d
Signed-off-by: zhiguo.zhang <zhiguox.zhang@intel.com>
2019-06-11 11:09:32 +08:00
zhiguo.zhang
d4aebcaf91 Upgrade rt kernel patch to CentOS7.6 3.10.0-957.12.2
New set of CVEs was reported against Intel CPUs: CVE-2018-12126,
CVE-2018-12127, CVE-2018-12130 and CVE-2019-11091.
For these CVEs there are RH and CentOS updates available.

CVE-2018-12126:
Microarchitectural Store Buffer Data Sampling (MSBDS):
Store buffers on some microprocessors utilizing speculative
execution may allow an authenticated user to potentially
enable information disclosure via a side channel with local access.
A list of impacted products can be found here:
https://www.intel.com/content/dam/www/public/us/en/documents/
corporate-information/SA00233-microcode-update-guidance_05132019.pdf

CVE-2018-12127:
Microarchitectural Load Port Data Sampling (MLPDS):
Load ports on some microprocessors utilizing speculative execution
may allow an authenticated user to potentially enable information
disclosure via a side channel with local access. A list of impacted
products can be found here:
https://www.intel.com/content/dam/www/public/us/en/documents/
corporate-information/SA00233-microcode-update-guidance_05132019.pdf

CVE-2018-12130:
Microarchitectural Fill Buffer Data Sampling (MFBDS):
Fill buffers on some microprocessors utilizing speculative execution
may allow an authenticated user to potentially enable information
disclosure via a side channel with local access. A list of impacted
products can be found here:
https://www.intel.com/content/dam/www/public/us/en/documents/
corporate-information/SA00233-microcode-update-guidance_05132019.pdf

CVE-2019-11091:
Microarchitectural Data Sampling Uncacheable Memory(MDSUM):
Uncacheable memory on some microprocessors utilizing speculative
execution may allow an authenticated user to potentially enable
information disclosure via a side channel with local access.
A list of impacted products can be found here:
https://www.intel.com/content/dam/www/public/us/en/documents/
corporate-information/SA00233-microcode-update-guidance_05132019.pdf

These are from the http://cve.mitre.org website.
These are the MDS security CVEs.

The patch is modified as follows:
1.Delete the 929-931 line of the arch/x86/kernel/cpu/cacheinfo.c
  file,
  because starlingx's Porting-Cacheinfo-from-Kernel-4.10.17.patch
  removes the ici_cpuid4_info structure.

2.Except for the modification of the file in 1, the other patches
  only modify the line number.

Closes-Bug: 1830487
Depends-On: https://review.opendev.org/663071
Change-Id: I16ac63df21eeb85b4fc3ab19d539986e77c8c0d3
Signed-off-by: zhiguo.zhang <zhiguox.zhang@intel.com>
2019-06-11 10:45:28 +08:00
Steven Webster
8dbd4fbcf8 Upversion ixgbe, ixgbevf drivers
This commit upgrades the ixgbe and ixgbevf drivers to the
latest versions.

ixgbe is upversioned to 5.5.5 from 5.5.3
ixgbevf is upversioned to 4.5.3 from 4.5.1

For ixgbe, Intel has noted that RHEL 7.6 support is introduced
in version 5.5.5, whereas only RHEL 7.5 support is present in version
5.5.3.  The 5.5.5 version will also pick up a bug fix needed
for SR-IOV operations.

For ixgbevf, Intel has verified RHEL 7.6 support in version 4.5.3,
while they have verified only RHEL 7.5 in version 4.5.1.

Depends-On: https://review.opendev.org/#/c/664280
Change-Id: Ic7e0089a7b218094f3367cdce17ec950359cedae
Closes-Bug: #1830636
Signed-off-by: Steven Webster <steven.webster@windriver.com>
2019-06-10 10:43:05 -05:00
Zuul
6c7632be90 Merge "Fix the runtime requirements for collectd-extensions" 2019-06-07 22:07:01 +00:00
Al Bailey
40e57e46a5 Fix the runtime requirements for collectd-extensions
The python code in the collectd-extensions requires several
python modules in order to run, but is missing the explicit
dependency against those modules in the package.

These include:
  fm-api
  httplib2
  influxdb
  oslo-concurrency
  tsconfig

Change-Id: I9ace889fdb7fac031792486c3e5ddf3bc2cae770
Story: 2004764
Task: 33630
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2019-06-07 14:47:32 -05:00
Zuul
8638c37d64 Merge "Enable Non-Occurring Flake8 Errors" 2019-06-07 18:24:31 +00:00
Zuul
368340a513 Merge "Create OVS docker image with version 2.11.0" 2019-06-07 14:36:55 +00:00
Kristine Bujold
7e56b74ee8 Add /var/log/armada/ to logrotate
This commit adds /var/log/armada, which stores application related
logs generated by Armada service, to logrotate.

Story: 2003908
Task: 28267
Depends-On: https://review.opendev.org/663347

Change-Id: I98c7caf85cfecf4de1f55be69a00697f9073a1a8
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
2019-06-06 13:43:05 -04:00
mid_one
5c0b0d9044 Create OVS docker image with version 2.11.0
The version of existing OVS docker image is too old. StarlingX
needs to build its own OVS docker image with latest version
2.11.0 which is the same as the version of OVS running on
hosts.

Change-Id: I819678b3309d7571b51e275718b17d87415cf894
Story: #2004649
Task: #30281
Co-Authored-By: Cheng Li<cheng1.li@intel.com>
Signed-off-by: Chenjie Xu <chenjie.xu@intel.com>
2019-06-04 02:34:14 +08:00
Eric Barrett
37aced703d Enable Non-Occurring Flake8 Errors
Flake8 currently ignores the following errors:
B301: Python3 does not include ".iter*" methods on dictionaries
H237: Module is removed in python3
W191: Indentation contains tabs
Which do not appear currently in the repo. Enable them
so that they do not get introduced

Change-Id: I0321ee40f869c03321a1cbd67c45056206437f6b
Story: 2004515
Task: 30076
Signed-off-by: Eric Barrett <eric.barrett@windriver.com>
2019-06-03 11:32:06 -04:00
Shuicheng Lin
8948026e7c de-fuzz fuzzy patch in systemd
Fuzzy patch will cause potential build failure issue. Correct the
line number in patch file to de-fuzz it.

Story: 2004660
Task: 33558
Change-Id: Iccc880025c791d38835e9cd535eab657529c6f47
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
2019-06-02 10:15:51 +08:00
Zuul
05bf7be027 Merge "Ceph build script improvements to prevent needless rebuilds" 2019-05-29 04:21:20 +00:00
Scott Little
526f115ede Add two sriov docker images to the stable docker image build.
The SRIOV network device plugin is Kubernetes device plugin for
discovering and advertising SRIOV network virtual functions (VFs) in a
Kubernetes host.

StarlingX support requires us to build the following plugin images:

starlingx/k8s-cni-sriov: derived from ...
    https://github.com/intel/sriov-cni

starlingx/k8s-plugins-sriov-network-device: derived from ...
    https://github.com/intel/sriov-network-device-plugin

Change-Id: I1ab9f642040dcacfc4e3494cbc6aef83816d3c20
Depends-on: Iea5eae32bd245557a4b02c9825297343a001e778
Story: 2005208
Task: 33485
Signed-off-by: Scott Little <scott.little@windriver.com>
2019-05-28 13:58:00 -04:00
Scott Little
663edb4567 Ceph build script improvements to prevent needless rebuilds
Problem: The file $SRC_DIR/src/.git_version is created by the
custom build script and is not cleaned upon exit.  The presence
of the new file under $SRC_DIR will trigger a rebuild on the
next iteration.

Solution: Add a cleanup of the file to cover the normal exit case.
This means $SRC_DIR/src/.git_version will not be present at the
start of the next build.

Problem: A lot of tarballs are copied into the build by the
build script, rather than being listed in the COPY_LIST.  This
breaks the md5 checksum mechanism used to determine if a rebuild
is required.  A change to a tarball will be ignored and not
trigger a rebuild.

Solution: Move the code that generates the list of input tarballs
into build_srpm.data.  The file is sourced, so COPY_LIST can
be populated dynamically.

Problem: Script returns success when rpmbuild fails.

Solution: Propogate the error code to the final exit.

Change-Id: I2e760c24ecd3ce2d237863b948863c2a876d24fa
Closes-Bug: 1830130
Co-authored-by: Shuicheng Lin <shuicheng.lin@intel.com>
Signed-off-by: Scott Little <scott.little@windriver.com>
2019-05-24 13:10:28 -04:00
Jim Gauld
6bd45c96dd Create k8s-infra cgroup path before kubelet launch
This adds a kubelet ExecStartPre script to ensure cgroup is setup
prior to kubelet launch. This creates k8s-infra cgroup for a minimal
set of resource controllers, and configures cpuset attributes to span
all online cpus and nodes. This will do nothing if the k8s-infra cgroup
already exists (i.e., assume already configured).

NOTE: The creation of directories under /sys/fs/cgroup is volatile, and
does not persist reboots. The cpuset.mems and cpuset.cpus is later
updated by puppet kubernetes.pp manifest.

Tests performed:
Standard system: system install, lock/unlock controller & computes,
forced reboot: active/standby controller, computes.

Change-Id: I6a7aad5c40fe8225e9e16c8d8b40a0cffd76715d
Closes-Bug: 1828270
Signed-off-by: Jim Gauld <james.gauld@windriver.com>
2019-05-17 00:28:04 -04:00
Dean Troyer
f8f2899038 Define base tox-functional jobs
Add flock-devstack-tox-base and flock-devstack-tox-base-min jobs
that will execute tox functional tests using the running DevStack.

This includes a test of running with a minimal tox functional
environment for cases where those tests have not been writen yet.

Change-Id: I436ba5a1c8f72b86bcdff4128b9960f4571be048
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2019-05-16 17:22:59 -05:00
zhipengl
ce0cc60346 Fix for pci-irq-affinity-agent failing to start in AIO
Ensure that pci-irq-affinity-agent is launched on worker nodes.
This includes AIO and standard configs.

Root cause is in this agent start script, it can be started
only if node type is worker. But for AIO, the node type is controller.
Then pmon will restart it again and again and cause controller degrade
in the end.

Below test for AIO pass
1) Pci-irq-affinity-agent started normally before openstack apply.
After openstack apply, related openstack config applied to
agent config file as expected.
2) Verified agent started normally in non-openstack worker node for
both AIO and multi-node.
No degrade in controller node.

Change-Id: I73e9dff0358b7ed86bfaaadac834e19fe227892f
Closes-Bug: #1828877
Signed-off-by: zhipengl <zhipengs.liu@intel.com>
2019-05-15 09:14:46 +00:00
Steven Webster
ed50f0230d Include SRIOV i40evf-rt, ixgbevf-rt modules
To enable the SRIOV device plugin feature on a low-latency system,
the i40evf-rt and ixgbevf-rt drivers need to be included in the
host image.

Note: these modules should not be included in a non low-latency
system and have been filtered out:

Depends-On: https://review.opendev.org/659134
Closes-Bug: #1828848

Change-Id: I3bcf4269572c5cafb1c93de2e413e228a56a0988
Signed-off-by: Steven Webster <steven.webster@windriver.com>
2019-05-14 13:20:52 -05:00
Stefan Dinescu
20ca6a6167 Container Openstackclient wrapper
With openstackclients moving to a container, issuing openstack
command to the application side would become difficult due to
log "kubectl" commands and the random nature of pod names.

This commit introduces a wrapper for the containerized client
that automatically passes the desired command to the pod.

This commit also introduces a wrapper for copying files dirrectly
to clients container for commands that need filesystem access
(for example creating images with "openstack image create").

We also alias the default openstack command to the containerized
client. The platform openstack command is aliased to
"platform-openstack".

Change-Id: I7b204bb05381d38f4f25066561e001bb8247943b
Signed-off-by: Stefan Dinescu <stefan.dinescu@windriver.com>
Story: 2005312
Task: 30603
Depends-on: I58a5d511cf54dacc018bfb88848899b92a774087
2019-05-14 17:00:23 +03:00
Zuul
e3778d017e Merge "Add support for multiple helm repositories in helm-upload" 2019-05-10 22:45:18 +00:00
Zuul
a35c3335fc Merge "Implement Pci Interrupt Affinity Agent" 2019-05-10 21:23:30 +00:00
Zuul
4ed0dde890 Merge "Reduce the collectd samples retention period" 2019-05-10 17:56:44 +00:00
Zuul
2c9949fe88 Merge "python-cephclient: delete finished requests" 2019-05-10 17:56:43 +00:00
Daniel Badea
72c3fa95b0 python-cephclient: delete finished requests
ceph-mgr REST API supports synchronous and asynchronous requests.
In asynchronous mode clients can run multiple requests in parallel
then poll to get status of finished requests.

ceph-mgr restful plugin keeps a list of requests that were initiated
by the client and forwarded towards ceph-mgr. It expects the client
to delete finished requests after retrieving current status.

python-cephclient is making synchronous requests (using POST to
"/request?wait=1") but the server is converting them asynchronus
then polls for status on its side. So after getting a response back
the client is still expected to DELETE "/request?id=..."

Currently it's not doing that and ceph-mgr restful plugin is
accumulating a list of all requests ever made by python-cephclient

Change-Id: If8d5c8b27135fde45116e05bb04b655d9574c5ca
Closes-Bug: 1828549
Signed-off-by: Daniel Badea <daniel.badea@windriver.com>
2019-05-10 12:18:47 +00:00
zhipengl
07a07c6bcc Implement Pci Interrupt Affinity Agent
Create an agent which runs on each worker node to do pci interrupt
affinity work.
nova-sriov installed by this new package instead of old nova-utils.

Below test done and pass, see detailed test spec in story link.
1) deployment test with/without openstack application
2) Periodic audit pci irq affinity
3) Remove VM without sriov pci port
4) Remove VM with sriov pci port
5) Add VM without sriov pci port
6) Add VM with sriov pci port
7) Add VM without pci_irq_affinity_mask
8) Add VM without cpu policy set
9) VM resize test
10) Remove one pci port for VM

Code framework is like below
+------------+        +--------------+        +------------+
|            |        |              |        |            |
|            |        |              |        |            |
|  Agent.py  | -----> | affinity.py  | -----> | driver.py  |
|            |        |              |        |            |
|   Daemon   |        |    Conduct   |        |   Drv      |
|            |        |              |        |            |
+------------+        +--------------+        +------------+

Story: 2004600
Task: 28850
Depends-on: https://review.opendev.org/#/c/640263/
Depends-on: https://review.opendev.org/#/c/654415/
Change-Id: Ie668036efe4d0013fed8cd45805f0321692c76f0
Signed-off-by: zhipengl <zhipengs.liu@intel.com>
2019-05-10 01:43:27 +00:00
Eric MacDonald
904da6755d Reduce the collectd samples retention period
Collectd creates a samples database within the
InfluxDB database which is stored in the rootfs.

The current 4 week retention period is too long
for larger systems and could lead to the rootfs
filling up.

This update reduces that retention perid to 1 week
to protect the rootfs from being filled up with
sample data until the samples database is moved
to a more appropriate location.

Change-Id: Ic59712849fa228f19d15919594d23edc43109a0b
Closes-Bug: 1827301
Signed-off-by: Eric MacDonald <eric.macdonald@windriver.com>
2019-05-09 15:46:58 -04:00
Jerry Sun
d863a1e2a3 Enable Local Docker Registry Image Delete
Change config file to allow controller local Docker registry images to
be deleted.

Story: 2002840
Task: 28621

Change-Id: I636a8e26f92c50ebc2222292cd21f7e7784ed2ac
Signed-off-by: Jerry Sun <jerry.sun@windriver.com>
2019-05-09 15:26:30 -04:00
Zuul
dcbe61a3f0 Merge "Docker logs are not rotating" 2019-05-08 19:01:12 +00:00
Zuul
e9727845df Merge "Remove unused openstack references from patch-restart and syslog" 2019-05-08 18:10:47 +00:00
Zuul
dd3778b138 Merge "Add i40evf driver" 2019-05-08 17:58:31 +00:00
Kristine Bujold
7b98e2679a Docker logs are not rotating
Add docker logs to logrotate.

Change-Id: Icd765ebdad1bad0ab53fdeafae6a447dde318c96
Closes-Bug: 1827322
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
2019-05-08 13:55:42 -04:00
Steven Webster
02466c6104 Add i40evf driver
This commit adds the i40evf driver to the host image, which is
needed to enable the SRIOV kubernetes CNI plugins for i40e NICs.

Testing has been performed to ensure this does not affect any
openstack (VM) based SRIOV operations.

Story: 2005208
Task: 29983
Task: 30054

Change-Id: If4d21f574070708d3f7b2fe76e93591705d2b348
Signed-off-by: Steven Webster <steven.webster@windriver.com>
2019-05-08 10:41:24 -05:00
Al Bailey
160e116555 Fix pylint failures due to keyring
The pylint tox setup failures are because keyring 19.x does not
work with python2.7.
This is now imposed in the tox.ini file.

Closes-Bug: 1828241
Change-Id: Ib6219a508ef843358b46b2f49c4da28d1a8bbec8
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2019-05-08 10:29:34 -05:00
Al Bailey
481f78f5bf Remove unused openstack references from patch-restart and syslog
Most of the openstack processes are containerized so there is no
need for them to be included in the patch restart scripts, or
the syslog configuration and log rotation files.

Story: 2004764
Task: 30668
Change-Id: Ib342fa7b594cdafa5d7c7575044ea28783daf9d0
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2019-05-08 08:01:59 -05:00
Zuul
d6a00726a5 Merge "Keystone DB sync - update syslog-ng for logging" 2019-05-07 20:52:15 +00:00
Zuul
9dbdc8f521 Merge "Change license on collectd and influxdb extensions to Apache 2.0" 2019-05-06 14:45:02 +00:00
Don Penney
f3c9e6640e Fix bug in logmgmt start check
The start function of the logmgmt init script
checks for a valid pidfile to see if the process is
already running. Unfortunately, the code has a couple
of typos that cause the check to fail if the "start"
is called when the process is already running.

This commit fixes the typos.

Change-Id: I5795d23cc9e41a18b62e35bf3df07817522efe52
Related-Bug: 1827326
Signed-off-by: Don Penney <don.penney@windriver.com>
2019-05-03 14:48:56 -04:00
Zuul
c3f3378587 Merge "Remove centos_dev_docker_images.inc" 2019-05-03 17:55:56 +00:00
Erich Cordoba
2e293e0894 Change license on collectd and influxdb extensions to Apache 2.0
Story: 2005542
Task: 30714

Change-Id: If1573deac5f1a0f0cfe8bb8c5fba5b4bb2e1a7c3
Signed-off-by: Erich Cordoba <erich.cordoba.malibran@intel.com>
2019-05-02 17:31:13 -05:00
Zuul
7367da9ed0 Merge "Add Cloud-Provider-Openstack to Build" 2019-05-02 18:42:23 +00:00