1356 Commits

Author SHA1 Message Date
Yue Tao
9f2dd136a9 k8s-pod-recovery: use src_path instead of dl_hook
The src_path points to the local source directory.

Story: 2009101
Task: 43697

Signed-off-by: Yue Tao <yue.tao@windriver.com>
Change-Id: I869a5e0d9aefb88cae78856bbd076778b0b7f37f
2021-11-17 17:59:10 +08:00
Zuul
722295007a Merge "Add debian package for libfdt" 2021-11-16 21:54:45 +00:00
Zuul
02845aa5e1 Merge "Add debian package for armada" 2021-11-16 21:48:14 +00:00
Zuul
06c317eb9e Merge "Add debian package for systemd" 2021-11-16 19:03:57 +00:00
Zuul
57ecb4ba1e Merge "Add debian package for dhcp" 2021-11-16 18:28:27 +00:00
Zuul
1f709e8338 Merge "Add debian package for dnsmasq" 2021-11-16 14:58:32 +00:00
Zuul
e395e6d8d1 Merge "networking: Update keepalived from 1.3.5 to 2.1.5" 2021-11-15 19:11:42 +00:00
Zuul
0d26d608fb Merge "Upgrade trident spec files for 21.04.1" 2021-11-15 16:08:58 +00:00
Zuul
7a28e31390 Merge "debian_pkg_dirs: add python-docker and lighttpd" 2021-11-15 16:01:12 +00:00
Yue Tao
6fb79d4dcf Add debian package for libfdt
Story: 2009221
Task: 43441
Signed-off-by: Yue Tao <yue.tao@windriver.com>
Change-Id: I1da499de95457726f87f35821d8720a0b05c4431
2021-11-15 13:41:41 +08:00
Zuul
060c0856c9 Merge "Add debian packages for ostree" 2021-11-15 04:22:36 +00:00
Yue Tao
5c5d01915a Add debian package for systemd
Porting 851/909/910/913/919/920/922/923/924/925/926/927
patches to debian, other patches are confirmed in
debian version.

Story: 2009221
Task: 43416
Signed-off-by: Yue Tao <yue.tao@windriver.com>
Change-Id: I5af677c90342bae7c30991bf465e0db79c71667d
2021-11-14 15:52:18 +08:00
M. Vefa Bicakci
6da5c5ecbe networking: Update keepalived from 1.3.5 to 2.1.5
This commit updates keepalived from v1.3.5 to v2.1.5 to avoid failures
encountered when building StarlingX flock container images, which
started to fail with the following errors after the recent iptables
update introduced by commit 36673774ee3c ("iproute-5.12, iptables-1.8.4,
and libnftnl-1.1.5", 2021-10-27):

=== 8< ===
Error: Package: keepalived-1.3.5-19.el7.x86_64 (base)
       Requires: libxtables.so.10()(64bit)
       Available: iptables-1.4.21-35.el7.x86_64 (base)
           libxtables.so.10()(64bit)
       Installing: iptables-1.8.4-21.tis.5.x86_64 (stx-mirror-distro)
           Not found
=== >8 ===

keepalived-2.1.5 was imported from CentOS 8-Stream where it is the
latest version as of this writing. It should be noted that rebuilding
keepalived-1.3.5 (i.e., CentOS 7's version) was not suitable as
keepalived-1.3.5 does not support iptables-nftables, and the CentOS
8-Stream keepalived RPM cannot be used as is due to the specific
versions of some of its dependencies.

During the preparation of this patch, an unexpected build failure had to
be worked around by disabling SNMP support in keepalived, which is
assumed to not have a negative impact on StarlingX according to a
software architect colleague at Wind River. Please see the description
of the patch named "keepalived.spec-Disable-dependency-on-snmp.patch"
for further details regarding the build failure.

Verification:
- Layered and monolithic StarlingX master branch builds succeed.
- StarlingX container builds, which used to fail without this commit,
  succeed as well.

Closes-Bug: #1950513

Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: I59bd7d4f8ed89c73248ecd97e6985f91b88c4623
2021-11-12 12:12:16 -05:00
Yue Tao
ddf1b6eca6 Add debian package for dhcp
Porting 3 source patches for CentOS

dhclient-ipv6-bind-to-interface.patch
dhclient-ipv6-conditionally-set-hostname.patch
dhclient-restrict-interfaces-to-command-line.patch

Ignore 0001-dhcp-set-the-prefixlen-to-64.patch, which disables
a patch from dhcp-4.2.5-82.el7.centos.src.rpm.

Porting dhclient-dhcp6-set-hostname.patch based on the
dhclient-script.linux of Debian package.

Story: 2009221
Task: 43440
Signed-off-by: Yue Tao <yue.tao@windriver.com>
Change-Id: I457230bc08a23abb1efa1a7619c0cba8ff4d7967
2021-11-12 08:58:34 +08:00
Joe Slater
b75f375c62 sriov-network-device-plugin: update repo commit id
Set SOURCE_REF to the latest release.

[Submitted on behalf of Vefa Bicakci.]
Closes-Bug: #1950513
Change-Id: Id9a0ac2e2c62fa4c829e280cd740fab1f40a2131
Signed-off-by: Joe Slater <joe.slater@windriver.com>
2021-11-11 21:57:40 +00:00
Zuul
79355056dc Merge "Revert "Add centos package for puppet-postgresql module"" 2021-11-11 15:32:02 +00:00
Daniel Stevens Torres Cardenas
b36ef4fab8 Revert "Add centos package for puppet-postgresql module"
This reverts commit d27206ec923f814018b3cde4e1106007c6cbac69.

Reason for revert: We have a red sanity and we believe it this is the commit that is causing ansible to fail.

Change-Id: Ia0ef3bb302be88e04849cd0343fd849895c455f0
2021-11-11 15:04:18 +00:00
Daniel Safta
db16f48a95 Add debian package for armada
Create debian package structure
for armada and armada-helm-toolkit.

Story: 2009221
Task: 43917
Change-Id: Ie1177d50b1c22239be18c109129308d13d75ba28
Signed-off-by: Daniel Safta <daniel.safta@windriver.com>
2021-11-11 16:28:11 +02:00
Zuul
82bad921f3 Merge "Add debian package for cluster-resource-agents" 2021-11-10 21:44:26 +00:00
Yue Tao
969edcff63 Add debian package for dnsmasq
Porting 2 patches
close-tftp-sockets-immediately.patch
dnsmasq-update-ipv6-leases-from-config.patch

Story: 2009221
Task: 43439
Signed-off-by: Yue Tao <yue.tao@windriver.com>
Change-Id: Ia1ad6965f6b8170252edf020733b4ce2b12b110c
2021-11-10 14:41:56 -05:00
Vinicius Lopes da Silva
892be55bbb Upgrade trident spec files for 21.04.1
Upgrade Trident to 21.04.1 version.
This supports Kubernetes 1.11 until 1.21.

Story: 2009654
Task: 43816
Signed-off-by: Vinicius Lopes da Silva <vinicius.lopesdasilva@windriver.com>
Change-Id: Ibd5bcfb9deb6b22e443d62cbfbbadde2c8fb3ded
2021-11-10 18:36:03 +00:00
Zuul
6a4a36844b Merge "Backport Kubernetes log level changes to reduce Kubelet log volume." 2021-11-10 17:00:30 +00:00
Zuul
a5674d5141 Merge "Add centos package for puppet-postgresql module" 2021-11-10 17:00:27 +00:00
Gleb Aronsky
43dfcca67f Backport Kubernetes log level changes to reduce Kubelet log volume.
Reduce Kubelet log volume by changing frequent log messages
from INFO to DEBUG by backporting Kubernetes commit
8f08db9164b9038b2a62ad3d0290d725bc860744.

Test Plan:  Verify that the Kubelet log volume is reduced during
execution

Pass:  Verified that the Kubelet log volume is greatly reduced by
examining the daemon.log file. Previously informational messages
relating to reconcileState no longer appear at the default log level.

Story: 2009272
Task: 43858

Change-Id: Ibfb2967f5b643269203fc91d884964f93e091d34
2021-11-09 15:48:41 -05:00
Roberto Nogueira
d27206ec92 Add centos package for puppet-postgresql module
This package is 3rdparty.
All patches were changed and lifted.
Did build puppet-postgresql.

Story: 2009242
Task: 43883

Signed-off-by: Roberto Nogueira <robertoluiz.martinsnogueira@windriver.com>
Change-Id: I1d473e34c703c6355bee4f33daf6ce12b71b4d19
2021-11-09 19:57:09 +00:00
Yue Tao
fcfb04d621 Add debian package for cluster-resource-agents
Except for Do-not-log-at-debug-log-level-when-HA_debug-is-unset.patch,
which is merged upstream, others local patches have been ported.

Depends-On: https://review.opendev.org/c/starlingx/tools/+/817010

Story: 2009221
Task: 43310

Signed-off-by: Yue Tao <yue.tao@windriver.com>
Change-Id: Id602f7b9dac55b06e46337fb800a865d3a82d701
2021-11-09 16:12:44 +08:00
Yue Tao
f41fbf7df7 debian_pkg_dirs: add python-docker and lighttpd
Story: 2009221
Task: 43608

Signed-off-by: Yue Tao <yue.tao@windriver.com>
Change-Id: I58671b8e14d2fee3400ac6aa54df3255d45d1530
2021-11-09 15:51:31 +08:00
Zuul
4eecd1f723 Merge "Add debian package for k8s-pod-recovery" 2021-11-08 19:49:16 +00:00
Zuul
f2ddb9a428 Merge "Add debian package for crictl" 2021-11-08 15:04:23 +00:00
Mihnea Saracin
3d5e021578 Add debian package for k8s-pod-recovery
- Built the package
- Built the iso
- Verified if the package was installed correctly

Story: 2009101
Task: 43697
Change-Id: Id10c87f9a24329b533864411075e096f56dbcd8b
Signed-off-by: Mihnea Saracin <Mihnea.Saracin@windriver.com>
2021-11-08 12:16:39 +02:00
Zuul
551f4b043a Merge "Add debian package for lighttpd" 2021-11-05 21:25:14 +00:00
Zuul
29169425a1 Merge "Add debian package for etcd" 2021-11-05 18:04:16 +00:00
Zuul
1f98c9855f Merge "Add debian package for runc" 2021-11-05 16:22:55 +00:00
Zuul
aaae5c2340 Merge "Add Debian packaging for helm and chartmuseum" 2021-11-05 15:18:46 +00:00
Zuul
f1955eff93 Merge "Add debian package for cni & isolcpus" 2021-11-05 05:31:04 +00:00
Zuul
c606892b4c Merge "Add debian package for containerd" 2021-11-04 17:35:12 +00:00
Zuul
4bc0c7dd3d Merge "Implement CNI cache file cleanup for stale files" 2021-11-04 17:22:43 +00:00
Zuul
b61ae69b36 Merge "Add debian package for docker-distribution" 2021-11-04 17:22:34 +00:00
Zuul
32bc1aa383 Merge "Add debian support for puppet-ceph" 2021-11-04 17:15:11 +00:00
Zuul
0ea8188869 Merge "Add debian package for python-docker" 2021-11-04 17:04:17 +00:00
albailey
6420975496 Updating the pylint tox target to reflect its targets
pylint is currently only running on storage_topology
however it was using generic dependencies and installing
and suppressing far more than it needed to.

This will get pylint to pass zuul as well, since some
of the un-needed dependencies are failing to install.

Story: 2008943
Task: 43850
Signed-off-by: albailey <Al.Bailey@windriver.com>
Change-Id: I250754ca8f4f904a902f6cfd6597fde54d1597a9
2021-11-04 10:22:24 -05:00
Daniel Safta
d37fe03fed Add debian package for cni & isolcpus
CNI was packaged in CentOS using
version 0.8.1 and now it is using
the stable debian version 0.9.0.

Story: 2009221
Task: 43783
Signed-off-by: Daniel Safta <daniel.safta@windriver.com>
Change-Id: Iaba754b304a26b01c048e8bc05de51ace1e88246
2021-11-04 11:36:04 +02:00
Daniel Safta
f7309453d4 Add debian package for crictl
Crictl was previously packaged
with containerd and runc.
It is now packaged separately.

Story: 2009221
Task: 43782
Signed-off-by: Daniel Safta <daniel.safta@windriver.com>
Change-Id: If9b46283bed424bcac630347fad4af4b2e75975b
2021-11-04 11:27:01 +02:00
Charles Short
d0ddf30a4c Add debian support for puppet-ceph
Add debian infrastructure to build puppet-ceph as a debian packaging.

Patches Dropped:

0005-Remove-puppetlabs-apt-as-ceph-requirement.patch
0004-US92424-Add-OSD-support-for-persistent-naming.patch
0006-ceph-disk-prepare-invalid-data-disk-value.patch
0008-ceph-mimic-prepare-activate-os.patch
0009-fix-ceph-osd-disk-partition-for-nvme-disks.patch

Re-diffed:

0001-Roll-up-TIS-patches.patch
0002-Newton-rebase-fixes.patch
0003-ceph-jewel-rebase.patch
0004-US92424-Add-OSD-support-for-persistent-naming.patch
0005-Add-StarlingX-specific-restart-command-for-Ceph-moni.patch

The patches that were dropped needs to be re-worked for ceph-volume,
since ceph-disk has been deprecated and not included in the Ceph version
from Debian.

0001-Roll-up-TIS-patches.patch and 0003-ceph-jewel-rebase.patch
were rebased because most of the systemd logic has
been dropped as well and need to be redone.

Story: 2009101
Task: 43431

Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I5a66bcd274f2752d4c050fab25a7b1b8347b650e
2021-11-03 10:28:15 -04:00
Daniel Safta
b9d28e3c56 Add debian package for docker-distribution
- created required debian structure
- successfully built the package
- applied stx config changes and updated
paths from /etc/docker to /etc/docker-distribution
through patches

Story: 2009221
Task: 43631
Signed-off-by: Daniel Safta <daniel.safta@windriver.com>
Change-Id: I5af677c90342bae7c10101bf100e1db79c716670
2021-11-03 13:11:40 +02:00
Tracey Bogue
f98b1a5b40 Add Debian packaging for helm and chartmuseum
These packages require the following files in a local mirror:
* chartmuseum-0.12.0.tar.gz
* chartmuseum-v0.12.0-amd64
* helm-v3.2.1-linux-amd64.tar.gz

Story: 2009101
Task: 43715

Signed-off-by: Tracey Bogue <tracey.bogue@windriver.com>
Change-Id: I85c854f1fb62efa993a030fe06105bb2c0751165
2021-11-02 06:48:57 -05:00
Zuul
73c793c973 Merge "Reduce log rates for daemon-ocf" 2021-11-01 17:15:58 +00:00
Steven Webster
5d1a26b89d Implement CNI cache file cleanup for stale files
It has been observed in systems running for months -> years
that the CNI cache files (representing attributes of
network attachment definitions of pods) can accumulate in
large numbers in the /var/lib/cni/results/ and
/var/lib/cni/multus/ directories.

The cache files in /var/lib/cni/results/ have a naming signature of:

<type>-<pod id>-<interface name>

While the cache files in /var/lib/cni/multus have a naming signature
of:

<pod id>

Normally these files are cleaned up automatically (I believe
this is the responsibility of containerd).  It has been seen
that this happens reliably when one manually deletes a pod.

The issue has been reproduced in the case of a host being manually
rebooted.  In this case, the pods are re-created when the host comes
back up, but with a different pod-id than was used before

In this case, _most_ of the time the cache files from the previous
instantiation of the pod are deleted, but occasionally a few are
missed by the internal garbage collection mechanism.

Once a cache file from the previous instantiation of a pod escapes
garbage collection, it seems to be left as a stale file for all
subsequent reboots.  Over time, this can cause these stale files
to accumulate and take up disk space unnecessarily.

The script will be called once by the k8s-pod-recovery service
on system startup, and then periodically via a cron job installed
by puppet.

The cleanup mechanism analyzes the cache files by name and
compares them with the id(s) of the currently running pods. Any
stale files detected are deleted.

Test Plan:

PASS: Verify existing pods do not have their cache files removed
PASS: Verify files younger than the specified 'olderthan' time
      are not removed
PASS: Verify stale cache files for pods that do not exist anymore
      are removed.
PASS: Verify the script does not run if kubelet is not up yet.

Failure Path:

PASS: Verify files not matching the naming signature (pod id
      embedded in file name) are not processed

Regression:

PASS: Verify system install
PASS: Verify feature logging

Partial-Bug: 1947386

Signed-off-by: Steven Webster <steven.webster@windriver.com>
Change-Id: I0ce06646001e52d1cc6d204b924f41d049264b4c
2021-11-01 10:39:39 -04:00
Zuul
60c72b6bb3 Merge "Add debian_iso_image.inc file" 2021-11-01 14:21:19 +00:00
M. Vefa Bicakci
36673774ee iproute-5.12, iptables-1.8.4, and libnftnl-1.1.5
This commit updates iproute from 5.9 to 5.12, iptables from 1.4.21 to
1.8.4, and libnftnl from 1.0.8 to 1.1.5:

- iproute 5.9 does not make use of libbpf, which causes the 'tc' utility
  (provided by iproute-tc) to report BTF debugging symbol-related
  warnings when eBPF programs are used with tc by the kernel's eBPF
  sample test programs, even though the programs appear to work: "BTF
  debug data section '.BTF' rejected: Invalid argument (22)!".

- iptables 1.4.21 does not support the --object-pinned option, which is
  required to be able to use eBPF programs to match packets.

- libnftnl >= 1.1.5 is a dependency for recent versions of iptables, and
  the version of libnftnl in StarlingX's CentOS 7 is 1.0.8.

The versions which are used by this commit are the latest versions in
CentOS 8-Stream as of this writing.

Notes:
- iptables software package bundles a version of ebtables different than
  the legacy version already included in StarlingX. The legacy version
  supports the broute table and the BROUTING chain and string matching,
  whereas the iptables version does not. The legacy version is
  deprecated by this commit based on feedback received from colleagues,
  mainly to avoid unexpected incompatibilities between ebtables-legacy
  and iptables' netfilter/nft-based versions.

Verification:
- All-in-One simplex installation and bootstrap was carried out
  successfully.
- Installation and bootstrap was successful on two separate systems: One
  system consisting of 2 controller hosts, 4 compute hosts and 2 storage
  hosts, and another system consisting of 2 controller hosts and 2
  compute hosts.
- Configuration of aggregated links (after using ifenslave manually) and
  configuration of virtual function (VF) interfaces (also manually set
  up) were carried out with the iproute tools successfully as basic
  sanity tests.
- The results of basic ebtables commands (insertion and removal of DROP
  rules) were observed in "ebtables -L" output and confirmed to take
  effect in a test bed consisting of two network namespaces connected by
  bridged interfaces, as a basic sanity test.
- Sample eBPF test programs and scripts shipped with the v5.10 kernel
  were executed successfully, with the caveat that there is a need to
  install a recent version of LLVM to compile the eBPF test programs.
  (I built LLVM-13.0 from scratch.)

Partial-Bug: #1949217

Depends-On: I24bb7c60e353643add5e63ae7ea7c6516d07c7bf
Depends-On: I12d20797db91fecdac409b0535632ac97bd6ad47
Depends-On: If95c2d24c98cb2add5e24548bc45f505c94b4b79

Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: I63d557112c653d59b88ac3a4798dee0e89246612
2021-10-29 19:47:27 -04:00