559 Commits

Author SHA1 Message Date
Zuul
2b8ad5a4f0 Merge "base: configure curl before first use" 2021-04-08 00:03:11 +00:00
Marcin Juszkiewicz
4328a5b3cc base: configure curl before first use
We want to have all uses of curl behave the same so we ship config file
for it in all images.

So do it at start of image so 'base' makes use of config too.

Change-Id: I95aa64f0b292cd020574dbedd320ef90529cc7a4
2021-04-07 11:55:23 +02:00
Marcin Juszkiewicz
0efc3ce3e4 base: we already install sudo for Debian/Ubuntu
Change-Id: I50e5355bd7dd96a5eeaa004c5fc230d1cb8e8fd2
2021-04-06 15:33:01 +00:00
Zuul
ce1cb06268 Merge "base: Add lsof for healthcheck_socket" 2021-03-26 20:36:19 +00:00
Zuul
20aad9dc48 Merge "base: print release version in case of running unsupported one" 2021-03-25 22:10:46 +00:00
Zuul
ea51b70a13 Merge "rabbitmq: bump Erlang requirements to v23 in Debian" 2021-03-25 19:39:08 +00:00
Michał Nasiadka
5b1f4eb698 base: Add lsof for healthcheck_socket
healthcheck_socket script requires lsof to check pids of processes
using that socket - that's needed for octavia-driver-agent healthchecks
in Kolla-Ansible.

Change-Id: I0f14c0f67fd81f13c58f231f800829ad759cc54c
2021-03-24 17:40:50 +01:00
Marcin Juszkiewicz
8c49148864 rabbitmq: bump Erlang requirements to v23 in Debian
Current rabbitmq requires erlang v22.3+ while we had v22.1 on aarch64.

On x86-64 we use upstream repo so have both v22 and v23.

On AArch64 we use Linaro backports repo so have v23.2

Change-Id: I21c3a0e423235301269d0a5b212d0ce6970ab88f
2021-03-24 09:28:01 +00:00
Marcin Juszkiewicz
cc5c164eb8 Debian: install newer firmware for qemu/aarch64
Nova looks into /usr/share/qemu/firmware/ directory. We need 2020.05
version of firmware package for it.

ERROR: nova.exception.InternalError: Failed to locate firmware descriptor files
Change-Id: I546bcf910e886eb75ece62f6be382fe1187efb32
2021-03-23 23:12:00 +01:00
Marcin Juszkiewicz
be544f4e66 switch to CentOS 8 Stream
We use official CentOS 8 Stream image as a base.

So far Stream does not have own SIG repos
RDO Wallaby packages will be built against Stream

Kolla CI jobs are renamed to centos8s. Kolla-Ansible ones will follow
in later patch.

Change-Id: Ibb23c9d0caf115fcbba7bd322ecebe8d80b6821a
2021-03-10 11:15:18 +01:00
Doug Szumski
c19a57a979 Upgrade from ELK6 to ELK7 FOSS release
Upgrade to the latest ELK OSS release. Due to the licensing change, this
effectively pins us to 7.10.x which is supported until 2022-05-11 [1].

This patch also takes advantage of the ARM artifacts which have
been kindly provided since the 7.8.0 release [2].

[1] https://www.elastic.co/support/eol
[2] https://www.elastic.co/blog/elasticsearch-on-arm

Change-Id: Icfa3db5788b25f70ee75411dbaf20d8d4a6a734b
2021-03-05 09:58:43 +01:00
Marcin Juszkiewicz
ad3c18608f base: print release version in case of running unsupported one
User may provide own base image but it needs to be supported one. If it
is not then we print info and error out.

So provide user with info which release is present in their image.

Change-Id: I0d8417fdeb2f02138d03cc57bd3b7b95bc43fc8f
2021-02-15 14:06:05 +01:00
Zuul
60bb16520c Merge "Make kolla_set_configs open files in binary mode" 2021-02-15 11:34:36 +00:00
Radosław Piliszek
8e3027c542 Make kolla_set_configs open files in binary mode
This fixes comparisons when files are not Unicode-encoded.

A relevant unit test is included.
It can be used as a base for other _cmp_file method unit tests
if the need arises.

Change-Id: Ic638516eb92d24ad247a7866fd1b5e2ac0400388
Closes-Bug: #1913952
2021-02-10 15:57:58 +01:00
Marcin Juszkiewicz
0e2c4f61f5 Debian: use libvirt 7.0.0 from Linaro OBS repository
Nova requires libvirt 6.0.0 or newer in Wallaby. So I backported 7.0.0
from Debian 'testing'.

AArch64 and x86-64 are covered.

Change-Id: Ia678cd43b5ce53aee528674280f674c596a278ff
2021-02-06 11:25:47 +01:00
Marcin Juszkiewicz
f782157245 base: drop Linaro OBS key for CentOS and Ubuntu
Debian still needs it but it is enabled for all architectures.

Change-Id: I6a3ba599af3deedd450b4ffff6b86a0c04628340
2021-01-25 18:20:40 +01:00
Marcin Juszkiewicz
9427122423 base: Debian has QEMU 5.2 now so update pinning
Change-Id: I4371daf7b28c8a057364d17e2e47f6e22665447c
2021-01-21 16:51:47 +01:00
Zuul
1d36ae4648 Merge "centos: do not enable PowerTools repo in base" 2020-12-17 12:19:36 +00:00
Zuul
0dd767c2de Merge "Add daxio for nvdimm scenario" 2020-12-16 10:34:02 +00:00
Martin Schuppert
6176d54179 Add daxio for nvdimm scenario
nova-compute uses daxio to cleanup vpmem backend device on instance
delete. If the daxio binary is missing in the nova-compute container
instance delete fails. daxio is provided in centos via daxio, in
ubuntu via the pmdk-tools package.

Change-Id: Ifb5948653565e2ae902783762e20e33527020efe
Closes-Bug: 1907124
2020-12-15 14:22:12 +01:00
Marcin Juszkiewicz
6cc97876cc centos: do not enable PowerTools repo in base
This repo is only needed for source openstack-base.

Change-Id: I835c18424153f8529a37fc78e16d2becacb71263
2020-12-11 19:58:55 +00:00
abraden
17799d6eb6 Support CentOS 8.3
Backport this until Train.

CentOS 8.3 renamed system repos. [1]
This caused failures when disabling and enabling.
More details in the referenced bug report.

Notice CentOS 8.2 and below will no longer work.

[1] https://wiki.centos.org/Manuals/ReleaseNotes/CentOS8.2011#Yum_repo_file_and_repoid_changes

Change-Id: I2d67b3ed7b7c521bc755ec9a87365c9962a9eeff
Closes-Bug: #1907213
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Depends-On: https://review.opendev.org/766211
2020-12-09 16:29:24 +00:00
Michał Nasiadka
22c32a30e6 Unify curl options
Currently we use couple of curl options throughout Dockerfiles, this change
adds all common options to curlrc (-sSLf) and removes usage of those in
Dockerfiles.

Change-Id: I46b77978926fc2b578a68d1aaa944b2198af0685
2020-12-02 18:48:22 +01:00
Marcin Juszkiewicz
753623b29b base: fix QEMU dependencies for Debian/aarch64
Change-Id: I2c82c6269b2c357b0815ea29bc6a7c322f84dac9
2020-11-23 09:44:06 +00:00
Marcin Juszkiewicz
6f1ff63ba9 base: use Victoria repos for Debian/x86-64
Change-Id: Ie5ae119dcf31e05112405c8a307bcd2613d45de8
2020-10-23 17:54:08 +00:00
Radosław Piliszek
032804e5a0 Be strict about errors in start
Make start.sh run with pipefail and nounset to avoid common errors
in the start scripts and detect them early.

Httpd code had to be patched to allow it to pass on Debuntu.

Also fix the two missed applications of httpd to make sure all
its path are covered.

And also fix Horizon's ENABLE_ZAQAR - K-A does not use Zaqar.
Yet another - Horizon's settings_bundle. :-)

Finally, fix Neutron for Debuntu (KOLLA_LEGACY_IPTABLES).

Change-Id: I39b8d78f6758df1f92b8b0d2c06ea99b038b843b
Depends-On: https://review.opendev.org/711923
2020-10-05 13:03:39 +02:00
Zuul
2e094adbbb Merge "Debian: use QEMU 5.0 from buster-backports" 2020-10-01 10:46:41 +00:00
Marcin Juszkiewicz
87730f8f7c create 'add_binary_source_envs' macro and use it
There is small subset of images where we need to know which install_type
or install_metatype is used. So add them only there (and their
children).

Change-Id: Ib7d5e36b958d6c8daf2989df32e29fa24b46c62a
Implements: blueprint infra-images
2020-09-21 12:10:57 +00:00
Gaël THEROND (Fl1nt)
b93c40a363 Improve pip install process for offline deployment.
* "Use distribution-provided pip"
    * "Use python's pip module invocation method"
    * "Install pip earlier in order to avoid multiple installation"
    * "Remove pip_version variable requirement and call"

Change-Id: Id0e738044a1931f9d611a7281a48ea4a593f1cf1
Closes-bug: #1893204
2020-09-17 13:50:15 +00:00
Mark Goddard
c5320eb223 Fix kolla_set_configs --check with a directory
There are several issues with kolla_set_configs --check:

1. We calculate the destination path incorrectly when comparing a file
   in a directory, due to passing arguments to os.path.relpath in the
   wrong order
2. For directories that have not changed, we also attempt to compare
   them as files, which fails when they are open()ed.
3. If the config JSON does not have a config_files key, it fails with a
   KeyError.

The first two issues affect the fluentd container, which specifies
directories as the source, without using a glob. The third affects OVN
containers.

This patch fixes these issues.

Closes-Bug: #1890567

Change-Id: I8921befe51da4282121443849177a7ca5ebe8822
2020-08-28 15:01:25 +00:00
Marcin Juszkiewicz
84038e337a Debian: use QEMU 5.0 from buster-backports
This will allow us to make sure of all updates.

'qemu-kvm' got dropped in Debian

Change-Id: If09a8188030baa284dd353b1c3f830d78c5091b7
2020-08-24 09:57:54 +02:00
Marcin Juszkiewicz
d2966452c5 ubuntu: move to 20.04 Focal
There is a time once every 2 years when ubuntu team releases new LTS
release. And then UCA joins with binary packages for current OpenStack
development cycle.

It is this time for Ubuntu 20.04 'focal'.

Depends-On: https://review.opendev.org/745156
Change-Id: I045aa6b4b4fd83fbe7d1fda89549f0ef1e88ec12
2020-08-07 14:38:02 +00:00
Zuul
acd0d4bf20 Merge "fluentd: migrate to td-agent 4 where possible" 2020-07-22 12:55:09 +00:00
Marcin Juszkiewicz
c19a222b4c fluentd: migrate to td-agent 4 where possible
td-agent 4 got released for all distributions we support. For both
x86-64 and aarch64 architectures.

Change-Id: I86ecdc1ac45dfd76b94a34d1b837f7c0bc975dcd
2020-07-20 12:58:49 +02:00
Zuul
c8679b4ac4 Merge "Upgrade to Logstash 6" 2020-07-10 09:36:22 +00:00
Doug Szumski
25f74fb943 Upgrade to Logstash 6
Elasticsearch and Kibana 6 are used on all distros, but Logstash
is still pinned at 2 which is not compatible. See the product
compatibility matrix [1].

[1] https://www.elastic.co/support/matrix#matrix_compatibility

Partial-Bug: #1884090
Change-Id: I1579e762bcfb07f0baa73507eb9955ae2f83c4ec
2020-07-07 13:16:00 +00:00
Michal Nasiadka
6c77d52cda Remove opendaylight image
It has been deprecated in Ussuri cycle.

Change-Id: I6d3174e8b52cb4d1f2eca4d85d506665dafe2406
2020-07-07 12:50:38 +02:00
Zuul
4f4b0126a9 Merge "Remove sensu images" 2020-07-07 08:50:48 +00:00
Michele Baldessari
dc2ddfa975 Drop systemd support from nsswitch.conf on RHEL-based distros
A bit like we did for I3e0e86026f5a4a78473bed824cd1682d3a020cd5 we
should remove the nss-systemd lookup from containers. The reasons for
this are as follows:
1) Just like for I3e0e86026f5a4a78473bed824cd1682d3a020cd5
when this nss module is triggered it tries to talk to dbus.
It triggers a bunch of selinux denials and it makes little sense
to open all containers to talk to dbus.
In particular, if a container is run as non-privileged and bind-mounts
/run from the host, we will hit selinux denials like the following:

  type=USER_AVC msg=audit(1592337775.860:74119): pid=1284 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc:  denied  { send_msg } for msgtype=method_call interface=org.freedesktop.DBus member=Hello dest=org.freedesktop.DBus spid=406228 scontext=system_u:system_r:container_t:s0:c162,c886 tcontext=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 tclass=dbus permissive=0  exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'UID="dbus" AUID="unset" SAUID="dbus"

2) It just makes little sense in a kolla-world to have containers
talk to dbus/systemd and it saves us some time when a lookup triggers
the systemd module for whatever reason. Especially because the
nss-systemd module does a few things which are not useful in a container
(ensures that the root and nobody users and groups remain resolvable,
SystemD's DynamicUser= feature, provide Lookup API via Varlink)

The sed regex gives us the wanted results:
$ diff -u /etc/nsswitch.conf.orig /etc/nsswitch.conf
--- /etc/nsswitch.conf.orig     2020-06-19 07:18:10.974580755 +0000
+++ /etc/nsswitch.conf  2020-06-19 07:20:12.260230103 +0000
@@ -53,9 +53,9 @@
 # group:     db files

 # In order of likelihood of use to accelerate lookup.
-passwd:      sss files systemd
+passwd:      sss files
 shadow:     files sss
-group:       sss files systemd
+group:       sss files
 hosts:      files dns myhostname
 services:   files sss
 netgroup:   sss

Related-Bug: #1883849

Change-Id: I81e5b7abf4571fece13a029e25911e9e4dece673
2020-07-06 12:16:48 +02:00
Marcin Juszkiewicz
0cfd36292c Remove sensu images
They have been deprecated in Ussuri [1].

[1]: https://review.opendev.org/#/c/711636/

Change-Id: I0cfde46c29eedce77c70c1c677220814a801ff3a
2020-07-06 11:50:12 +02:00
Zuul
1d4d23cf0e Merge "CentOS/AArch64: RabbitMQ is back!" 2020-07-01 17:36:42 +00:00
Zuul
5b4e00decc Merge "Change lang package block order" 2020-06-26 15:16:55 +00:00
Marcin Juszkiewicz
b2c86fd3ad CentOS/AArch64: RabbitMQ is back!
CentOS 8.2 has Erlang and RabbitMQ available in 'messaging/rabbitmq-38'
repository. We use it to grab Erlang while RabbitMQ comes from upstream
(like on x86-64).

Change-Id: I2559267d120081f2e5eabc9d966b019517a5ad5d
2020-06-23 09:41:38 +00:00
Zuul
2b140296fc Merge "base/centos: take care of whitespaces" 2020-06-17 04:28:50 +00:00
Gaël THEROND
f9def021d8 Change lang package block order
Fix offline installation of language pack packages.
Change-Id: I9f4a21bbf1c8383e21e59350206aba3424602270
Closes-bug: #1883233
2020-06-12 15:42:38 +00:00
Alfredo Moralejo
e37e876c82 Use CentOS official mirrors for Ceph packages
It's still using temporary mirror in RDO infra, but now that packages
are properly synced to CentOS mirrors, let's switch to use it.

Change-Id: I913efffe6a1d8a0210b1158261c77d0d45ac3147
2020-06-02 15:45:04 +02:00
Marcin Juszkiewicz
09604a7492 base/centos: take care of whitespaces
Let's get rid of this warning:

Change-Id: I121e2ecb30ec6e8d1b6d88d9b921894438c3e094
INFO:kolla.common.utils.base:ESC[91m[WARNING]: Empty continuation line found in:
INFO:kolla.common.utils.base:    RUN dnf -y install centos-release-opstools epel-release dnf-plugins-core     && dnf config-manager --enable PowerTools     && dnf config-manager --disable centos-ceph-nautilus     && dnf config-manager --disable centos-nfs-ganesha28     && dnf config-manager --disable influxdb && rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-OpsTools && dnf -y distro-sync --security --sec-severity=Important --sec-severity=Critical
INFO:kolla.common.utils.base:[WARNING]: Empty continuation lines will become errors in a future release.
INFO:kolla.common.utils.base:ESC[0m
INFO:kolla.common.utils.base:Step 1/39 : FROM centos:8
2020-05-17 17:38:10 +02:00
Marcin Juszkiewicz
1dfd937cba CentOS: disable more repos
'messaging-rabbitmq' and 'opstools' are enabled by RDO
delorean-deps.repo file

'influxdb' was also enabled

Depends-On: https://review.opendev.org/728687

Change-Id: Ibb0a7edc9e5632c1b89a7d52601f55a223b49dfb
2020-05-17 11:34:12 +00:00
Marcin Juszkiewicz
62c8dc95bd Debian: use Ussuri binary packages
There is a repo with binary packages for Ussuri. Let us use them instead
of Train ones.

amd64 only

Change-Id: I071eebc4f60094d881a68e201e8c3bb6e6742ff5
2020-04-29 23:09:06 +02:00
Marcin Juszkiewicz
345f0555bc kibana/debian: use upstream package on all architectures
Upstream kibana package ships x86_64 binaries of NodeJS. I usually
repacked package without them to have something for other architectures.

Instead let us install upstream. Then on non-x86 archs we add nodejs
from distribution and remove x86-64 binaries.

Change-Id: Ia9feac726a60250215b1bc78bf90dc68ac6f956a
2020-04-17 10:46:44 +02:00