Only the final image in a hierarchy should have the footer block.
neutron-infoblox-ipam-agent image still has two footers because it
inherits from the neutron-server image. We set the user to root at the
beginning of that Dockerfile, so that headers and footers have the right
context.
Change-Id: I95e5bb5583be2a76e289749e8adfc303a4e7197a
(cherry picked from commit 4b792b9a43)
backup_full() saves the history name in $BACKUP_DIR/last_full_date.
backup_incremental() attempts to read from $BACKUP_DIR/last_full_date
into $LAST_FULL_DATE. After the read attempt; if $LAST_FULL_DATE is zero
length then $(date +%d-%m-%Y) is used.
Change-Id: I4172b814d5d95ae3debbc1cf3feaeee2e2bec9bd
Closes-bug: 1897948
(cherry picked from commit ccfe65b5f7)
Add mariadb_clustercheck image with xinetd and clustercheck binary for haproxy
checking of galera status.
Reorganise existing mariadb images by creating mariadb-base and mariadb-server.
Existing mariadb image will be deprecated (in a separate change).
Change-Id: Ib601f64e1514521154beeaac40f0c058a7119408
The disable_extra_repos macro accepts a list as its only argument. We
were calling it like this to disable EPEL:
disable_extra_repos('epel')
The macro interpreted this as a request to disable three repos, e, p, l.
Thanks Python! Type validation to be improved separately.
Additionally, on CentOS 8 the EPEL repository was not included in the
repository mapping file, repos.yaml. There is also another EPEL
repository on CentOS 8, epel-modular, which is enabled by default after
installing epel-release.
This change adds mappings for epel and epel-modular repos to repos.yaml,
and fixes the disabling of epel in the base image, as well as disabling
epel-modular.
There are some cases where EPEL is still used (it seemed a bit too
easy...), and the repository has been enabled for these images:
* bifrost-base (nginx)
* ironic-conductor (C7 only, shellinabox)
* freezer-base (C7 only, trickle)
* gnocchi-base (C8 binary only, python3-boto3)
* mariadb (pv)
* mongodb (C7 only, mongodb)
* nova-spicehtml5proxy (C7 only, spice-html5)
* telegraf (C7 only, python2-pip)
A few other things were changed:
* ironic-conductor does not require the ceph repo
* python3-pika is no longer installed in the openstack-base image
Related: blueprint remove-epel
Change-Id: I3761825239dfc462072383cde6276c4fb3e1bf12
Disable external repositories by default and enable only when needed.
Depends-on: https://review.opendev.org/696480
Implements: blueprint repos-off-by-default
Change-Id: Icf2a8397a8349e0fe849d88d160409fd234480a9
Without this, a failure in mariabackup would be ignored due to the pipe
to gzip.
Change-Id: I84536b302991e8fb04d5fa67ea266bc428437ce1
Related-Bug: #1843043
After switching from innobackupex to mariabackup [1] for database
backups, the kolla_mariadb_backup.sh script was not modified to account
for the difference in arguments between innobackupex and mariabackup.
There is a compatibility mode, the documentation [2] for which covers
some of the differences.
The following have been changed:
- Add explicit --backup argument, now required
- Remove './' positional argument - not required with --stream
- Remove --no-timestamp argument - only supported in innobackupex
compatibility mode
- Remove --incremental argument - implied by --incremental-history-name
- Remove deprecated --compress argument, which requires qpress to be
installed (it is not). The stream is now passed through gzip instead
[3]
[1] https://mariadb.com/kb/en/library/mariabackup-overview/
[2] https://mariadb.com/kb/en/library/mariabackup-options/#-innobackupex
[3] https://mariadb.com/kb/en/library/using-encryption-and-compression-tools-with-mariabackup/
Change-Id: I67cff47cbf56570b8eaeb66092dd87c2769fc8a6
Closes-Bug: #1843043
XtraBackup started failing to install on Ubuntu and does not support
MariaDB 10.3 version.
Related-Bug: #1843043
Change-Id: I9cf62c8e03a02574d85e5349df1d23a51d4c99be
MariaDB in Ubuntu packages is backlevel (10.1).
Devstack gates are tested on MySQL 5.7 which is not compatible
with MariaDB 10.1.
Neutron recently introduced a db migration
that breaks on MariaDB 10.1, but works on MariaDB 10.3 [0].
Bumping up version to 10.3 on Ubuntu to match CentOS (RDO) version.
[0]: https://review.opendev.org/#/c/677221/
Closes-Bug: #1841907
Change-Id: I16e4c40208912691e3015bfe2065ea0ce2358edb
This retains the default, but allows the directory to be customised
if required. This is useful, for example, when using Kolla Ansible
to deploy multiple instances of MariaDB.
Partially Implements: blueprint support-nova-cells
Change-Id: I042fbe140ee84f2ed72e27635be247cb145aac5f
Usage of dumb-init can also be disabled from cli with
`--nouse-dumb-init` when building images.
Also moves ENTRYPOINT outside of dumb_init_installation
block.
Change-Id: Id32d54fc9913b83cf121ede5a5e265e0772062ae
Related-Bug: #1821970
While mysql_upgrade seems to work on CentOS images without specifying a
host and port, it fails on Ubuntu without these.
Change-Id: Ie140de57c4fe2f2e4db6a90c18e3535d9cc71ff2
Related-Bug: #1820325
There are a couple of issues with the mariadb image currently.
1. it doesn't include mysql_upgrade on CentOS/RHEL. It is necessary to
run this after an upgrade of MariaDB. This is fixed by installing
mariadb-server-utils.
2. it inherits the standard dumb-init entrypoint from the base image,
which includes a --single-child argument. That causes signals to be
forwarded only to the process executed by dumb-init, which in the
case of kolla-ansible is mysqld_safe, however mysqld_safe ignores
SIGTERM. After a timeout, Docker then forcibly kills the container.
Executing dumb-init without --single-child ensures the TERM signal is
sent to mysqld, allowing it to shutdown gracefully.
Change-Id: I1a450736721863b91be1ea3d3c8948990de9fdeb
Closes-Bug: #1820325
This change updates the docker files to use base_package_type instead
of doing specific distro checks for the rhel/deb generic cases. The
base_distro is still available and is used when a specific distro needs
a customization but if the differences are purely rpm vs deb, then the
base_package_type can be used.
Change-Id: I8d720bb185df65a0178061ccf20b1ab2265da2c5
Mysql-python is a deprecated package and fails to build currently.
Make use of mysqlclient as is a fork of mysql-python with many bug
fixes a python3 support.
percona xtrabackup doenst work with mariadb 10.3,
replaces to use mariadb-backup tool.
https://jira.mariadb.org/browse/MDEV-15774
Change-Id: Icefe3a77fb12d57c869521000d458e3f58435374
Under heavy load, when the security_reset script is being run to set
up root password, it might happen that the mysql server doesn't get a
change to evaluate the SQL statement sent by security_reset.expect
before the hardcoded 10 seconds timeout is reached.
Allow the timeout to be overriden via env variable DB_MAX_TIMEOUT,
which is already used as a timeout variable in extend_start.sh
Change-Id: If82c6daca82a48d899605829dc99cf4a352cddcd
Closes-Bug: #1775864
Ubuntu 18.04 has been released. We need upgrade to it this cycle.
Changes made in this patch
- Bump ubuntu repo from xenial to bionic. Some repos do not have bionic
packages, so still using xenial repos.
- Drop mariadb, percona and nfs-ganesha repo, because these packages are
already provided by Ubuntu official repo.
- Since qdrouterd does not have bionic repo and xenial packages does not
work on Ubuntu bionic, just add it into ignore image list.
- Use python-rtslib-fb and targetcli-fb rather than python-rtslib
and targetcli
- use xen-utils package name rather than with package version one.
- Seems unsigned repo won't work on Ubuntu bionic, stop using infra
mirrored unsigned repo in gate building job.
Needed-By: https://review.openstack.org/568728
Implement blueprint: ubuntu-bionic
Change-Id: I4e3b0ca669cfbf6316bf591f2d8428fa1a0d6182
Ia2acb09e877a586243fc1acb49d8d140cf27d7b5 removed the
"--wsrep-new-cluster" option and left "--wsrep-on=OFF".
But it seems "--wsrep-on=OFF" won't work with mariadb 10.0. and In
Ubuntu, we are still using mariadb 10.0.
Try to add --wsrep-new-cluster back
Change-Id: I4d382d1b9c83def62dc85a773580c8c6b2500f68
RDO packaged mariadb recieves far more testing and qualification
than any other mariadb package on CentOS. Lets use that instead
of a hard-pinned old version of mariadb from mariadb.org.
This patch also upgrade MariaDB from 10.0 to 10.1.20 for
RHEL/CentOS/Oracle Linux.
Depends-On: I8374ac2219ad7880970cd789727d01af7cac1077
Depends-On: Ia2acb09e877a586243fc1acb49d8d140cf27d7b5
Co-Authored-By: Xinliang Liu <xinliang.liu@linaro.org>
Change-Id: I071362fc1b8d60199a77e2fe0475d2b4c3b5341b
Option "wsrep_on" behaves differently between 10.0 and 10.1 [1]:
"Before MariaDB 10.1, even though this variable is ON by default,
its value get automatically adjusted based on whether mandatory
configurations to turn on Galera replication have been specified.
Since MariaDB 10.1, it is set to OFF by default and must be turned
on to enable Galera replication."
So on MariaDB 10.0 although wsrep_on set to OFF, its value will
automatically adjusted to ON because wsrep mandatory configurations
are set. Whereas on MariaDB 10.1, it will failed at galera provider
'wsrep_ready' checking because wsrep_on is set to OFF.
We can set wsrep_on to ON to handle both 10.0 and 10.1.
But, the point is that the cluster bits, which are creating a cluster
and wsrep_ready checking, are no need at bootstrap stage.
At bootstrap stage it mainly do two things:
Creating required system DBs (by mysql_install_db)
and securing the mariadb (by bootstrap_db).
After the bootstrap stage, it will create a cluster when starting the
first node.
[1] https://mariadb.com/kb/en/library/galera-cluster-status-variables/#wsrep_ready
Partial-Bug: 1740060
Co-Authored-By: Steven Dake <stdake@cisco.com>
Change-Id: Ia2acb09e877a586243fc1acb49d8d140cf27d7b5
With latest master, mariadb is failing to bootstrap in kolla-kubernetes
cluster. The issue is with the name of mysql pid file. When it is not forced
in the command line, in kolla kubernetes it gets created in the form of
mariadb-init-element-xqg4b.pid where mariadb-init-element-xqg3b is pod name,
but bootstrap scripts explicitely checks for mysql.pid.
Change-Id: Ic0ee577d73d196ef40354efa51ca3ba60bf4e125
Closes-Bug: #1704671
set_configs.py has logic to handle chown of directories. Simplify
the codebase by removing these unnessary chowns. Further the chowns
cause some forms of NFS backed storage to not work properly.
Change-Id: I8df95d06b1010778deb3e2a3065aaab26ed2eb6a
Closes-Bug: #1693973
Ubuntu builds use external MariaDB repository on x86-64 and ppc64le. But
that repo does not provide packages for AArch64 so I use mariadb/mysql
packages from distribution instead.
Partially-Implements: blueprint multiarch-and-arm64-containers
Change-Id: Ia3c3d3858082613718626670f9ff11b69d913c40
centos based images have wrong label info,
these changes fix own image's name and build-date.
Change-Id: I1d13f8f386c8db12b5fbe5f8ecbbf9e3fbb4ba1c
Closes-Bug: #1680341
Use LABEL instruction instead of MAINTAINER (deprecated) instruc-
tion as suggested by Docker's official dockerfile guide.
docs.docker.com/engine/reference/builder/#maintainer-deprecated
Closes-Bug: #1683652
Change-Id: Ie87a1ddf31aefcd0b623fd2837d78de420e76898
Debian 'jessie' even with backports lacks some packages Kolla expects. Due
to this we (Linaro) move to Debian 'stretch' release with our OpenStack
work.
Stretch is now frozen so there should not be many changes.
Partially-Implements: blueprint multiarch-and-arm64-containers
Change-Id: Iab7232cbe6f8fc3e3de10c67f3d89f134b185f05
MariaDB-* packages are from external 'x86_64 only' repository. Other
architectures use CentOS packages.
Split needs to be done as 'mariadb-devel' is provided by three different
packages on x86-64.
Partially-Implements: blueprint multiarch-and-arm64-containers
Depends-On: I71ddb7ef57c64d2505cac96724b1ab6772a57d6a
Change-Id: I9f3394eb35723f2bbf3205c6af6101b5a9daed38
Debian support is not maintained in Kolla so it got a bit behind Ubuntu
one. This changeset enables Debian for all images. Jessie (even with
backports) may be too old for some images though.
Also unify distro check to ['debian', 'ubuntu'] to keep alphabetical order
like it is done for RPM distributions.
Partially-Implements: blueprint multiarch-and-arm64-containers
Change-Id: I056233fbfa277e0e2360c07c3f80d9558c554357
Some images have packages sorted alphabetically and some not.
Unify common style between all images.
Change-Id: I906ed89c10b12886665618752f525ba71d83d991
in extend_start.sh, if Mariadb galera is not ready at first time, it will sleep 60s and then exit 1.
we should check it`s status every time in the loop.
Change-Id: I89174969687eecbf467ca3cad9aad1bfbfb8c4fe
Closes-Bug: #1676316
include_header and include_footer parameter is already removed, remove
them in all Dockerfiles.
Add missing footer block.
Change-Id: I90da03eb9f95a3827361d5f5ede65fde7d6be2b3
This centralizes all user and group creation into a single source. This
will fix any current and furture uid/gid mismatches (such as with
nova-libvirt).
In the process, we also unify users between the distros in a standard
way. The users in the following containers change from thier defaults:
Ubuntu: _chrony user is now chrony
Ubuntu: memcache user is now memcached
All: qemu user is used for ownership and socket permissions
All uid and gid numbers are customizable via kolla-build.conf
Co-Authored-By: Kris Lindgren <klindgren@godaddy.com>
Change-Id: I120f26ab0683dc87d69727c3df8d4707e52a4543
Partially-Implements: blueprint static-uid-gid
There is no reason to have networking active while bootstrapping the
initial mysql folder structure. It is an additional attack vector that
serves no purpose.
Change-Id: Ia636ba9f563dbfb969dbfb613198c98748ee8a3c
Mariadb galera will be ready only when wsrep_ready to be ON, rather then
wsrep_cluster_status is equal to Primary.
Closes-Bug: #1639838
Change-Id: I9ef60a39a195057eeee0404a39b174cc8feed793
The change c7c87909 has introduced an unconditional check
for wsrep cluster during bootstrap process whether is ready or not
and if it is not, bootstrap fails.
In kubernetes environment wsrep driver is set to none, cluster will
NEVER reach ready state hence causing kubernetes mariadb bootstrap job
failure.
Closes-Bug: #1623662
Change-Id: I0e6fc098861b7eeab544229d0b04a78fa498ddb9
Change needed to add header blocks to all Dockerfiles, similar to the
base.
Use case is to easily run something before packages are installed, e.g.
to COPY a local rpm in that can be added to the package list.
Change-Id: I1bbfdf0b762da0a392aa8bf47781315b45377bee
Closes-Bug: 1618969
Wait until the mariadb cluster is fully functional
- check existence of mariadb.pid is insufficient because the cluster
may not be fully prepared
Change-Id: I611b38f7dbc8032c42aee2a040fb1210b4bee7eb
closes-bug: #1614363
This patchset contains customization of Dockerfile of the MariaDB
container.
Change-Id: Id234f549376ec68c7f6120d058692aa64dc97de0
Partially-implements: blueprint third-party-plugin-support
Kolla-kubernetes requires there be an alternative way to bootstrap
the container. Kolla-kubernetes uses the Kubernetes API endpoint to
read information about the cluster so that the bootstrap process runs
serially.
Partially-implements: blueprint kolla-kubernetes-extended-dockerfiles
Change-Id: I6185f2522b1151c934871a1ad2387f0001c0a320
Co-authored-by: Ken Wronkiewicz <wh-openstack@wirewd.com>
The lightsout recover patch broke multinode mysql. Also the lightsout
recovery didnt probably pass the --wsrep-new-cluster flag. This
updates the mariadb bootstrap to work with multinode again.
Closes-Bug: #1559480
Related-Id: I903c3bcd069af39814bcabcef37684b1f043391f
Change-Id: I1ec91a8b2144930ea8f04cc1c201b53712352e4e
This playbook only matters for multinode since AIO can recover from
power outage without additional configuration.
DocImpact
Implements: blueprint mariadb-lights-out
Change-Id: I903c3bcd069af39814bcabcef37684b1f043391f
Mark Goddard
abraden
Michal Nasiadka
Christian Berendt
Marcin Juszkiewicz
Doug Szumski
Rabi Mishra
Alex Schultz
Eduardo Gonzalez
Damien Ciabrini
Jeffrey Zhang
Martin Mágr
Steven Dake
Xinliang Liu
Kevin Zhao
Serguei Bezverkhi
Jawon Choo
Chen
huweihua
Sam Yaple
Paul Bourke
Hui Kang
Shaun Smekel
Jeffrey Zhang
Ryan Hallisey
Swapnil Kulkarni (coolsvap)