76 Commits

Author SHA1 Message Date
Martin Mágr
293a1d9f8c Add socat to base image
We would need socat tool to perform healthchecks on the various containers.

Closes-Bug: #1749227
Change-Id: Ibda77a8d041d894f5efac6578e52b11651bca97f
2018-02-14 11:57:34 +01:00
Zuul
9d47c13a54 Merge "Consume RDO packaged mariadb version" 2018-01-23 06:55:01 +00:00
Jeffrey Zhang
502196d8fa Add --wsrep-new-cluster option back in the bootstrap_db function
Ia2acb09e877a586243fc1acb49d8d140cf27d7b5 removed the
"--wsrep-new-cluster" option and left "--wsrep-on=OFF".
But it seems "--wsrep-on=OFF" won't work with mariadb 10.0. and In
Ubuntu, we are still using mariadb 10.0.

Try to add --wsrep-new-cluster back

Change-Id: I4d382d1b9c83def62dc85a773580c8c6b2500f68
2018-01-22 17:36:22 +08:00
Steven Dake
a729d132d6 Consume RDO packaged mariadb version
RDO packaged mariadb recieves far more testing and qualification
than any other mariadb package on CentOS.  Lets use that instead
of a hard-pinned old version of mariadb from mariadb.org.

This patch also upgrade MariaDB from 10.0 to 10.1.20 for
RHEL/CentOS/Oracle Linux.

Depends-On: I8374ac2219ad7880970cd789727d01af7cac1077
Depends-On: Ia2acb09e877a586243fc1acb49d8d140cf27d7b5
Co-Authored-By: Xinliang Liu <xinliang.liu@linaro.org>
Change-Id: I071362fc1b8d60199a77e2fe0475d2b4c3b5341b
2018-01-21 19:19:57 +01:00
Xinliang Liu
6ff3a55aed Fix MariaDB bootstrap for 10.1 version
Option "wsrep_on" behaves differently between 10.0 and 10.1 [1]:
"Before MariaDB 10.1, even though this variable is ON by default,
its value get automatically adjusted based on whether mandatory
configurations to turn on Galera replication have been specified.
Since MariaDB 10.1, it is set to OFF by default and must be turned
on to enable Galera replication."

So on MariaDB 10.0 although wsrep_on set to OFF, its value will
automatically adjusted to ON because wsrep mandatory configurations
are set. Whereas on MariaDB 10.1, it will failed at galera provider
'wsrep_ready' checking because wsrep_on is set to OFF.

We can set wsrep_on to ON to handle both 10.0 and 10.1.

But, the point is that the cluster bits, which are creating a cluster
and wsrep_ready checking, are no need at bootstrap stage.

At bootstrap stage it mainly do two things:
Creating required system DBs (by mysql_install_db)
and securing the mariadb (by bootstrap_db).

After the bootstrap stage, it will create a cluster when starting the
first node.

[1] https://mariadb.com/kb/en/library/galera-cluster-status-variables/#wsrep_ready

Partial-Bug: 1740060
Co-Authored-By: Steven Dake <stdake@cisco.com>
Change-Id: Ia2acb09e877a586243fc1acb49d8d140cf27d7b5
2018-01-04 09:31:22 +08:00
Kevin Zhao
7192508e36 Creates directory /var/run/mysqld on aarch64 when building MariaDB docker image
Closes-bug: #1733262

Change-Id: I2d0a3422ba6a077a309c1fe967a2a62ef5ce6a36
Signed-off-by: Kevin Zhao <kevin.zhao@arm.com>
2017-12-05 15:33:03 +08:00
Serguei Bezverkhi
5e07cc1ab4 Fixing mariadb bootstrap failure in kolla-kubernetes
With latest master, mariadb is failing to bootstrap in kolla-kubernetes
cluster. The issue is with the name of mysql pid file. When it is not forced
in the command line, in kolla kubernetes it gets created in the form of
mariadb-init-element-xqg4b.pid where mariadb-init-element-xqg3b is pod name,
but bootstrap scripts explicitely checks for mysql.pid.

Change-Id: Ic0ee577d73d196ef40354efa51ca3ba60bf4e125
Closes-Bug: #1704671
2017-07-16 15:15:47 -04:00
Steven Dake
4607ab5e53 Remove sudo operations that are no longer necessary
set_configs.py has logic to handle chown of directories.  Simplify
the codebase by removing these unnessary chowns.  Further the chowns
cause some forms of NFS backed storage to not work properly.

Change-Id: I8df95d06b1010778deb3e2a3065aaab26ed2eb6a
Closes-Bug: #1693973
2017-05-26 21:40:31 -07:00
Jenkins
cf6209f2df Merge "handle mariadb for aarch64/ubuntu/source" 2017-05-10 11:13:53 +00:00
Marcin Juszkiewicz
4ff615d6d4 handle mariadb for aarch64/ubuntu/source
Ubuntu builds use external MariaDB repository on x86-64 and ppc64le. But
that repo does not provide packages for AArch64 so I use mariadb/mysql
packages from distribution instead.

Partially-Implements: blueprint multiarch-and-arm64-containers

Change-Id: Ia3c3d3858082613718626670f9ff11b69d913c40
2017-05-04 18:51:27 +02:00
Jawon Choo
31259fa595 Override image's meta info.
centos based images have wrong label info,
these changes fix own image's name and build-date.

Change-Id: I1d13f8f386c8db12b5fbe5f8ecbbf9e3fbb4ba1c
Closes-Bug: #1680341
2017-05-03 11:08:17 +09:00
Jenkins
128393b4ac Merge "mariadb: handle lack of external repos on non x86" 2017-04-24 16:34:13 +00:00
Jenkins
b9e320a7cd Merge "Use LABEL instead of MAINTAINER (deprecated) in all Dockerfile.j2" 2017-04-20 12:32:55 +00:00
Chen
8c463a47a9 Use LABEL instead of MAINTAINER (deprecated) in all Dockerfile.j2
Use LABEL instruction instead of MAINTAINER (deprecated) instruc-
tion as suggested by Docker's official dockerfile guide.
docs.docker.com/engine/reference/builder/#maintainer-deprecated

Closes-Bug: #1683652

Change-Id: Ie87a1ddf31aefcd0b623fd2837d78de420e76898
2017-04-20 16:50:05 +09:00
Marcin Juszkiewicz
f98a8d6287 debian: move to stretch
Debian 'jessie' even with backports lacks some packages Kolla expects. Due
to this we (Linaro) move to Debian 'stretch' release with our OpenStack
work.

Stretch is now frozen so there should not be many changes.

Partially-Implements: blueprint multiarch-and-arm64-containers

Change-Id: Iab7232cbe6f8fc3e3de10c67f3d89f134b185f05
2017-04-13 20:37:00 +02:00
Jenkins
5d653fa9df Merge "check mariadb galera status in every loop." 2017-04-12 11:12:53 +00:00
Marcin Juszkiewicz
21b04376e3 mariadb: handle lack of external repos on non x86
MariaDB-* packages are from external 'x86_64 only' repository. Other
architectures use CentOS packages.

Split needs to be done as 'mariadb-devel' is provided by three different
packages on x86-64.

Partially-Implements: blueprint multiarch-and-arm64-containers

Depends-On: I71ddb7ef57c64d2505cac96724b1ab6772a57d6a

Change-Id: I9f3394eb35723f2bbf3205c6af6101b5a9daed38
2017-04-10 09:13:27 +02:00
Marcin Juszkiewicz
69fef5cd59 debian: enable all images enabled for Ubuntu
Debian support is not maintained in Kolla so it got a bit behind Ubuntu
one. This changeset enables Debian for all images. Jessie (even with
backports) may be too old for some images though.

Also unify distro check to ['debian', 'ubuntu'] to keep alphabetical order
like it is done for RPM distributions.

Partially-Implements: blueprint multiarch-and-arm64-containers

Change-Id: I056233fbfa277e0e2360c07c3f80d9558c554357
2017-04-04 22:48:18 +02:00
Eduardo Gonzalez
623e54da37 Alphabetize packages
Some images have packages sorted alphabetically and some not.
Unify common style between all images.

Change-Id: I906ed89c10b12886665618752f525ba71d83d991
2017-03-28 16:45:16 +01:00
huweihua
f1d143128e check mariadb galera status in every loop.
in extend_start.sh, if Mariadb galera is not ready at first time, it will sleep 60s and then exit 1.
we should check it`s status every time in the loop.

Change-Id: I89174969687eecbf467ca3cad9aad1bfbfb8c4fe
Closes-Bug: #1676316
2017-03-27 17:04:17 +08:00
Jeffrey Zhang
e5903d5fa9 Remove include_header and include_footer in all Dockerfiles
include_header and include_footer parameter is already removed, remove
them in all Dockerfiles.
Add missing footer block.

Change-Id: I90da03eb9f95a3827361d5f5ede65fde7d6be2b3
2017-02-05 10:44:48 +08:00
Sam Yaple
58eee09c15 use static uid/gid in images
This centralizes all user and group creation into a single source. This
will fix any current and furture uid/gid mismatches (such as with
nova-libvirt).

In the process, we also unify users between the distros in a standard
way. The users in the following containers change from thier defaults:

Ubuntu: _chrony user is now chrony
Ubuntu: memcache user is now memcached
All: qemu user is used for ownership and socket permissions

All uid and gid numbers are customizable via kolla-build.conf

Co-Authored-By: Kris Lindgren <klindgren@godaddy.com>
Change-Id: I120f26ab0683dc87d69727c3df8d4707e52a4543
Partially-Implements: blueprint static-uid-gid
2017-01-17 09:02:21 -03:00
Sam Yaple
ae8df40b03 Disable networking during bootstrapping
There is no reason to have networking active while bootstrapping the
initial mysql folder structure. It is an additional attack vector that
serves no purpose.

Change-Id: Ia636ba9f563dbfb969dbfb613198c98748ee8a3c
2016-12-31 21:17:16 +00:00
Jeffrey Zhang
4417ffbfab Wait the wsrep_ready to be ON in mariadb
Mariadb galera will be ready only when wsrep_ready to be ON, rather then
wsrep_cluster_status is equal to Primary.

Closes-Bug: #1639838
Change-Id: I9ef60a39a195057eeee0404a39b174cc8feed793
2016-11-07 15:30:35 +00:00
Christian Berendt
5cd30d4914 Remove Fedora support
Closes-bug: #1616387
Change-Id: Id97f88b9baa3d48d33ce120962450a374282d044
2016-11-03 10:50:22 +01:00
Serguei Bezverkhi
cafdbb35d9 Mariadb bootstrap - no need to check for cluster ready in Kube
The change c7c87909 has introduced an unconditional check
for wsrep cluster during bootstrap process whether is ready or not
and if it is not, bootstrap fails.
In kubernetes environment wsrep driver is set to none, cluster will
NEVER reach ready state hence causing kubernetes mariadb bootstrap job
failure.

Closes-Bug: #1623662

Change-Id: I0e6fc098861b7eeab544229d0b04a78fa498ddb9
2016-09-14 17:22:26 -04:00
Paul Bourke
b41247c656 Add header blocks to all Dockerfiles
Change needed to add header blocks to all Dockerfiles, similar to the
base.

Use case is to easily run something before packages are installed, e.g.
to COPY a local rpm in that can be added to the package list.

Change-Id: I1bbfdf0b762da0a392aa8bf47781315b45377bee
Closes-Bug: 1618969
2016-09-13 16:53:31 +01:00
Hui Kang
c7c879095f Fix mariadb bootstrap error
Wait until the mariadb cluster is fully functional
- check existence of mariadb.pid is insufficient because the cluster
  may not be fully prepared

Change-Id: I611b38f7dbc8032c42aee2a040fb1210b4bee7eb
closes-bug: #1614363
2016-09-13 04:25:38 +00:00
Shaun Smekel
a05d4d71a0 Customizations for MariaDB
This patchset contains customization of Dockerfile of the MariaDB
container.

Change-Id: Id234f549376ec68c7f6120d058692aa64dc97de0
Partially-implements: blueprint third-party-plugin-support
2016-08-27 17:29:19 +10:00
Jeffrey Zhang
3f79e37d21 Fix the overwrite sudoers file issue
Change-Id: I4b109f7fdc3b8e49defed26979b04ca158842e98
Closes-Bug: #1598423
2016-07-05 05:49:13 +00:00
Ryan Hallisey
994d1e50a5 Add kolla-kubernetes bootstrap capability to mariadb
Kolla-kubernetes requires there be an alternative way to bootstrap
the container.  Kolla-kubernetes uses the Kubernetes API endpoint to
read information about the cluster so that the bootstrap process runs
serially.

Partially-implements: blueprint kolla-kubernetes-extended-dockerfiles
Change-Id: I6185f2522b1151c934871a1ad2387f0001c0a320
Co-authored-by: Ken Wronkiewicz <wh-openstack@wirewd.com>
2016-06-05 11:28:24 -04:00
Swapnil Kulkarni (coolsvap)
435b21b90d Update ubuntu dockerfiles for formatting
Change-Id: If4be00b937e14ec93443dcb7249cf17099d57cbe
Closes-Bug: #1569417
2016-05-26 04:09:22 +00:00
Swapnil Kulkarni (coolsvap)
7294ed9d90 Update Mariadb dockerfile for formatting
Change-Id: I6f5b28dc6520e3d31f413b0da708d9b14c9e9671
Partial-Bug:#1569417
2016-04-12 21:45:20 +05:30
SamYaple
afce10dea4 Fix mysql bootstrap
The lightsout recover patch broke multinode mysql. Also the lightsout
recovery didnt probably pass the --wsrep-new-cluster flag. This
updates the mariadb bootstrap to work with multinode again.

Closes-Bug: #1559480
Related-Id: I903c3bcd069af39814bcabcef37684b1f043391f
Change-Id: I1ec91a8b2144930ea8f04cc1c201b53712352e4e
2016-03-21 12:23:00 +00:00
SamYaple
2aaaed770e MariaDB lights out recovery
This playbook only matters for multinode since AIO can recover from
power outage without additional configuration.

DocImpact
Implements: blueprint mariadb-lights-out
Change-Id: I903c3bcd069af39814bcabcef37684b1f043391f
2016-03-16 15:53:44 +00:00
Paul Bourke
33e997b79d Add missing 'yum clean all' to mariadb
Found another missing clean in mariadb, halves the size of the image.

TrivalFix

Change-Id: Idcbb925a6f232bd1172710456662a52784e025b6
2016-03-10 12:36:23 +00:00
Éric Lemoine
4445c81991 Make Heka collect MariaDB logs
Partially implements: blueprint heka
Change-Id: Ib5e740683cee296bcac69228f525594850d62a27
2016-02-19 21:49:21 +00:00
SamYaple
7e2ce01431 Cleanup mariadb and make compatiable with mesos
There is no reason to have a hostname-unique pidfile in the container
as we currently have. This posed problems with kolla-mesos reusing
the same script. Since there is no reason for this pidfile to be
configurable in path _at_ _all_, we hardcode the path.

Additionally, we adjust the file perm change to only update the perms
on the folder if it is not already properly set.

This also incorperates a kolla-ansible file in the bootstrap process
which follows our other container techniques of using the idempotent
creation of a volume in the bootstrap process (see nova)

TrivialFix
Related-Bug: #1538136
Change-Id: I2380529fc7146a9603145cdc31e649cb8841f7dd
2016-01-28 20:26:41 +00:00
Sidharth Surana
37d44444d7 Make galeradb bootstraping robust
Currently, there are arbitrary wait for mariadb service startup.
However, this leads to nondeterministic results in the current
workflow. This patch tries to make the workflow more deterministic.

Change-Id: I3c6245cce93c7ff0d3d57cb2ae065a1ed1487769
Closes-Bug: #1491782
2016-01-04 09:10:04 -08:00
Swapnil Kulkarni (coolsvap)
59a1c8b9fb Remove hardcoded MAINTAINER in Dockerfiles
Added new option in kolla-build.conf

Change-Id: I45fe51966bcb59ea19d112281ba3d5a1ba091a56
Closes-Bug:#1514304
2015-11-23 11:03:47 +00:00
Sam Yaple
ea9d5cd067 Move USER operation after footer
The USER operation affects all docker commands after it. This causes a
problem with our {{ include_footer }} implementation since commands in
that footer may require elevated permissions to perform.

In the current implementation I can no longer remove my proxy settings
once the USER has been changed.

Change-Id: I9b2bab5a15f595f6d52a46c64ddf59ba5608b938
Partially-Implements: blueprint drop-root
2015-11-12 14:34:09 +00:00
Steven Dake
4c9e15b94e Drop root privileges for mariadb
Drop root privileges for mariadb.  This isn't perfect.  If somemone
breaks out of the container and can run sudo within the contianer,
it would be possible to replace the root credentials of the database.

Any container that uses sudo suffers from some extra attack vector
related to the sudo command.  That said, the sudo commands are
locked down to minimize harm.

Change-Id: I4b3573725d940bb8aa90d43a6235d8cf7d30fc64
Partially-Implements: blueprint drop-root
2015-11-12 03:12:40 -05:00
Steven Dake
09e9b1be33 Move the mariadb expect code to a script
Atleast in a script, sudo can be made to only allow the script to
run from the mysql process in the future, versus all the proceesses
being able to be executed as root presently.

Change-Id: I030b57086e37e4dc8f668f98c04335d94ab9d2b0
Partially-Implements: blueprint drop-root
2015-11-11 18:42:07 -05:00
Sam Yaple
cb4e875ae1 Common start.sh
The majority of the start.sh code is identical. This removes that
duplicate code while still maintaining the ability to call code in a
specific container.

The start.sh is moved into /usr/local/bin/kolla_start in the container

The extend_start.sh script is called by the kolla_start script at the
location /usr/local/bin/kolla_extend_start . It always exists because
we create a noop kolla_extend_start in the base directory. We override
it with extend_start.sh in a specific image should we need to.

Of note, the neutron-agents container is exempt from this new
structure due to it being a fat container.

Additionally, we fix the inconsistent permissions throughout. 644 for
repo files and the scripts are set to 755 via a Docker RUN command to
ensure someones local perm change won't break upstream containers.

Change-Id: I7da8d19965463ad30ee522a71183e3f092e0d6ad
Closes-Bug: #1501295
2015-10-06 03:30:26 +00:00
Jenkins
6a622ec82c Merge "Implement a install_type and install_metatype" 2015-09-28 10:49:53 +00:00
Steven Dake
0e99b69de4 Implement a install_type and install_metatype
This prepares for the RHEL OSP implementation by making the build
tool convert all binary-* into an install_type of binary and * into
an install_metatype variable substitution inside the Dockerfiles.
Further binary-* is substituted as install_name to enable proper
building only.

Change-Id: Ib681b29176eb79a3cab12ec824313fdecb6e7a5f
Partially-Implements: blueprint rhel-based-image-support
2015-09-28 03:16:48 -07:00
Sam Yaple
160f1cc011 remove set_configs
Change-Id: I9bb0a1fac63cc326234b0f06b1e56b43e0753279
Partially-Implements: blueprint replace-config-external
2015-09-27 13:44:50 +00:00
Sam Yaple
a1b0518b9a Fix removal of config-external
I removed the files but not the COPY commands thus breaking all of
Kolla

Change-Id: I37d3e0cb94a1ecc12971f485f953310ba8fee53c
Partially-Implements: blueprint replace-config-external
2015-09-25 20:08:40 +00:00
Sam Yaple
e2e0fd288f Remove config-external.sh
Removes config-external for all services that have been replaced in
Ansible

Change-Id: I839a14418638b977fbc1d02ba6839811b0f909ea
Partially-Implements: blueprint replace-config-external
2015-09-25 13:00:55 +00:00
Sam Yaple
77800984bc Refactor set_configs.py
This refactor brings the logging in line with the rest of Kolla.
The fucntion names were updated to reflect thier new role.

Additionally, it fixes several issues with the permissions which
currently break all containers that use set_configs.py

It will also work with source being a directory or a file now.

Change-Id: I4a197a343e3baf3bd31532debdff5972adb8aefa
Partially-Implements: blueprint replace-config-external
2015-09-17 20:10:58 +00:00