104 Commits

Author SHA1 Message Date
David Moreau Simard
e8ad7488f6 Make the yum repositories configurable in the base image
- Removed hardcoded yum repository configuration in favor of
  commands dynamically generated based on repo-url and repo-file
  arguments. We maintain a sane default set of repositories.
- Added generic rpm_setup_config parameter to add support for
  installing .rpm or .repo files before building containers.

Co-Authored-By: Ryan Hallisey <rhallise@redhat.com>
Implements: blueprint custom-repos
Change-Id: I1b3a7647a9e7239de3cd162cb6f464f05632bde1
2015-12-24 02:08:52 +00:00
Michal Jastrzebski
9cd603dfde Add kolla version to base container
We want to record kolla version of running containers to be able to
detect whether or not we need to perform certain downtime-causing
actions during upgrade.

Change-Id: Ie113029da98303e6809d56edbf6d8de37be128d7
Implements: blueprint record-version
2015-12-17 17:55:04 +00:00
Jeffrey Zhang
551c0fa2ce import the rpm gpg key
this will purge the package pgp key warning

Closes-Bug: #1525276
Change-Id: I956c8b0b6f1e1bfbceaa6abc06da16a980f023c1
2015-12-16 14:17:16 +08:00
Jeffrey Zhang
b35ba2b1b1 do not replace the systemd-container with systemd
Systemd-container no longer exist in centos latest docker image.

Change-Id: Icbfc7fce99885551f7f4caa707a600b20db849ae
Closes-Bug: #1526606
2015-12-16 12:41:59 +08:00
Jenkins
5d762a117a Merge "Do not purge yum-plugin-fastestmirror package" 2015-12-15 03:57:55 +00:00
Jeffrey Zhang
aabafd343b Do not purge yum-plugin-fastestmirror package
It will be helpful for speed up the package installation.

Closes-Bug: #1525514
Change-Id: Id40a0afa30dd0a142bee3b13f7622cb0d42d9298
2015-12-12 21:04:12 +08:00
Jeffrey Zhang
0932a0f7b2 add python-kazoo and python-six when using centos source install type
Closes-Bug: #1525055
Change-Id: Id800da1a0cce4db20c32e8daf09c5f0f2c9f8929
2015-12-11 13:24:35 +08:00
Michal Rostecki
dab8cae9b2 Add Python 3.x support in set_configs.py
Change-Id: I348c69e5411135da8434e9278a48a5777ff2a72d
Partially-Implements: blueprint python3
2015-12-07 09:41:31 +01:00
Jenkins
939b464d12 Merge "Add support for copying files from a "zk://" source" 2015-12-04 17:04:59 +00:00
Jeffrey Zhang
b3a695e934 Remove the meaningless comment line in base image Dockerfile
Use jinja2 comment syntax instead.

Closes-Bug: #1521932
Change-Id: Ida5a9204e7affe880ac6156d3fed3206b1eae7b9
2015-12-02 20:37:32 +08:00
Angus Salkeld
27c0ae0624 Add support for copying files from a "zk://" source
Co-Authored-By: Michal Rostecki <mrostecki@mirantis.com>
Implements: blueprint zookeeper

Change-Id: I176f063d3802716846b921e210c1569d28bd90d8
2015-12-02 10:25:43 +01:00
Jeffrey Zhang
ce05819d6c Fix the confused comment in the base image Dockerfile
Use jinja2 comment syntax instead.

Closes-Bug: #1521078
Change-Id: I261f7bdd9e1286b4ad71fc4529228449ffe21af2
2015-11-30 10:03:13 +00:00
David Moreau Simard
a9d570695a Install yum-plugin-priorities on RHEL-derivatives
RDO provides all of the OpenStack services, clients, libs and their
dependencies self-contained in it's repositories.
We have had users that were impacted by sudden updates from EPEL
when it was enabled because EPEL provided a more up-to-date version.
Packages may also be found in both the delorean and delorean-deps
repositories. yum-plugin-priorities will ensure the right package
candidates are chosen for installation.

Change-Id: I043ec1f60381dc7f5baab5f320ed5f1edde8ae82
Related-bug: https://bugzilla.redhat.com/show_bug.cgi?id=1284978
Closes-bug: #1520620
2015-11-27 15:21:58 +00:00
Michal Rostecki
7910044803 Pin specific version of Ceph
Change-Id: Ie358064b507729b9606683ce1a0e8feac7646030
Implements: blueprint pin-ceph
2015-11-24 19:15:47 +01:00
Swapnil Kulkarni (coolsvap)
59a1c8b9fb Remove hardcoded MAINTAINER in Dockerfiles
Added new option in kolla-build.conf

Change-Id: I45fe51966bcb59ea19d112281ba3d5a1ba091a56
Closes-Bug:#1514304
2015-11-23 11:03:47 +00:00
Steven Dake
6cf5928ff1 Base image changes for drop-root
The reason we are doing drop root is so that a network exposed
software component (i.e. glance) cannot be used to affect the
immutability of the container which it runs in.  I have tried
several different approaches and this is the only approach which
puts glance in PID=1 while ensuring no files may be written by
the glance process in the container image except for the log files.

Change-Id: Ifd3c8c361b78d0e4791dade3afa6435290407c41
Partially-Implements: blueprint drop-root
2015-11-09 11:00:26 -05:00
David Moreau Simard
4a1db7a08d Switch to RDO untested master repository
RDO does not yet provide a CI tested Mitaka repository.
As such, the current-passed-ci repository is the last tested
repository before the stable/liberty branch was cut.

To be able to test against the latest packages, we need to
use the untested repositories until the CI tested repository
is in place.

TrivialFix

Change-Id: I4a125eb3c84fa790746a9a8eca19e4fb2d9ecf38
2015-11-06 12:30:18 -05:00
Ryan Hallisey
0340bc4043 Fix RHEL builds
Register with RHEL on the host machine and use yum to setup
the repos in the container.

Change-Id: I38aaf43fffaf7a235e69b330d5d9f0f1be31fe83
Backport: Liberty
Closes-Bug: #1513088
2015-11-04 09:32:08 -05:00
Sam Yaple
2e07be310a Download, install, and enable rabbitmq_cluster
Change-Id: I84d7587fca64dc8a9f05e326a27511ded79e902d
Partially-Implements: blueprint rabbitmq-clusterer
2015-11-03 04:51:45 +00:00
Steven Dake
a40989c002 Use EL7 repostitory instead of EL6 for percona
This patch uses the EL7 binary bits for percona's software instead
of EL6.  To match binary ABIs, it is recommended to use the same
major version of EL for CentOS.

backport: liberty

Change-Id: I1d2b146a036806c7fd2baef97a6ed861a570d26e
Partial-Fix: #1509281
2015-10-23 02:55:44 -07:00
Steven Dake
c7a9835b68 Improve dry-ness of base dockerfile.j2
The delorean repositories no longer have a separate location for
openvswitch.  Now openvswitch is located in delorean-deps.repo
and the rest of delorean master is located in the delorean.repo
file.  These files can be installed for both RDO and SOURCE, but
not for RHOS.  This patch uses the install_metatype to make a
determination as to when to install these two repos.  In the
process, we can remove the source RPM installation.

Change-Id: Ieedddd9d7ee234b6acdb03f7043d57c18e024951
Closes-Bug: #1508326
2015-10-21 04:56:52 -04:00
Steven Dake
1353538644 Make Ceph work for RPM based distributions
Ceph packages need to be installed in nova, glance, and cinder.
Once that is done, Ceph works like a champ!

Change-Id: I296da1d04d0c1bcb729f22e65e432d53d561b49c
backport: liberty
Closes-Bug: #1505549
2015-10-13 21:48:17 -04:00
Jenkins
c4888d3310 Merge "Remove debug output in base image yum command" 2015-10-08 03:06:41 +00:00
Steven Dake
fbb1842fc8 Enable source for openstack-base for RPM distros
Make openstack-base optimized for from source builds for RPM based
distributions.

backport: liberty

Change-Id: I5f1056ebc09fd55cd5d46da7a09331e38940d888
Implements: blueprint openstack-common-container
2015-10-06 22:59:50 -07:00
Steven Dake
c5b6a436bd Remove debug output in base image yum command
The base image contains -d 10 in a yum command which prints unhelpful
debug output for one of the yum commands.  Remove the extranous debug
output.

backport: liberty

TrivialFix

This patch is a trivial fix and is a test to set boundaries on what
a trivial fix looks like that doesn't require a bug tracker entry.  If
reviewers feel a trivial fix isn't documented properly by this commit
message, please weigh in, as this is a slight suggested change to our
policy as first institued by the Magnum community and brought to my
attention by Jeff Peeler.  Hopefully this compromise won't be abused :)

Change-Id: I57f00978d0baaefd73592f4ad6a655e663d83478
2015-10-06 21:50:39 -07:00
Sam Yaple
94162bd08f Bring Kolla inline with FHS
This brings Kolla images inline with FHS and should make finding
locations of things more consistent and reliable with the linux world
at large.

Change-Id: Iece5b4da4bace0fb8b1f41a65ab2c852ec73e6f8
Closes-Bug: #1485742
2015-10-06 03:30:53 +00:00
Sam Yaple
cb4e875ae1 Common start.sh
The majority of the start.sh code is identical. This removes that
duplicate code while still maintaining the ability to call code in a
specific container.

The start.sh is moved into /usr/local/bin/kolla_start in the container

The extend_start.sh script is called by the kolla_start script at the
location /usr/local/bin/kolla_extend_start . It always exists because
we create a noop kolla_extend_start in the base directory. We override
it with extend_start.sh in a specific image should we need to.

Of note, the neutron-agents container is exempt from this new
structure due to it being a fat container.

Additionally, we fix the inconsistent permissions throughout. 644 for
repo files and the scripts are set to 755 via a Docker RUN command to
ensure someones local perm change won't break upstream containers.

Change-Id: I7da8d19965463ad30ee522a71183e3f092e0d6ad
Closes-Bug: #1501295
2015-10-06 03:30:26 +00:00
Steven Dake
571c5bc878 Make a openstack-common container for CentOS variants
Only implements CentOS (RPM family) binary in openstack-common.

Change-Id: I5249d0ffaf4c0c946ee2347d5519391ded160cea
Partially Implments: blueprint openstack-common-container
2015-09-29 03:45:38 -07:00
Steven Dake
f74cd0ecb8 Significant refactor of base image
This refactor organizes the code as follows:

If RPM family:
    execute YUM repo setup
    if binary
        install binary tools
    if source
        install source tools

IF DEB family:
   install source tools

I have tested building and everything seems fairly solid except
possilby RHOS.  The long term goal of this patch is to allow us to
add our own repos in the "execute YUM repo setup" section of the base
file based upon OpenStack Infrastructure YUM repos.

Partially-Implements: blueprint rhel-based-image-support
Change-Id: I759fb8fc050973460c9c69bb500de68e88a68b3b
2015-09-28 03:16:52 -07:00
Steven Dake
0e99b69de4 Implement a install_type and install_metatype
This prepares for the RHEL OSP implementation by making the build
tool convert all binary-* into an install_type of binary and * into
an install_metatype variable substitution inside the Dockerfiles.
Further binary-* is substituted as install_name to enable proper
building only.

Change-Id: Ib681b29176eb79a3cab12ec824313fdecb6e7a5f
Partially-Implements: blueprint rhel-based-image-support
2015-09-28 03:16:48 -07:00
Sam Yaple
39aba34b2a Remove all fail sections for Ubuntu
Ubuntu binary is not supported and may never be. Installing from
cloud-archive packaging is only for the current stable distros, Ubuntu
does not have a Delorean type repo. We place a fail message in the
base image to catch this and remove the messages throughout the
project.

An additional fail message is placed to catch all other things.

Change-Id: Id2953f503ebd42226f6a08e75979ae56511c40f7
Implements: blueprint install-from-ubuntu
2015-09-27 08:10:59 +00:00
Jenkins
ccffeb7768 Merge "Add missing dependency repository for Liberty for EL" 2015-09-21 14:20:48 +00:00
Jenkins
0045fd2c57 Merge "Remove ceph as external repo" 2015-09-20 10:42:07 +00:00
Jenkins
7a8f82aded Merge "Add sudo for RHEL builds" 2015-09-20 08:53:17 +00:00
Jenkins
b8831e4c17 Merge "Install which if its not installed in base" 2015-09-20 08:48:45 +00:00
Sam Yaple
18a32f5e0c Remove ceph as external repo
* The cloud-archive now provides the most recent version of Ceph
  (hammer) and should be preferred over adding a new repo.
* The ceph signing key changed.[1]

[1] https://securityblog.redhat.com/2015/09/17/important-security-notice-regarding-signing-key-and-distribution-of-red-hat-ceph-storage-on-ubuntu-and-centos/

Change-Id: I5992184f115106a9c853d205d7d79c35e1b0cf27
Closes-Bug: #1497507
2015-09-19 05:37:41 +00:00
David Moreau Simard
4b7163abfb Add missing dependency repository for Liberty for EL
There are three repositories for liberty in entreprise
linux:
- delorean
- delorean-liberty-testing
- delorean-common-testing

This commit adds the two missing ones so that delorean
can pull the proper dependencies. Otherwise, it might
have either broken dependencies or outdated ones from EPEL.

Change-Id: Ia815b38d3a83a89c582ea874e39c170d618b99e5
Closes-Bug: #1497436
2015-09-18 21:47:26 -04:00
Jenkins
b0cd947c7a Merge "Fix broken centos-source- build: add rdo repository" 2015-09-18 20:10:24 +00:00
Steven Dake
1d9659d2e0 Add sudo for RHEL builds
RHEL does not include sudo in the default container image, while other
RPM based distros do.  As a result, some services don't work.  Add
sudo to the base image.

Change-Id: I7014d8cbd90288456e5ab3badcd601ef9cdd0c87
Partially-Implements: blueprint rhel-based-image-support
2015-09-17 07:12:19 -07:00
Steven Dake
f39396cf7f Install which if its not installed in base
Some variants of RHEL containers are missing the 'which' command.
Percona's software used by mariadb depends on which, but doesn't have
a hard dependency requirement since it is usually installed in the
distribution.  To make mariadb work multinode on RHEL, install which
all the time for RPM distros.

Change-Id: I2c6aaa77a1fc8b7028b658a0dfd1935c1f1d738e
Partially-Implements: blueprint rhel-based-image-support
2015-09-17 07:10:46 -07:00
Vladislav Belogrudov
aec595f7f7 Fix broken centos-source- build: add rdo repository
openvswitch requires rdo repository to be installed

Change-Id: I173bb3d32bcef2b890ab482083b0ffb830c3b7fb
Closes-Bug: #1495470
2015-09-17 16:58:07 +03:00
Jenkins
1b33345fb8 Merge "Use Liberty cloudarchive repo for Ubuntu" 2015-09-17 11:33:08 +00:00
Sam Yaple
84ea8fbc9d Use Liberty cloudarchive repo for Ubuntu
Additionally remove rabbitmq since the correct version is now provided
in cloudarchive repo.

Change-Id: I215f03c4770f3049a9498c36c6ee9e819ce0acae
Closes-Bug: #1495836
2015-09-16 10:15:00 +00:00
rthallisey
3d7f9fd624 Replace config-external with a JSON file
Replace config-external with JSON file. The JSON file will
be placed in each of the services directories with expected
location and destination.

Set-configs.py will be responsible for interpreting the JSON file,
creating the necessary directories, moving config files, and
and providing a command line for start.sh to run the service
with the correct config files specified.

Partially-Implements: blueprint replace-config-external
Change-Id: I5e2e69dfe3ae7f938fcf51f1cd450aaa10e7f1e3
2015-09-15 10:25:45 -04:00
Steven Dake
4a79ce4041 Make RHEL build from source
This patch enables building from source in RHEL.  It works by making
the repositoroies specific to RHEL7 run in either from source or
from binary/rhos builds.

Change-Id: I795284cccc87c4a87d58c93ec73b3f9ea08878f2
Partially-Implements: blueprint rhel-based-image-support
2015-09-12 18:39:31 -07:00
Steven Dake
360999929d Add support to the base image for RHEL
Add support to the base image for running RHEL containers.  Document
how to register a system with RHN in image-buliding.rst.

Change-Id: Id69c114cf734a8242066b8d7d05ba19c07af036a
Partially-Implements: blueprint rhel-based-image-support
2015-09-04 07:06:56 -07:00
Sam Yaple
3487d8da14 Rename Galera to MariaDB
Change-Id: I3134ba72e2ce4e85a17facbc3b5af4c1c3438d33
Closes-Bug: #1491740
2015-09-03 21:00:23 +00:00
Sam Yaple
9d3f3f28e2 Create openstack-base container
This creates and moves the dependencies for Ubuntu into a common
openstack-base container. This commit shows dramatically smaller
sizes for all non-openstack containers. The Openstack container remain
the same size.

Change-Id: I2f46420d4b9edcfddda374caddcce906fc708f6c
Partially-Implements: blueprint openstack-common-container
2015-09-03 10:37:45 +00:00
Sam Yaple
1cbfbcc48d Reduce install size
This is a trade-off, it increases the base size some _but_ it
decreases the total size by a few hundred MB.

Partially-Implements: blueprint openstack-common-container

Change-Id: I1216deed122aa734e0c84325264a53711d94ea4c
2015-09-03 09:31:07 +00:00
Jenkins
be230d47eb Merge "Add Ansible support for Heat" 2015-08-31 11:12:24 +00:00