Commit Graph

101 Commits

Author SHA1 Message Date
Michael Johnson
a720f76c84 Remove Babel requirement
Babel is no longer a requirement for OpenStack projects.
Let's remove it and save some disk space.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014227.html

Change-Id: Ib4944fb44fea53c8dd1908b7f264d45a84884639
2020-04-17 09:46:14 -07:00
Michael Johnson
c8b044af4a Stop "dnf makecache" in the amphora images
On base operating systems that use DNF there can be a systemd timer
that attempts to call out and update the dnf cache every hour.
This patch adds an diskimage-builder element that disables this
timer as most amphora will not be successful in updating this cache
because they do not have network access or DNS services avaialable.

Change-Id: I5dec631d3e66e1dfb6b8741caea89938ee18a7d8
Story: 2007408
Task: 39019
2020-03-16 16:53:27 +00:00
Adam Harwell
d973322529 Add diskimage-create arg for Octavia Git branch
Use `-g` to select the current Git branch to use for the amphora
build. This has historically been confusing for people, and this should
make it a little easier.

Change-Id: Iab01faaa17507c7bea399db7ee8f27f88d8de183
2020-02-05 13:14:48 -08:00
Carlos Goncalves
dc2dd711e6 Stop supporting CentOS 7 controllers and images
The diskimage-create.sh tool will now default to CentOS 8 when building
CentOS-based amphora images.

This patch also removes leftover references to support for Ubuntu Trusty
and Xenial.

Change-Id: I3aba59c8dd86aeeee28cc6a67af93697912fb55b
2020-01-08 17:59:09 +01:00
Michal Arbet
7f929bab96 Fix diskimage-create.sh for Debian
As Ubuntu is debian-like platform, there is no reason
to not support building image on Debian platform. Currently diskimage-create.sh is
using head -n 1 to read /etc/os-release to fill platform bash variable.
Bash platform variable is filled with NAME="Ubuntu" string as it is first
line of Ubuntu's /etc/os-release. In debian the first line is for example
PRETTY_NAME="Debian GNU/Linux 10 (buster)".

This commit is fixing this behavior and fill platform bash variable
to exact NAME variable from /etc/os-release.

Change-Id: If7b507c30720516dc8dfd181170476f9ef003748
2019-12-25 11:28:43 +01:00
Michael Johnson
7471129c27 Stop supporting xenial amphora images
OpenStack is dropping support for xenial[1].

This patch removes references to support for xenial amphora images
and stops the nightly build of xenial amphora test images.

[1] https://review.opendev.org/#/c/695114/

Change-Id: I1bc0a37f35839a5f9a95650af7c82239f6544c50
2019-12-06 08:43:39 -08:00
Michael Johnson
7ad61bb451 Remove unnecessary interface configurations
The image building process adds some unnecessary network interface
configuration files to /etc/network/interfaces.d. This element cleans those
out as they are not needed because cloud-init will create the required
configuration file.

Change-Id: If5fbfc34ff0e91608f402811aa2b78c9cbcb00ec
2019-11-20 16:02:42 -08:00
Zuul
6b58e66868 Merge "Add Gentoo platform detection in diskimage-create/diskimage-create.sh" 2019-10-22 03:28:58 +00:00
Michael Johnson
9be2e4d4d2 Add the Amphora image building guide to the docs
This patch adds the Amphora image building guide from the
diskimage-create README.rst to the Administration documentation.
It also re-organizes the Adminstration guides to be broken down
by category as the old page was becoming a long list of guides.
(I like that kind of problem)
The diskimage-create README has a few formating corrections to make
it render better for the documentation.

Change-Id: Ice4071e1f872c8c0d0595427cff6f02ffbcf7968
2019-10-09 15:33:41 -07:00
Zuul
ca80bc9e03 Merge "Bump diskimage-builder minimum to 2.24.0" 2019-10-09 01:32:49 +00:00
Daniel 'f0o' Preussker
d429af5d53 Add Gentoo platform detection in diskimage-create/diskimage-create.sh
Change-Id: Iac685b424d6ab03a23c35c9eef8356f50a2fe952
Task: 36882
Story: 2006646
2019-10-02 10:56:41 +02:00
Michael Johnson
fbe328397b Fix the diskimage-create tox "build" environment
The recently added tox "build" environment had a few problems:
1. It was not honoring the DIB_* environment variables which meant
in always built a master branch image.
2. It also failed to run repeatedly due to a cache directory path issue.
3. The built images were stored in a hidden folder ".amp_tox_test".

This patch fixes those issues, resolves a confusing
"dpkg Broken pipe" message, and adds a "Successfully built" message
at the end of the built that highlights which branch the image was
built against (master, stable/stein, etc.).

Change-Id: I826c5f753f159b2d5dee97d4e2922826444ea6da
2019-09-19 04:12:37 +00:00
Michael Johnson
1aeb416a7e Bump diskimage-builder minimum to 2.24.0
There was a bug[1] in diskimage-builder when using pypi mirrors
if the host doesn't have "python" available DIB will fail.
So, I am going to bump the requirements minimum in Octavia for
diskimage-builder to be the minimum version with this fix.

[1] https://bugs.launchpad.net/diskimage-builder/+bug/1577105

Change-Id: If0ff2a855ad5b9d9ef3742ad0596c97a6dbf81ed
2019-09-10 22:20:42 +00:00
Zuul
60bd083ce6 Merge "Force DIB Python version for py2 in diskimage-create" 2019-08-27 23:49:22 +00:00
Michael Johnson
f8d4f8aee8 Add the DIB_REPO* variables to the README.rst
This patch adds the DIB_REPOLOCATION_amphora_agent and
DIB_REPOREF_amphora_agent optional environment variables to the
Octavia diskimage-create README.rst to clarify how to override the
amphora agent version in image builds.

Change-Id: Ic9beec74f84176d27fbefcc2fba281444ab02fa2
2019-08-05 14:30:26 -07:00
Gregory Thiemonge
2504c10916 Force DIB Python version for py2 in diskimage-create
DIB automatically picks a Python version based on the distro and
version. For Ubuntu Bionic, that default is Python 3 which means the
amphora-agent does not run on Python 2 when it was expected to on -py2
jobs [1].

[1] http://logs.openstack.org/10/661310/8/check/ \
    octavia-v2-dsvm-py2-scenario/664a91b/controller/logs/dib-build/ \
    amphora-x64-haproxy.qcow2_log.txt.gz#_2019-07-27_15_51_58_393

Change-Id: I6fd12be06bc848b3e29828c091f61229703e3a57
2019-07-31 14:39:10 +02:00
Carlos Goncalves
262ffaa9d7 Build amphora image using centos-minimal element
This patch changes CentOS amphora images to be built using the
centos-minimal DIB element. The generated image size is reduce from
approximately 702 MB to 520 MB. Compatibility with 'centos' AMP_BASEOS
is kept.

Change-Id: Iffc4a9725bff41618edbf246dd5a167b4e0ef6d3
2019-07-27 11:26:52 +02:00
Zuul
f4094381fd Merge "Make Ubuntu bionic the default for amphora images" 2019-07-27 03:46:17 +00:00
Michael Johnson
24fc7d615e Make Ubuntu bionic the default for amphora images
This patch updates the diskimage-create script to select Ubuntu
bionic as the default version of Ubuntu for the amphora image.

Change-Id: Ie8bbcbe073644137b8eb04c42d9457f507513007
2019-07-23 14:52:50 -07:00
Carlos Goncalves
b12e5bb25e Add cloud-init-datasources DIB element
The cloud-init-datasources DIB element configures cloud-init to only use
an explicit list of data sources. Until the switch from ubuntu element
to ubuntu-minimal, cloud-init was being configured to only use
ConfigDrive datasource because the ubuntu element had
cloud-init-datasources as element dependency. The ubuntu-minimal and
other OS elements (e.g. RHEL-based) do not have it, so the export
DIB_CLOUD_INIT_DATASOURCES was not being used.

Change-Id: I74ecc826596c5f739a49defe6588bcc741b03594
2019-07-16 14:34:26 +02:00
Michael Johnson
e3aacb67ad Switch the default Ubuntu kernel to -virtual
A previous patch changed the default Ubuntu kernel for the amphora
images to the -kvm kernel[1]. This kernel is missing a module required
for cryptsetup.
This patch switches the default to the -virtual kernel, which is
slightly larger than the -kvm kernel, but includes the required
kernel module for cryptsetup.

[1] https://review.opendev.org/665861

Change-Id: Ia05d7e9f8ac2692e5769648255ef0e4e10a18c03
2019-07-08 10:30:43 -07:00
Michael Johnson
cd50ae5c0c Specify the linux-image-kvm kernel for ubuntu
This patch switches to using the linux-image-kvm kernel by default.
It produces images that are about 200MB smaller than ubuntu-minimal with
linux-image-generic by using the linux-image-kvm instead of the
linux-image-generic kernel.

Depends-On: https://review.opendev.org/666063
Change-Id: I4103bcbaab65ed5d7683d35aaf04955e59ed2471
2019-06-26 23:32:12 +00:00
Michael Johnson
f1542afba0 Add a note about nova hardware architectures
This patch adds a note to the diskimage-create README that
reminds users to set the hardware architecture property on the
amphora image if the cloud has multiple architectures available
in nova.
It also sets this property in our devstack plugin to provide an
example/reminder.

Change-Id: I15c2cabb1bced0a5646a3e1aed50bece78afcdce
2019-06-17 10:37:57 -07:00
Carlos Goncalves
8821737359 Add RHEL 8 amphora support
Network scripts are deprecated in RHEL 8. This patch makes sure the
package is installed and the service enabled. Sometime in the future
(Train release or newer), support for Network Manager will be added as
it became the default network configuration tool in RHEL/CentOS 8 and
Fedora 28+.

This patch also reflects a change to diskimage-builder that will now
have a version-less 'rhel' element as opposed to a separate 'rhel7'
and 'rhel8' elements.

Change-Id: Id11459ea70479aa0145059f88af847dddcd93553
2019-06-07 18:34:29 +02:00
Trevor Vardeman
4c9097c681 Adding changes to support ppc64le amphora builds with CentOS
Updated diskimage create script to include an argument for disabling the temporary filesystem.
Updated diskimage create to support ppc64le as an argument.
Updated backports to support a properly configured PaaS repository for CentOS on power.

Change-Id: I8897278b1ac8b76d564f45bd0c7cbc26b29a8e5d
2019-04-18 11:11:09 -05:00
Frode Nordahl
d1518c5c79
Fix diskimage-create tox, add `build and test` targets
Improve end user experience with building a Octavia Amphora
diskimage through tox/virtualenv.

Change-Id: I8672d039a5b7ded57323aeb99f484fe7f2838fef
Task: 29059
Story: 2004847
2019-03-07 08:26:55 +01:00
Zuul
560d5ed6ee Merge "Update requirements for ubuntu" 2019-02-21 03:04:50 +00:00
Arnaud Morin
ca632987ea Update requirements for ubuntu
My test to build octavia amphora on Ubuntu Xenial was missing kpartx
package.
So I propose to update README.rst to help users build the image
correctly.

Also refactor a little bit how it is presented.

Change-Id: I65cf2ce25e96b4f44e2ca76884ec79831eb1c7f4
Signed-off-by: Arnaud Morin <arnaud.morin@corp.ovh.com>
2019-02-20 22:32:00 +00:00
Zuul
d0664582ee Merge "Workaround for a dnf bug" 2019-01-24 09:37:26 +00:00
Zuul
6786235f02 Merge "Exit with error if DIB output dir doesn't exist" 2019-01-18 20:15:16 +00:00
Carlos Goncalves
0d5eb67eee Workaround for a dnf bug
Work-around RHBZ #1636909 by using rpm instead of yum or dnf. The
alternative would be running with privileges, which would start
requiring sudo permissions.

Change-Id: Ie20a4865a464c7840644fd790ccac0d201b91187
2018-12-29 10:54:33 +01:00
Sven Wegener
4669ceeba7 diskimage-create: Support testing other image types
When building image types other then qcow2 there's no way to test them
currently. When specifying an existing file, just use that for testing.

Change-Id: I99f1a9e10642c4611c9b8985a6945bd82f1cce50
Signed-off-by: Sven Wegener <sven.wegener@inovex.de>
2018-12-27 02:49:25 +00:00
Carlos Goncalves
1ce8b2bd5d Exit with error if DIB output dir doesn't exist
Change-Id: Ic8ea57cdc753dc9e0a242924bbbf1ad3747a7493
2018-12-24 02:48:51 +01:00
Jacky Hu
f9fd702b20 Allow release id to be specified with fedora
This resolves the fedora image build failure below:
http://logs.openstack.org/periodic/git.openstack.org/openstack/octavia/ \
master/publish-openstack-octavia-amphora-image-fedora/6be692a/job-output.txt.gz

Change-Id: Ide8ebbc6264afb2f33cf2d6803aaf21631449424
2018-12-10 16:41:32 +08:00
Zuul
75c1d7c1ab Merge "Make disk image buildable for fedora" 2018-12-08 14:52:21 +00:00
Zuul
22c5028533 Merge "Redirect disk-image-builder logs, make verbose" 2018-10-31 14:28:39 +00:00
Ian Wienand
5bec6f70c2 Redirect disk-image-builder logs, make verbose
Update the diskimage-create.sh to have a "-l" option to pass into
dib's "--logfile" option.  In the devstack job, use this to redirect
the dib output to a separate logfile.  Turn on verbose logging by
default for this log.

Copy this in the v1 jobs.  v2 jobs are covered by a similar change in
octavia-tempest-plugin: I5bc0d9a3b071733cbe31d618f7236a3c2285b3e5

Depends-On: https://review.openstack.org/612865
Change-Id: Ie639af4266152d4626f8312b1849deeba5048348
2018-10-30 11:39:12 +01:00
Michael Johnson
4b1c374bdc Fix diskimage-create README.rst
We missed updating the diskimage-create README.rst when we added
some new command line options. This patch corrects those omisions.

Change-Id: I28dbc5b83321a69e060ea7fef5a1889ec623a3be
2018-10-29 11:53:23 -07:00
Jacky Hu
a587614756 Make disk image buildable for fedora
1. Enforce amphora image size >= 3G
2. Exclude debs required by ubuntu minimal for redhat

Change-Id: I343eaf19d4bd595d5b10dc330e53d41ceccc560a
2018-10-01 12:24:35 +08:00
Carlos Goncalves
90ec2ae622 Support DNF package manager in diskimage-create
DNF is the next upcoming major version of Yum. It has been the default
package manager since Fedora 22. This patch tries to use DNF if
available, and falls back to Yum.

Change-Id: Ic011ef8281eb99ae010c61f47a8423bc186c25b7
2018-09-08 02:52:27 +02:00
German Eichberger
464d8f9371 Introduce ipvsadm to the amphora image
This patch adds the "ipvsadm" tool to the amphora image.

Change-Id: I7ccf823cae3dbd57dc3f40aec66629e27e5db98b
2018-07-06 00:02:54 +00:00
Vadim Ponomarev
7a702fc638 Add raw format support to image creator script.
This patch adds new type 'raw' to the '-t' option of the
diskimage-create script for creating raw Octavia Amphora images.

Change-Id: Ib369009de5b42bd9f65bf43f9a0f23189a89de41
2018-05-14 17:48:37 +03:00
Carlos Goncalves
c5c7770e30 Add sos element to Red Hat family images
sosreport is a tool that collects information about a system.

The sos plugin for Octavia can gather information of installed packages,
log and configuration files for Octavia controller components and
amphora agent. The result is a generated report that can be used for
troubleshooting. The plugin redacts confidential data such as passwords,
certificates and secrets.

At present sos only installs in Red Hat family images as the plugin does
not support other distributions.

Change-Id: I5131a4cfdedd8b78fb673b4264ef1d7a1d613972
2018-04-19 15:51:50 +02:00
Michael Johnson
f935a51fe4 Switch to ubuntu-minimal for default amphora image
Previously we were using the "ubuntu" diskimage-builder base element as the
default base OS to build the amphora image.
The "ubuntu" element is based on the ubuntu cloud image. This image includes
packages we do not need for the amphora image. At this point it's not clear
that Ubuntu will ship an 18.04 LTS cloud image in the format the "ubuntu"
element requires.
This patch switches the default Ubuntu amphora image to build with the
"ubuntu-minimal" diskimage-builder element.

This patch also moves the amphora agent into a virtual environment inside
the amphora.

It also sets up support for Ubuntu 18.04 (bionic beaver) and HAProxy 1.8.

Change-Id: I84a85ca1363bce2e0f13da64540ec7ba3575e818
2018-04-16 18:07:31 -07:00
Adam Harwell
1020a3bceb Fix kvm-centos.7 gate
Somehow it was pulling in barbican-tempest-plugin incorrectly, and
didn't have image size set right for centos.

Also fix the devstack plugin to size the flavor correctly based on the
image size variable.

Change-Id: I724f5064309d07fe05f86fcf2c7a488d9319e54c
2018-03-08 12:54:37 -08:00
Bernard Cafarelli
579eaafe7f
Do not install pip in amphora when using distribution packages
pip-and-virtualenv element is not needed in that case, and can cause
image build failures in environments without direct Internet access

Change-Id: I37616d76dd78ffb1419a898509e9466e7c54f69f
2018-02-02 13:12:26 +01:00
Bernard Cafarelli
d7535e4558 Allow setting full mandatory access control in amphora
Currently this will set SELinux mode to Enforcing on RHEL variants

This allows to test all types of images, source/distribution with and
without security enforcing

Also sort some getopt options

Change-Id: If303181cf40247b050e60775254d958a055b3f0b
2018-01-23 11:07:31 -08:00
Bernard Cafarelli
d43d3fce86 Rework amphora agent installation element
Merge source and RHEL elements, allowing both source and package based
installations.

Allow amphora agent install from distribution packages (not limited to
RHEL)

Add a new option to diskimage-create.sh script to do so (default is kept
to source installation from Octavia git tree)

For now, amphorae built with distribution packages will have SELinux
(when available) running in permissive mode.

Made the rebind-sshd element generic to streamline the script
Use POSIX syntax for logrotate kill command

Change-Id: I391b2a95d54c7b9fd8f31d3e2c136ff9cc3451f1
2018-01-22 18:04:41 -08:00
Adam Harwell
8975d7acbd Merge keepalived elements
Change-Id: I2676b26ee59ad18583df937b258549bae241e100
2017-11-10 23:59:12 +00:00
Michael Johnson
e50ba48c1b Merge haproxy-octavia elements
Currently there are two haproxy-octavia elements, haproxy-octavia and
haproxy-octavia-ubuntu in the Octavia project.  The have minimal changes
between them, so this patch merges them into one element with a
backward compatible element left for haproxy-octavia-ubuntu.

Change-Id: I990802726d24e319988bfb614b3bf3fb560512b4
2017-11-09 09:09:01 -08:00