As a followup to Id99948aec64656a0532afc68e146f0610bff1378, adding auto
detection to haproxy_amphora.user_group
haproxy is capable[1] handling a list of configuration files.
This patch leverages that capability by simply providing haproxy with an
additional configuration file, which is baked in the amphora image via a
diskimage-builder element.
The above-mentioned element will specify the following values for user group:
Ubuntu: 'nogroup'
RHEL/CentOS/Fedora: 'haproxy'
The amphora-agent will parse and remove any user_group configuration provided
by Octavia controller worker.
This is in order to maintain amphora-agent backward compatibility to old
Octavia workers, who still provide user_group to the amphora-agent.
Octavia Workers that include this patch will no longer provide user_group
configuration to the amphora-agent.
[1] https://cbonte.github.io/haproxy-dconv/1.7/management.html#3
Related-Bug #1548070
Change-Id: Ia8fede9d7da4709a48661d1fc595a16d04fcbfa9
ecryptfs was dropped from RHEL/CentOS, use LUKS on a RAM-backed block
device (brd) instead.
Made the element name more generic
Added systemctl enable call in postinstall (for systemd init), so that
the service is correctly started and listed as wanted by amphora-agent
Change-Id: Id8c7ff93ae244ef14480e22c85dc79355a902105
Closes-Bug: #1642982
Closes-Bug: #1662952
This patch removes outdated kernel tuning parameters that were set
inside the amphora. With current kernel versions the performance
issues no longer out weigh the benefits.
Change-Id: I6435257ec1f0ee0cc8c38df0d1ff0247707174e4
Closes-Bug: #1661105
Not all Linux flavors accept the same type of configuration to manage
NICs. The amphora-agent must be able to distinguish between different
Linux flavors and choose the appropriate type of jinja2 NIC
configuration template for each one, respectively.
Up until now, The amphora-agent had no notion of the operating system
it is running on, therefore it used NIC configuration templates that
only match Debian based Linux flavors (mostly Ubuntu). Making it
unusable for flavors such as RHEL, Fedora and CentOS.
This fix enhances how the amphora-agent is handling NIC hot plugs.
It will use the appropriate jinja2 template by checking the Amphora
distribution name when needed.
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Closes-Bug #1548070
Change-Id: Id99948aec64656a0532afc68e146f0610bff1378
With the recent changes, gcc is not pulled in anymore on CentOS, and
compilation of the python modules fails in the amphora-agent element
To be on the safe side, this adds the build-essential dependency
to make sure these modules can be buillt
Change-Id: I842b07cbc3e48209fd500bff5cc798be655f0ae9
This patch sets up a seperate log file for the amphora-agent
and logrotate to manage this new log.
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Change-Id: Ia7b057642d7a567d685d989d1c689d5f3481e73e
The previous patch that switched the agent install to use pip install
did not include the -U switch or the upper constraints flag.
This patch adds those to the amphora-agent element.
Change-Id: If907909704fcff4c7be10690eb4f50d0ba54b1c1
There were only two elements we were using from this repo -
os-svc-install which we werent actually calling anywhere, we only
depended on it needelessly and sysctl. The sysctl element has been moved
in to dib as dib-sysctl so we can now stop depending on
tripleo-image-elements entirely.
Depends-On: If312d199388036d6f4103e94dca99249cb3bcbaf
Depends-On: Ia730850a48e2478fd5461710a9d2619408725cd8
Change-Id: Ie78c4f3ebe506214f0ce7c456fcbbee09d35ba2a
pip install is greatly preferred over python setup.py install, so lets
use that to install our amphora-agent.
Change-Id: I5e4d169a1e6eb0e175f51943c08b025b09ffdc05
We only need ecryptfs-utils, as package managers will take care of the
sub-dependencies. Tested on an Ubuntu amphora, installed packages list
was identical.
This also fixes image creation on Fedora, as ecryptfs-utils package name
is identical there
Change-Id: Idab8c66e6bca137e79bef050fbaecd2f6c4add7a
Closes-Bug: #1640832
This patch enables auto-detection of the init system used in the
amphora image and adds support for systemd amphora.
This patch allows Ubuntu xenial amphora images to work.
It also merges two functional test files into one file to reduce
code duplication.
This is a scenario gate fix.
Change-Id: I5fec1680bd47719ae9f2fcb6abaaba8a78e2ae8b
Closes-Bug: #1640866
This patch adds an element that causes the terminated HTTPS
certificates and keys to be stored in an encrypted ramfs path
so they are encrypted at rest.
Change-Id: Id0f80f311d37d5691087e855fb1291011451c851
Closes-Bug: #1627370
There are upstream methods for setting a mirror which are much more
flexible, lets just document and point users at them.
Change-Id: I442e4695d56086932403fb5893da89ead203c86d
We currently install this via its own element, but this could be made
more simple / obvious by using the python requirements tooling.
Change-Id: I2a2eed2b8ee481189c1694659ac8100062a25cf2
This reverts commit fdde4e69846bda02e49c636fed991ba86318183d.
This was reverted due to an issue with starting the service on boot. A
trick of using symlinks was being attempted in order to refer to files
in the base source directory from an element. This seems to not work, so
copying init scripts in to the elments themselves.
Change-Id: If6ae07fa4ab4f39e2a339e17fbcc163b863ccd3a
This reverts commit 687a972960915ef01008701dbbb35a7b52c728ae.
Something about this seems to have possibly broken amphora boots.
Testing reverting this patch to see if it fixes the scenario runs.
Change-Id: I824cb1c6fa38f2320f84456a551585114215f625
This adds a rhel type and gives some pointers on needed environment
variables for base image, registration, needed subscriptions.
no-resolvconf is also updated to work on rhel-type systems
Depends-On: Icb0e20b01479fea345e01309fc4bf3f7f639900c
Change-Id: Ic452feaf32fe7699ebd072f21dd630a2ac93bb5f
Creating separate elements per-distro is an anti-pattern for DIB. This
leads to a lot of code duplication across all the elements being used.
In DIB we try and create facilities to allow elements to easily perform
cross-distro tasks so this isn't necessary.
Change-Id: Ie3018eacfa0e2726449cfb580ded09d027bed349
There are some new facilities in DIB which simplify our element code -
declarative package installs, automated init script installation.
Change-Id: I8b91325999af5cfc9e455c4d6228453d8567c919
Our present amphora image create scripts set up the ssh daemon on the
amphora to bind to the wildcard interface (which is the default).
However, this causes problems for anyone who tries to set up a listener
on TCP port 22, since haproxy will not be able to bind to the same IP.
This patch introduces a dhclient post-bind script to the amphora image
to gracefully rebind the SSH-daemon to only the load balancer management
net IP when it comes up on the network, solving the above use case. This
patch has the secondary benefit of making the amphora's SSH daemon only
respond to requests on the management network, which incrementally
increases the security of the amphora.
Change-Id: Iab93cec1f4dc4a2e37ad3cb8a92c132383dcda6a
Closes-Bug: #1551505
A few elements were needed to define specific repositories.
amphora-agent: Updated and now buids with sysvinit instead
of upstart Octavia requries the use_upstart parameter to be set
to False to work with RHEL systems.
amphora-agent-ubuntu: original element that supported upstart
keepalived-octavia: Supports RHEL repositories
Change-Id: I961663d105ad09e7f05c77cb92efaa469f386fdc
Closes-Bug: #1531092
This patch does the following:
* Reduces interval between checks when waiting on
amphora status in a devstack environment. At the same time we
increase the number of retries so we are effectively waiting
the same total time before a timeout error occurs.
* Disables DNS resolution on the amphora image. Amphorae don't
need to do any DNS resolution anyway since any outbound
connections are done specifying IP addresses. This means
that the amphora never waits for DNS timeouts to occur
when booting and performing other tasks when operating in an
environment where DNS resolution doesn't work (ie. devstack)
In preliminary tests, the above optimizations seem to shave
about 50-55 seconds off per test. (This is mostly due to the
elimination of DNS timeouts.)
Change-Id: Icc7d8e0ac18a4a18ed9eb0950081ddd198cf4684
This patch disables the default haproxy process from starting in
the amphora image.
Closes-Bug: #1527691
Change-Id: Iccac03b982e47fc85f8e6beb0cc55a80b73ab465
This patch fixes two things:
1. The sysctl settings were failing because some conntrack modules
were not loaded anymore.
2. I fixed the sysctl-set-value scripts to be able to handle
multi-value settings, such as tcp_rmem, in tripleo-image-elements[1].
Here I have removed the workaround we had in Octavia.
[1] https://review.openstack.org/#/c/134616/
Change-Id: Ib7ab4f487c1b792b70a110098bf7a28cb565ee55
Closes-Bug: #1527392
In the dependent patch we pinned the amphora-agent to pull the patch set
version so that scenario tests will pass. This patch removes that pin.
Change-Id: I1421939423916fd9dd5dc6f754a87e316ec7c110
This patch implements the Active/Standby blueprint in
https://blueprints.launchpad.net/octavia/+spec/activepassiveamphora
The following points describe the main changes:
1. The patch introduces new flows and subflows to create M amphorae. The
controller worker parses the loadbalancer_topology configuration. If the
loadbalancer_topology value is ACTIVE_STANDBY, the controller invokes a new flow
independent from the SINGLE topology case, which is left untouched. The new
flow uses conditional taskflows to check for spare amphorae at runtime. This
removes the need for the exception workaround we earlier had. The controller
creates the amphorae in parallel using an unordered flow. A new database task
alter an amphora role as either MASTER or BACKUP and assigns a VRRP priority to
each amphora. After the amphorae are created, the controller invokes a separate
flow for post amphora configuration including plug_vip methods, vrrp
configuration upload, and keepalived service start.
2. The patch introduces new data models that include a new table for VRRP group
configuration per loadbalancer, and update the amphora, loadbalancer, and
listener tables to support the new active/standby capability. The VRRPGroup
table hides authentication data, and makes future extensions of VRRP
capabilities easy.
3. This patch updates the existing Haproxy configuration templates to include
peer synchronization. In case of ACTIVE_STANDBY configuration, the jinja
configuration renders the peer section in the Haproxy configuration and assigns
short names to the amphorae as listener peers. As listeners implies different
Haproxy process, each listener synchronizes on a different port evaluated as
BASE_PORT (1024) + NUMBER_OF_LISTENERS accounting for ports in use.
4. This patch introduces a new Jinja configuration templater and a REST driver
for Keepalived (developed as a Mixin). By default, Keepalived runs "all" check
scripts found in a predefined directory. The keepalived driver is a Mixin that
can be plugged in other services' drivers. It is the responsibility of these
services drivers to introduce their own check scripts. In this patch a
lightweight check script for Haproxy was introduced along with changes in the
amphora agent installation script.
5. The VRRP requires enabling protocol 112 for Master/Backup advertisements,
and enabling protocol 51 for authentication header. This patch enables these
protocols as needed in the loadbalancer security group.
Note: Updates to the failover flow to support active/standby will come in
a dependent patch.
Note: The amphora-agent is pinned to this patch in this patch set. This
is required so the scenario tests will pass. It will be removed in a
follow up patch.
Co-Authored-By: Sherif Abdelwahab <sherif.abdelwahab@hp.com>
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Implements: blueprint activepassiveamphora
Depends-On: Ifdf20378b26cdd13e0a3ff87cec8990fe89c0661
Change-Id: Ic4e04594e114ba682088d68d5f1af3f8f376db83
Used binary compressed encoding of json dumped object. To reduce
the size needed to send heart beats incase some stats objects
start getting sent later on. Also used sha256 instead of sha1
with hmac.
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Co-Authored-By: German Eichberger <german.eichbeger@hp.com>
Co-Authored-By: Carlos Garza <carlos.garza@rackspace.com>
Partially implements: health-manager
Change-Id: I932c693101b94c9132e1741291610508876eab43
1. Creates a new element for pyroute2
2. Adds this element to the amphora image
3. Updates the amphora REST interface to pass additional network information
4. Creates the policy based routes and rules on the amp during plug vip
5. Updates the REST API spec
Change-Id: Ibd622ec302cf78c12ae2bd5d76d012ab619939a6
Added support of keepalived in the amphora image as follows:
1. diskimage-create.sh: append the vrrp-octavia by default.
2. image-test.sh: checks if the image has keepalived installed.
3. 77-vrrp: checkout the right keepalived source code version, compile, and
install binaries.
4. svc-map: clone the keepalived rep.
5. fixed some typos in comments and readme
Change-Id: I483a66590fc343c07ca37bc11c2ad5482594d4cc
Adds rest driver methods
Adds rest driver tests
Add cert task for generating server certs
Modified compute task/flow
Fixed local certificate stuff
Refactored to use requests-mock inetad of responses
Added a "conditiobal flow" for REST
Cleaned up and changed the code to work with
https://review.openstack.org/#/c/160034/
Replaces:
https://review.openstack.org/#/c/144348/https://review.openstack.org/#/c/145637/14
Change-Id: Ibcbf0717b785aab4c604deef1061e8b2fa41006c
Co-Authored-By: Phillip Toohill <phillip.toohill@rackspace.com>
Co-Authored-By: German Eichberger <german.eichberger@hp.com>
Co-Authored-By: Stephen Balukoff <sbalukoff@bluebox.net>
Implements: bp/haproxy-amphora-driver
Updated files to use openstack instead of stackforge
Updated devstack samples to not point to old references
Change-Id: I501d76173475bb1619006819d971493854fdd15c
- Added configuration
- Added uploading of haproxy config
- Added start, stop, reload (async)
- Added get_details
- Fixed returns of API -- they are now all spec conform
- Added info, get haproxy file
- Added function to get listener status
- Added class to parse haproxy stats socket
- Added methods to handle certs
- Added client cert validation to the sever
- Added script to generate example certs
- Added init script for agent
- Added network and vip plugging
- Added devstack stuff
- Added diskimage scripts; upstart ini file
Change-Id: Ib1db8da9e019e68e9a0f4a16a622b8b1286afd3e
Implements: blueprint amphora-api
Adds haproxy 1.5.x from ubuntu trusty backports
Adds a git pull for the amphora agent code
Removes check for argparse (internal for python 2.7)
Adds sysctl net.ipv4.ip_nonlocal_bind=1
Change-Id: I7aecf727fb5d9be08982c5a32ae3c6e280ebda14