Commit Graph

61 Commits

Author SHA1 Message Date
Michael Johnson
cc97397d1c Followup patch for UDP support
1. Removes the misc_dynamic setting from the UDP-CONNECT health monitor
   as our script does not use it.
2. Adds a release note for the UDP features.
3. Updates the API reference for UDP support.
4. Adds a comment to the keepalived config with the LB ID.
5. Updates the status message type to be the correct UDP protocol.
6. Fix error during deleting a listener if there are multiple amphoraes.
7. Refactors systemd service script handling.

Story: 2003306
Task: 24258
Change-Id: I09240023d066ac5a71836d01045cda6ce5678712
2018-08-07 20:55:11 -07:00
Michael Johnson
294e0fc128 Fixes unlimited listener connection limit
When using the Octavia/amphora driver, unspecified or unlimited (-1)
settings would lead to a 2000 connection limit in HAproxy.
This patch updates that to be 1,000,000 connections.
1,000,000 was selected to amphora memory usage at a reasonable level.

Change-Id: Iddeb62412bb71b69cf1e9198be6131c59a3051b0
Story: 1635416
Task: 5159
2018-07-19 09:47:31 +02:00
German Eichberger
464d8f9371 Introduce ipvsadm to the amphora image
This patch adds the "ipvsadm" tool to the amphora image.

Change-Id: I7ccf823cae3dbd57dc3f40aec66629e27e5db98b
2018-07-06 00:02:54 +00:00
chengebj5238
52eaaa7583 Amend the spelling error of a word
Change-Id: I7a2da9597d601740921c8e6fc16f9c37b1a60eef
2018-06-06 18:03:02 +08:00
Adam Harwell
b0a845554b Update HAProxy version for Centos
ALso tweak the systemd service config for haproxy 1.8 since it no longer
ships with a systemd wrapper.

Change-Id: If4f230dcba8c360c919f6c2d93705bf67089b2cf
2018-05-07 18:57:34 +00:00
Adam Harwell
aa1b32df54 Fix keepalived vrrp check script to be in PATH
ACTIVE_STANDBY is currently broken because keepalived can't access the
main check script.

Change-Id: I66a5df575583b0545fb00bc1170ad8064d2963aa
2018-04-25 16:49:05 -07:00
Carlos Goncalves
c5c7770e30 Add sos element to Red Hat family images
sosreport is a tool that collects information about a system.

The sos plugin for Octavia can gather information of installed packages,
log and configuration files for Octavia controller components and
amphora agent. The result is a generated report that can be used for
troubleshooting. The plugin redacts confidential data such as passwords,
certificates and secrets.

At present sos only installs in Red Hat family images as the plugin does
not support other distributions.

Change-Id: I5131a4cfdedd8b78fb673b4264ef1d7a1d613972
2018-04-19 15:51:50 +02:00
Michael Johnson
f935a51fe4 Switch to ubuntu-minimal for default amphora image
Previously we were using the "ubuntu" diskimage-builder base element as the
default base OS to build the amphora image.
The "ubuntu" element is based on the ubuntu cloud image. This image includes
packages we do not need for the amphora image. At this point it's not clear
that Ubuntu will ship an 18.04 LTS cloud image in the format the "ubuntu"
element requires.
This patch switches the default Ubuntu amphora image to build with the
"ubuntu-minimal" diskimage-builder element.

This patch also moves the amphora agent into a virtual environment inside
the amphora.

It also sets up support for Ubuntu 18.04 (bionic beaver) and HAProxy 1.8.

Change-Id: I84a85ca1363bce2e0f13da64540ec7ba3575e818
2018-04-16 18:07:31 -07:00
Bernard Cafarelli
579eaafe7f
Do not install pip in amphora when using distribution packages
pip-and-virtualenv element is not needed in that case, and can cause
image build failures in environments without direct Internet access

Change-Id: I37616d76dd78ffb1419a898509e9466e7c54f69f
2018-02-02 13:12:26 +01:00
Bernard Cafarelli
d43d3fce86 Rework amphora agent installation element
Merge source and RHEL elements, allowing both source and package based
installations.

Allow amphora agent install from distribution packages (not limited to
RHEL)

Add a new option to diskimage-create.sh script to do so (default is kept
to source installation from Octavia git tree)

For now, amphorae built with distribution packages will have SELinux
(when available) running in permissive mode.

Made the rebind-sshd element generic to streamline the script
Use POSIX syntax for logrotate kill command

Change-Id: I391b2a95d54c7b9fd8f31d3e2c136ff9cc3451f1
2018-01-22 18:04:41 -08:00
Michael Johnson
2897b340f7 Fix health monitor type "PING" to use icmp ping
Currently with Octavia, if the user specifies a health monitor of type
"PING" we are still using a TCP connect to check for health.
This patch fixes that to actually ping the member to validate health.

Change-Id: I8a67efb7113ffa49b2805b37c3855373b17e5789
Story: 2001280
Task: 5826
2017-11-29 15:23:28 -08:00
Adam Harwell
8975d7acbd Merge keepalived elements
Change-Id: I2676b26ee59ad18583df937b258549bae241e100
2017-11-10 23:59:12 +00:00
Michael Johnson
e50ba48c1b Merge haproxy-octavia elements
Currently there are two haproxy-octavia elements, haproxy-octavia and
haproxy-octavia-ubuntu in the Octavia project.  The have minimal changes
between them, so this patch merges them into one element with a
backward compatible element left for haproxy-octavia-ubuntu.

Change-Id: I990802726d24e319988bfb614b3bf3fb560512b4
2017-11-09 09:09:01 -08:00
Adam Harwell
7bf8804177 Add flag to disable SSHD on the amphora image
Also deprecate the amp_ssh_access_allowed option.

Change-Id: Icb61a65fac57e74235fac904639c411b0fa2b495
2017-08-16 11:18:10 -07:00
Bernard Cafarelli
bb1c16b165
Fix invalid escape sequence in certs-ramfs.service
awk variable needs doubled '$' and '\' characters to be properly escaped
(and working properly)

Change-Id: I7703ad64e03c7afe52e49194e3bbed9f228b5760
Closes-Bug: #1689412
2017-07-18 15:53:59 +02:00
Jenkins
ac788fc304 Merge "DIB: drop custom mirror elements" 2017-07-16 20:39:21 +00:00
Bernard Cafarelli
d17dcea645
DIB: drop custom mirror elements
diskimage-builder supports a generic DIB_DISTRIBUTION_MIRROR that can
replace all existing mirror elements

Change-Id: Ia91dabf10e591f953440459edad35ebfc20c5890
Closes-Bug: #1703624
2017-07-12 10:57:24 +02:00
Bernard Cafarelli
1b3fdbef51 Install amphora agent from distribution package on RHEL
This ensures we use the appropriate amphora package (and also allows
amphora image creation with local repositories)

Repositories were already needed for openstack-selinux package, so this
does not change the build procedure

Change-Id: I837f73ec896405b9a648febfaf2cf0704458825b
2017-07-04 09:56:22 +00:00
Nir Magnezi
26a55415ab Auto-detect haproxy user_group
As a followup to Id99948aec64656a0532afc68e146f0610bff1378, adding auto
detection to haproxy_amphora.user_group

haproxy is capable[1] handling a list of configuration files.
This patch leverages that capability by simply providing haproxy with an
additional configuration file, which is baked in the amphora image via a
diskimage-builder element.

The above-mentioned element will specify the following values for user group:
Ubuntu: 'nogroup'
RHEL/CentOS/Fedora: 'haproxy'

The amphora-agent will parse and remove any user_group configuration provided
by Octavia controller worker.
This is in order to maintain amphora-agent backward compatibility to old
Octavia workers, who still provide user_group to the amphora-agent.
Octavia Workers that include this patch will no longer provide user_group
configuration to the amphora-agent.

[1] https://cbonte.github.io/haproxy-dconv/1.7/management.html#3

Related-Bug #1548070

Change-Id: Ia8fede9d7da4709a48661d1fc595a16d04fcbfa9
2017-04-23 18:24:23 +03:00
Bernard Cafarelli
0dd4649f37 Use cryptsetup/LUKS for encrypted ramfs
ecryptfs was dropped from RHEL/CentOS, use LUKS on a RAM-backed block
device (brd) instead.

Made the element name more generic

Added systemctl enable call in postinstall (for systemd init), so that
the service is correctly started and listed as wanted by amphora-agent

Change-Id: Id8c7ff93ae244ef14480e22c85dc79355a902105
Closes-Bug: #1642982
Closes-Bug: #1662952
2017-02-14 10:37:45 +00:00
Michael Johnson
6eb9a7cf17 Remove outdated kernel tuning for haproxy
This patch removes outdated kernel tuning parameters that were set
inside the amphora.  With current kernel versions the performance
issues no longer out weigh the benefits.

Change-Id: I6435257ec1f0ee0cc8c38df0d1ff0247707174e4
Closes-Bug: #1661105
2017-02-01 14:43:33 -08:00
Nir Magnezi
c00488143d Fix the amphora image support for RH Linux flavors
Not all Linux flavors accept the same type of configuration to manage
NICs. The amphora-agent must be able to distinguish between different
Linux flavors and choose the appropriate type of jinja2 NIC
configuration template for each one, respectively.

Up until now, The amphora-agent had no notion of the operating system
it is running on, therefore it used NIC configuration templates that
only match Debian based Linux flavors (mostly Ubuntu). Making it
unusable for flavors such as RHEL, Fedora and CentOS.

This fix enhances how the amphora-agent is handling NIC hot plugs.
It will use the appropriate jinja2 template by checking the Amphora
distribution name when needed.

Co-Authored-By: Michael Johnson <johnsomor@gmail.com>

Closes-Bug #1548070

Change-Id: Id99948aec64656a0532afc68e146f0610bff1378
2017-01-31 20:05:07 +02:00
ZhaoBo
8eddb0fd22 Fix multi-typo error in Octavia
This patch fix some typo in Octavia.

Change-Id: I2699f85f07e3207a0438b8127e9992c553fa40fc
2017-01-05 11:56:20 +08:00
Jenkins
87fdb48645 Merge "Add build-essential package dependency to amphora-agent element" 2016-12-20 18:14:03 +00:00
Bernard Cafarelli
32f3c4de8c
Add build-essential package dependency to amphora-agent element
With the recent changes, gcc is not pulled in anymore on CentOS, and
compilation of the python modules fails in the amphora-agent element

To be on the safe side, this adds the build-essential dependency
to make sure these modules can be buillt

Change-Id: I842b07cbc3e48209fd500bff5cc798be655f0ae9
2016-12-19 17:02:58 +01:00
Adam Harwell
bf8aac5561 Amphora-agent should log to a distinct location
This patch sets up a seperate log file for the amphora-agent
and logrotate to manage this new log.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Change-Id: Ia7b057642d7a567d685d989d1c689d5f3481e73e
2016-12-16 22:02:44 +00:00
Michael Johnson
34a47353f0 Fix the agent install to use pip -U
The previous patch that switched the agent install to use pip install
did not include the -U switch or the upper constraints flag.
This patch adds those to the amphora-agent element.

Change-Id: If907909704fcff4c7be10690eb4f50d0ba54b1c1
2016-12-15 13:17:18 -06:00
Gregory Haynes
3c5c13c69c Stop depending on tripleo-image-elements
There were only two elements we were using from this repo -
os-svc-install which we werent actually calling anywhere, we only
depended on it needelessly and sysctl. The sysctl element has been moved
in to dib as dib-sysctl so we can now stop depending on
tripleo-image-elements entirely.

Depends-On: If312d199388036d6f4103e94dca99249cb3bcbaf
Depends-On: Ia730850a48e2478fd5461710a9d2619408725cd8
Change-Id: Ie78c4f3ebe506214f0ce7c456fcbbee09d35ba2a
2016-12-15 13:17:18 -06:00
Gregory Haynes
83661b7bcc Use pip install rather than setup.py
pip install is greatly preferred over python setup.py install, so lets
use that to install our amphora-agent.

Change-Id: I5e4d169a1e6eb0e175f51943c08b025b09ffdc05
2016-12-09 23:32:40 +00:00
Bernard Cafarelli
013f44db21
Simplify cert-ramfs-ecrypt package dependencies
We only need ecryptfs-utils, as package managers will take care of the
sub-dependencies. Tested on an Ubuntu amphora, installed packages list
was identical.
This also fixes image creation on Fedora, as ecryptfs-utils package name
is identical there

Change-Id: Idab8c66e6bca137e79bef050fbaecd2f6c4add7a
Closes-Bug: #1640832
2016-11-18 16:00:35 +01:00
Michael Johnson
c4408c4c78 Adds support for systemd amphora images
This patch enables auto-detection of the init system used in the
amphora image and adds support for systemd amphora.
This patch allows Ubuntu xenial amphora images to work.
It also merges two functional test files into one file to reduce
code duplication.

This is a scenario gate fix.

Change-Id: I5fec1680bd47719ae9f2fcb6abaaba8a78e2ae8b
Closes-Bug: #1640866
2016-11-16 00:42:37 +00:00
Michael Johnson
f85e617a84 Terminated HTTPS certs and keys in encrypted ramfs
This patch adds an element that causes the terminated HTTPS
certificates and keys to be stored in an encrypted ramfs path
so they are encrypted at rest.

Change-Id: Id0f80f311d37d5691087e855fb1291011451c851
Closes-Bug: #1627370
2016-09-25 01:42:49 +00:00
Jenkins
71797b6a59 Merge "Deperecate apt-mirror element" 2016-08-25 02:56:09 +00:00
Gregory Haynes
f5c1549ebd Deperecate apt-mirror element
There are upstream methods for setting a mirror which are much more
flexible, lets just document and point users at them.

Change-Id: I442e4695d56086932403fb5893da89ead203c86d
2016-08-24 15:23:03 -05:00
Jenkins
0d52a66ecf Merge "Make pyroute2 a requirement of octavia" 2016-08-23 20:36:37 +00:00
Gregory Haynes
579e24b727 Make pyroute2 a requirement of octavia
We currently install this via its own element, but this could be made
more simple / obvious by using the python requirements tooling.

Change-Id: I2a2eed2b8ee481189c1694659ac8100062a25cf2
2016-08-22 17:56:00 +00:00
Gregory Haynes
bce460e367 Condense amphora-agent-ubuntu in to amphora-agent
This reverts commit fdde4e6984.

This was reverted due to an issue with starting the service on boot. A
trick of using symlinks was being attempted in order to refer to files
in the base source directory from an element. This seems to not work, so
copying init scripts in to the elments themselves.

Change-Id: If6ae07fa4ab4f39e2a339e17fbcc163b863ccd3a
2016-08-17 15:55:38 +00:00
Adam Harwell
fdde4e6984 Revert "Condense amphora-agent-ubuntu in to amphora-agent"
This reverts commit 687a972960.
Something about this seems to have possibly broken amphora boots.
Testing reverting this patch to see if it fixes the scenario runs.

Change-Id: I824cb1c6fa38f2320f84456a551585114215f625
2016-07-14 17:51:02 +09:00
Jenkins
ed45d5f9c4 Merge "Condense amphora-agent-ubuntu in to amphora-agent" 2016-07-13 21:10:18 +00:00
Jenkins
62c3353a44 Merge "Support RHEL base image in DIB" 2016-07-13 21:04:08 +00:00
Bernard Cafarelli
6e97a37744 Support RHEL base image in DIB
This adds a rhel type and gives some pointers on needed environment
variables for base image, registration, needed subscriptions.

no-resolvconf is also updated to work on rhel-type systems

Depends-On: Icb0e20b01479fea345e01309fc4bf3f7f639900c
Change-Id: Ic452feaf32fe7699ebd072f21dd630a2ac93bb5f
2016-07-12 10:24:37 +02:00
Gregory Haynes
687a972960 Condense amphora-agent-ubuntu in to amphora-agent
Creating separate elements per-distro is an anti-pattern for DIB. This
leads to a lot of code duplication across all the elements being used.
In DIB we try and create facilities to allow elements to easily perform
cross-distro tasks so this isn't necessary.

Change-Id: Ie3018eacfa0e2726449cfb580ded09d027bed349
2016-06-29 17:16:31 +00:00
Gregory Haynes
866dbb3339 Use git.o.o rather than review.o.o for cloning
This will hit the git mirrors which are scaled out much better than
gerrit.

Change-Id: I302ccd093f6cba36846ee5f3d86bc9dbee681e98
2016-06-29 17:15:35 +00:00
Gregory Haynes
aec22a79d3 Modernize amphora-agent element
There are some new facilities in DIB which simplify our element code -
declarative package installs, automated init script installation.

Change-Id: I8b91325999af5cfc9e455c4d6228453d8567c919
2016-06-29 17:06:17 +00:00
Stephen Balukoff
81c73bd1b5 Make SSH bind to management net only
Our present amphora image create scripts set up the ssh daemon on the
amphora to bind to the wildcard interface (which is the default).
However, this causes problems for anyone who tries to set up a listener
on TCP port 22, since haproxy will not be able to bind to the same IP.

This patch introduces a dhclient post-bind script to the amphora image
to gracefully rebind the SSH-daemon to only the load balancer management
net IP when it comes up on the network, solving the above use case. This
patch has the secondary benefit of making the amphora's SSH daemon only
respond to requests on the management network, which incrementally
increases the security of the amphora.

Change-Id: Iab93cec1f4dc4a2e37ad3cb8a92c132383dcda6a
Closes-Bug: #1551505
2016-03-04 13:58:33 -08:00
ptoohill1
63111acc90 Fixes Octavia Amphora RHEL based images
A few elements were needed to define specific repositories.

    amphora-agent: Updated and now buids with sysvinit instead
of upstart Octavia requries the use_upstart parameter to be set
to False to work with RHEL systems.
    amphora-agent-ubuntu: original element that supported upstart
    keepalived-octavia: Supports RHEL repositories

Change-Id: I961663d105ad09e7f05c77cb92efaa469f386fdc
Closes-Bug: #1531092
2016-02-03 15:51:48 -06:00
Stephen Balukoff
ea6fb75cda Optimizations for tempest tests
This patch does the following:

* Reduces interval between checks when waiting on
  amphora status in a devstack environment. At the same time we
  increase the number of retries so we are effectively waiting
  the same total time before a timeout error occurs.
* Disables DNS resolution on the amphora image. Amphorae don't
  need to do any DNS resolution anyway since any outbound
  connections are done specifying IP addresses. This means
  that the amphora never waits for DNS timeouts to occur
  when booting and performing other tasks when operating in an
  environment where DNS resolution doesn't work (ie. devstack)

In preliminary tests, the above optimizations seem to shave
about 50-55 seconds off per test. (This is mostly due to the
elimination of DNS timeouts.)

Change-Id: Icc7d8e0ac18a4a18ed9eb0950081ddd198cf4684
2016-02-01 13:36:17 -08:00
Michael Johnson
8c3e2e1602 Disable default haproxy process in amphora image
This patch disables the default haproxy process from starting in
the amphora image.
Closes-Bug: #1527691

Change-Id: Iccac03b982e47fc85f8e6beb0cc55a80b73ab465
2016-01-13 19:50:41 +00:00
Michael Johnson
12b1e57561 Fix sysctl settings for amphora image
This patch fixes two things:
1. The sysctl settings were failing because some conntrack modules
were not loaded anymore.
2. I fixed the sysctl-set-value scripts to be able to handle
multi-value settings, such as tcp_rmem, in tripleo-image-elements[1].
Here I have removed the workaround we had in Octavia.

[1] https://review.openstack.org/#/c/134616/

Change-Id: Ib7ab4f487c1b792b70a110098bf7a28cb565ee55
Closes-Bug: #1527392
2015-12-18 01:01:19 +00:00
Jenkins
c85ccfc82e Merge "Add backport keepalived" 2015-12-04 19:39:46 +00:00