Convert all code to not require six library and instead
use python 3.x logic.
Created one helper method in common.utils for binary
representation to limit code changes.
Change-Id: I2716ce93691d11100ee951a3a3f491329a4073f0
Eventually hacking will move to 2.0.0 (diskimage-builder
is holding it back), and when it does there will be a few
errors to fix. We can get ahead of it so it doesn't
break us with some small changes for these items:
F601 dictionary key $item repeated with different values
F632 use ==/!= to compare str, bytes, and int literals
E501 line too long
While doing this noticed the lower-constraints.txt for
hacking was set at 0.12.0, when test-requirements.txt
had it at 1.1.0, so fixed that as well.
Change-Id: I80d2a5f97e7a4896a8fa765c1971c8bb7e72d211
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Change-Id: I72520a2ca010c2c27315d9dff839a4f9d7540b6b
Code was not using the correct filenames for the 'route',
'route6', 'rule' and 'rule6' files on Red Hat images.
Changed to use config option 'agent_server_network_file'
if it's specified, else the file of the correct name, and
added unit tests for each.
Change-Id: I335287da66524d026f0c42086d885b478c568bbd
Task: 37881
Story: 2007051
Fix unit test test_cmd_get_version_of_installed_package_mapped when run
on a CentOS 7 system.
testtools.matchers._impl.MismatchError: !=:
reference = 'rpm -q --queryformat %{VERSION} haproxy'
actual = 'rpm -q --queryformat %{VERSION} haproxy18'
Change-Id: I81f76afc7e3fa6f190c30f33198197cb627cce26
The single process patch changed the way listeners and load balancers
are deployed inside the amphora. This caused listeners with SNI
enabled to load all of the certificates for all of the TLS enabled
listeners on a load balancer.
This patch corrects that by configuring each listener with a
specific list of certificates.
Change-Id: I2f3c7ab4137dbd84d77a6a6b675975af406249d0
Story: 2006758
Task: 37252
Mocks were done slightly incorrectly for pyroute2, correct this and add
a missing mock for os.path.isfile since it only coincidentally worked.
Change-Id: Id178784ebb4a9da348d295201fd6aa715c5b4abe
Re-factored code to not insert a log-format line in haproxy
config when 'no log' is set in the template. Adjusted tests
as necessary.
Story: 2006181
Task: 35706
Change-Id: I8a0cdf7ee912d39d9ac24244728094a1d1e15c63
Multi-listener LB commit (Idaccbcfa0126f1e26fbb3ad770c65c9266cfad5b)
introduced a v2 message for octavia healthmonitor.
This commit fixes an issue with healthmonitor messages for UDP
listeners, they didn't follow the v2 message specification: pools
dictionaries were stored in listener objects (v1 format) instead of
being stored as in the root dictionary of the message.
Story: 2005736
Task: 33394
Change-Id: I93e5eb5bc69fe4de4c450c09367b319769ef07db
Currently jinja_combo.build_config method expect to use single
tls cert, though with multiple listeners there could be multiple
certs. Also in case of HTTP and TERMINATED_HTTPS listeners on the
same loadbalancer - creation of the second listener will fail.
Change-Id: Iad3b55e5add4283256f7836c3d4a501aa57ffc2f
Story: 2006513
Task: 36510
Currently the amphora agent will lookup interfaces using the
interface name determined earlier in the plug method. This can
lead to a race condition with the udev interface renaming rule.
This patch changes the interface lookup to use the MAC address
directly and not rely on the interface name.
Story: 2006300
Task: 36013
Change-Id: I5bc21d5abdeb67a3a8ae88456735643463f15694
In some environments running older versions of gunicorn in the
amphora image, gunicorn can fail to start do to /dev/log socket
issues (timing, configuration, etc.).
This patch sets up a dedicated rsyslog socket /run/rsyslog/octavia/log
for gunicorn and haproxy to use. This should resolve any issues with
systemd overriding the /dev/log socket.
This also bumps the gunicorn minimum verison to 19.9.0.
Change-Id: I1e1ad8fde2ad8c1ffba95b1867afb130503b0a5b
This commit fixes pool and members status when using UDP loadbalancers.
Story: 2005736
Task: 33394
Change-Id: I75cde3ff820f085aebbdffd1e40c5ff40f16835d
Load balancers with multiple listeners, running on an amphora image
with HAProxy 1.8 or newer can experience excessive memory usage that
may lead to an ERROR provisioning_status.
This patch resolves this issue by consolidating the listeners into
a single haproxy process inside the amphora.
Story: 2005412
Task: 34744
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: Idaccbcfa0126f1e26fbb3ad770c65c9266cfad5b
Pyroute2 recently changed the way they import their base namespace
ojects. Now they are somewhat platform dependent, so on OSX unit tests
will fail for lack of linux-specific libs. Using `create=true` allows
the mocks to be created even if the library doesn't think they exist.
b69999ec82
Change-Id: I8f9c25e979a7e86ff0b8e19267e4525817802ba2
An exception handler in the amphora-agent has a python3 string
comparison bug that will cause a TypeError.
This patch fixes that bug and adds test coverage for the
start_stop_listener.
Change-Id: I6f5d95c5f875edda530f54ae72386d6495235ca6
Story: 2005898
Task: 33760
Configure rsyslog to forward logs to a target host
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Story: 1665069
Task: 33646
Change-Id: I00703f86555cbb574b943794b14a36fbc644f1b2
This patch configures the primary components of the amphora to log
to syslog using consistent logging facilities.
By default, user traffic logs will go to LOG_LOCAL0 and the amphora
processes (haproxy, keepalived, etc.) will log to LOG_LOCAL1.
This is a patch supporting log offloading.
Change-Id: Ifda91e0310e812e34f1e398dd3176af8a9c58f89
Story: 1665069
Task: 5486
Occasionally the test jobs[1] will fail with:
octavia.amphorae.drivers.haproxy.rest_api_driver [-]
Could not connect to instance. Read timed out. (read timeout=120.0)
This patch increases the default read timeout to 180 and changes the
directory copy that would subsequently fail to be more idempotent.
[1] http://logs.openstack.org/09/613709/14/check/ \
octavia-v2-dsvm-scenario-two-node/d83db12/controller2/logs/ \
screen-o-cw.txt.gz#_Feb_08_21_58_23_919928
Change-Id: Ia0bd6762c2605ce240a549b3e90e5c44b65897a5
This patch validates that a flavor is compatible with using spares
pool amphora. It will also update the amphora-agent config after
a spares pool amphora has been allocated.
This patch enables the ability to update a running amphora's agent
configuration and have the mutatable options be adopted.
The following amphora agent configuration options can be updated:
heartbeat_key
controller_ip_port_list
heartbeat_interval
loadbalancer_topology
This patch adds the support to the amphora-agent and the amphora
driver. A follow on patch will expose this capabililty via the
amphora admin API.
Change-Id: I97bdf5188808193516509f20767e82c0f8d2f5a5
This patch fixes an oversight in the addition of flavors support in the
amphora driver[1]. The amphora-agent configuration file was still getting
the topology selected in the configuration file as opposed to the
topology selected in the flavor.
This is an additional patch at the end of the chain as it leverages
changes that were made in later flavor patches that pass the flavor
into the taskflow flow.
A follow on patch will address spares pool amphora.
[1] https://review.openstack.org/#/c/621323
Change-Id: I4c2b398b562970f128e06794690ffd7c2977db08
Currently we calculate prefix based on netmask when writing the vip
interface file. Since netmask has been converted to prefix in ipv6,
this patch will avoid converting it to prefix twice which could
result in a wrong prefix length.
Also fix a bug in another test that relies on osutils, but wasn't
mocking correctly.
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: I9ee0cce12a975f4ab8f3df2707b355aab35c6cb3
Octavia is plugging member networks, but only bringing up the first
fixed IP address on that network. This can mean that a secondary
fixed IP on the network, such as the IPv6 address is not brought
up at member creation time.
Change-Id: Ic5b19a303e53ab62875c4fc4be6ac03f926a6832
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Story: 2004113
Task: 27535
Load balancers with IPv6 VIP addresses would fail to create due to
a duplicate address detection issue. The keepalived process would also
crash with a segfault due to a known bug[1].
This patch resolves both issues and allows load balancers with IPv6
VIP addresses to be created in active/standby topology.
[1] https://github.com/acassen/keepalived/issues/457
Story: 2003451
Task: 24657
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Change-Id: I15a4be05740e2657f998902d468e57763c3ed52e
Currently the amphora-agent is not reporting UDP listener health
when the UDP listener does not have a pool and members.
This patch changes that behavior to report the listener as healthy
if the keepalived process is started and running in the amphora.
This patch also introduces message versioning for the health
heartbeat messages.
It also corrects a few assertEqual tests that had the reference and
actual values backwards.
Change-Id: Ifc28b4991852e59c0d27b4ab3d1afc4e9965e88b
Story: 2003592
Task: 24911
This patch adds a few optimizations when using the amphora driver.
1. It increases the amp_active_retries from 10 to 30. This increases
the time we wait for nova to mark an instance "ACTIVE". The old default
of 10 was one minute forty seconds, but in some clouds it's been observed
that the nova schedule can get overloaded and take longer than a minute
forty to schedule the instance. Setting this to 30 means we will wait
five minutes for nova to schedule the instance.
2. It enables TCP kernel splicing in HAProxy. This has been shown to
reduce the CPU overhead for very high rate TCP load balancers.
3. Finally it enables "safe" HTTP keepalives on the backend member
connections [1]. This increases the request rate possible while using HTTP
protocol listeners and members.
[1] http://cbonte.github.io/haproxy-dconv/1.6/configuration.html#4-http-reuse
Change-Id: I3af009cac9a9edc8aef793b52c6a1488fde2c59b
Provide a sane return value for get_keepalivedlvs_pid
to ensure that underlying Python libary code does not
throw a TypeError when joining values:
TypeError: join() argument must be str or bytes, not 'MagicMock'
Change-Id: I174bbaf618a845cc246f3b1926d600d03e710de3
In Pike[1], we introduced a user_group auto detection for haproxy.
The default user group name is auto-detected for any OS distribution
we support as a base for Amphorae.
user_group remained as an option for admins but was also
marked deprecated in Pike[2].
This patch removes that option altogether.
Story: 2003323
Task: 24357
[1] Ia8fede9d7da4709a48661d1fc595a16d04fcbfa9
[2] https://review.openstack.org/#/c/429398/45/octavia/common/config.py@175
Change-Id: Iddd4162674f116705d2b47062cbf7ca88f2677a6
1. Remove 'inhibit_on_failure' option from realserver configuration block.
Then when the member is failed which means the MISC_CHECK is failed,
the amp agent can not get the member info from kernel ip_vs file, and
it will return status DOWN to health manger, then the member status
will be ERROR, and its associated pool and loadbalancer will be in
DEGRADED status.
Alternative:
Using notify_up/notify_down to write the status into a member specific
file, then we can read the member status from that file directly.
Change-Id: I940681b8109b05334ebb5befc58b43f08d2a1148
Story: 2003198
Task: 23354
1. Removes the misc_dynamic setting from the UDP-CONNECT health monitor
as our script does not use it.
2. Adds a release note for the UDP features.
3. Updates the API reference for UDP support.
4. Adds a comment to the keepalived config with the LB ID.
5. Updates the status message type to be the correct UDP protocol.
6. Fix error during deleting a listener if there are multiple amphoraes.
7. Refactors systemd service script handling.
Story: 2003306
Task: 24258
Change-Id: I09240023d066ac5a71836d01045cda6ce5678712
These files will split with the current Octavia repo, before other parts
are ok.
Patch List:
[1] Finish keepalived LVS jinja template for UDP support
[2] Extend the ability of amp agent for upload/refresh the keepalived
process
[3] Extend the db model and db table with necessary fields for met the new
udp backend
[4] Add logic/workflow elements process in UDP cases
[5] Extend the existing API to access udp parameters in Listener API
[6] Extend the existing pool API to access the new option in
session_persistence fields
Change-Id: Ib4924e602d450b1feadb29e830d715ae77f5bbfe
When using the Octavia/amphora driver, unspecified or unlimited (-1)
settings would lead to a 2000 connection limit in HAproxy.
This patch updates that to be 1,000,000 connections.
1,000,000 was selected to amphora memory usage at a reasonable level.
Change-Id: Iddeb62412bb71b69cf1e9198be6131c59a3051b0
Story: 1635416
Task: 5159
This is useful for debugging purposes, and could theoretically be
interesting data for custom elements.
Story: 2001236
Task: 5756
Change-Id: If5f7793bd16fd6c8841586f29cdcf5e53908b1fe
Brian Haley