Task CertComputeCreate requires SERVER_GROUP_ID but this dependency is
missing when nova anti-affinity is enabled and load balancer topology is
SINGLE. This patch fixes this issue.
Task: 40408
Story: 2007943
Closes-Bug: #1888397
Change-Id: I757d84e4624f488382ec4ff9af3f489cd9a85ed3
The configuration option tls_cipher_blacklist has been
deprecated and replaced by tls_cipher_prohibit_list.
Change-Id: I6152838c697e12d19b27343e3a0714e55ca52d88
With the removal of Python 2.x we can remove the unittest2 compat
wrappers and switch to assertCountEqual instead of assertItemsEqual
We have been able to use them since then, because
testtools required unittest2, which still included it. With testtools
removing Python 2.7 support [3][4], we will lose support for
assertItemsEqual, so we should switch to use assertCountEqual.
[1] - https://bugs.python.org/issue17866
[2] - https://hg.python.org/cpython/rev/d9921cb6e3cd
[3] - testing-cabal/testtools#286
[4] - testing-cabal/testtools#277
Change-Id: I3e8725eb77ea947e71d85ea406a60ed94c7bf971
The recent patch[1] that adds a neutron subnet lookup using the user
context is not honoring the interface/endpoint_type or region_name
settings for neutron in the octavia configuration file.
This is causing problems for deployments that use the "internal"
endpoint for neutron and the current code will always return
the "public" endpoint.
This patch corrects this problem by including those filter
parameters when the neutron endpoint is looked up in keystone.
[1] https://review.opendev.org/726042
Change-Id: I7b8f7c7d653b37395f9a660be67f954a3a6f26d9
Story: 2007863
Task: 40173
Running amphora failover against the amphora noop driver was raising a
TypeError (reload() takes from 2 to 3 positional arguments but 4 were
given).
Change-Id: I64172d6995959cf377364584ad9a2395f9ec0605
This patch refactors the failover flows to improve the performance
and reliability of failovers in Octavia.
Specific improvements are:
* More tasks and flows will retry when other OpenStack services are
failing.
* Failover can now succeed even when all of the amphora are missing
for a given load balancer.
* It will check and repair the load balancer VIP should the VIP
port(s) become corrupted in neutron.
* It will cleanup extra resources that may be associated with a
load balancer in the event of a cloud service failure.
This patch also removes some dead code.
Change-Id: I04cb2f1f10ec566298834f81df0cf8b100ca916c
Story: 2003084
Task: 23166
Story: 2004440
Task: 28108
Assertions were using the same expressions on both side: optionals and
lb_listener are both parameters to the API (and the lb_listener dict
contains all optionals items).
Those assertions should compare the parameters to the API results.
Change-Id: I6f372a3f82fdf4f41e661e640e4a983cf484ed6d
We run the octavia scenario test failed when the OpenStack env
enable TLS. So we need add the verify for the session.
Story: 2007662
Task: 39754
Closes-Bug: #1877818
Change-Id: Ie71db27dc383c93496c1dfd69f486a4fd02b597e
This patch changes 'defiend' to 'defined'
in the explanatory notes in octavia/
tests/functional/db/test_repositories.py
Change-Id: Ibb7f0f416a013b98edf72a5803aada71015cfade
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.
Also added and enabled a hacking check that would have caught this.
Change-Id: Idb10f84fd32c50db24f844352cb85de452181439
In https://review.opendev.org/#/c/613709/ octavia was
changed to use octavia-lib for a lot of API driver-related
code and deprecation warnings put in place. Now that
we're in Victoria remove all the deprecation shims and
use octavia-lib exclusively.
Change-Id: If92988150479a7daf465af5f8df22818664a0fce
Even though the API and database schema enforce a pool to have a load
balancing algorithm set and validate its value, the auxiliary transform
pool method had an invalid default value 'roundrobin'. Round robin in
LVS is 'rr'.
Change-Id: I72b669e7755c0851867453977946891d7074d92b
SNI certificates were not being set in the database on listener update.
A listener GET would not show the certificates in the sni_container_refs
attribute. Also, the API was allowing set of SNI certs on non
TERMINATED_HTTPS listeners.
Task: 39042
Story: 2007421
Story: 2007430
Change-Id: If5b6411a0b7c75441a406234c2792ea68d35d0fe
Current octavia has no l7policy and l7rule quota definitions. But
they are necessary for some scenarios. For example, implement
product design compatible with Neutron Lbaas.
Story: 2003382
Task: 24457
Change-Id: I09ee23dcb83f5f08a56e25cc05ff77caa3ad4230
A previous patch[1] missed batch_member_update when adding database
repository "get" method retries for new object creation actions.
This patch fixes batch member create to retry the database get call
when new members are being created via batch member update.
This issue only impacts the v1 amphora driver as the v2 driver
does not need to get these objects from the database.
Story: 2007581
Task: 39503
[1] 48e85569f7
Change-Id: Ia3476ab7b24dc3fd6e29ff2abe6eb6bacd9908ed
Add new configuration option "minimum_tls_versions" to octavia.conf.
Listeners, pools, or the default values for either will be blocked from
using lower versions.
Change-Id: Ifa0d695c2227772d6b37987a7857fe58ca660dc8
Story: 2006733
Task: 37171
Depends-On: I480b7fb9756d98ba9dbcdfd1d4b193ce6868e291
Add field tls_versions to pools for restricing TLS versions used.
This is a colon-separated string of versions to be used.
Available values (as defined in octavia-lib):
SSLv3, TLSv1, TLSv1.1, TLSv1.2, TLSv1.3
Add default_pool_tls_versions in octavia.conf
Note: TLSv1.3 connections will use haproxy's default ciphers
instead of the listener's tls_ciphers field
Change-Id: I480b7fb9756d98ba9dbcdfd1d4b193ce6868e291
Story: 2006733
Task: 37173
Depends-On: Ic33d9b9a256490ae1b048cdfd2475d6340509fdb
Add field tls_versions to listeners for restricting TLS versions used.
This is a list of versions to be used.
Available values (as defined in octavia-lib):
SSLv3, TLSv1, TLSv1.1, TLSv1.2, TLSv1.3
Add default_listener_tls_versions in octavia.conf.
Note that at this time TLS 1.3 ciphersuites are not impelemented,
so any TLS 1.3 connections will use haproxy's default ciphers
instead of what's specified by tls_ciphers.
Change-Id: Ic33d9b9a256490ae1b048cdfd2475d6340509fdb
Story: 2006733
Task: 37170
Task: 37169
Fixes failing unit tests in
octavia.tests.unit.certificates.manager.test_barbican_legacy.TestBarbicanManager
for Python 3.8
Some of the tests fail setting up a mock.Mock(spec=secrets.Secret)
because a ValueError exception is raised unexpectedly.
The reason is that test_get_cert_no_registration_raise_on_secret_access_failure
patches the `payload` property of barbicanclient.v1.secrets.Secret to
raise a ValueError.
When a subsequent test tries to set up a mock.Mock(spec=secrets.Secret)
in Python 3.8 the Mock class will try to look at the properties of the spec
class and accessing `payload` doesn't behave normally anymore: it raises
ValueError now.
Fixed by using a different approach of mocking `payload` in
test_get_cert_no_registration_raise_on_secret_access_failure
so that it does not influence subsequent tests.
Change-Id: Ic534a4715c85c2216c7251209507acf74a999153
Story: 2007490
Task: 39212
The base64_sha_string method is used to set a base64-encoded peer name
in HAProxy. There are cases where the peer name can start with
an hypen which is troublesome when used in HAProxy CLI. Specifically,
HAProxy fails to reload when local peer name starts with '-x' [1]. When
this is the case, an amphora goes to provisioning status ERROR and later
is scheduled for failover by the Octavia Health Manager service. A new
amphora UUUID is assigned and base64 encoded, hopefully not starting
with '-x' again. However, this is far from being ideal -- we incur in a
dataplane disruption (single topology) or reduce HA capabilities
(active-standby topology) for some time.
Four possible options:
a) add prefix to peer name
b) change b64encode altchars
c) quote peer name in haproxy CLI command
d) substitute first character if hyphen
Option a) and b) are not backward compatible with running amphorae. Peer
names of existing amphorae that do not start with hypen but contain
hyphen at any other position would get different peer names.
Option c) would nonetheless still require an amphora image update to add
quotes in the HAProxy init service file. Continuing to generate peer
names with hyphens at begininng of the string is avoidable and
recommended.
Option d), while also requiring an amphora image update, it would get
rid of hyphens in begining of the peer names. It is also backward
compatible with all running amphorae, except for those starting with
hyphen but are broken anyways.
This patch takes option d). It substitutes hyphen with 'x' character.
[1] https://github.com/haproxy/haproxy/issues/644
Task: 39850
Story: 2007714
Change-Id: Ib0fc26877710dea423a5ebcf1f71077665404377
This patch adds a warning log message that the Octavia API is being
run under simple_server.
Change-Id: Ida3acd6f3ba81facc929a4a2c3bd75c6971059c7
Story: 2007702
Task: 39818
Use token and endpoint URL to initialize neutron client for the
request user.
Story: 2007619
Task: 39641
Change-Id: I05a541a77f254a77ad5036e1062b61c8ce93b754
-w (timeout) option doesn't do anything in nmap-ncat (default netcat in
CentOS/RHEL) for UDP datagrams, and nmap-ncat has a default idle timeout
set to 2 seconds.
We can get the same behavior as netcap-openbsd (Debian/Ubuntu) by
setting that idle timeout (-i) option to 1 second.
This commit detects the flavor of the netcat binary (nmap vs other) and
uses it to adapt the parameters.
Story: 2007688
Task: 39800
Change-Id: I0100aaa428477f011bd39a90dd4ec98199b4bebc
E741 ambiguous variable name 'l'
Change 'l' to another variable in affected code.
Also had to set the latex_engine to 'xelatex' in doc/source/conf.py
in order to get past an openstackdocstheme change the broke the pdf
doc build.
Change-Id: Idd176e40ccf2a79832a5c99140bd30e5e1f9c0d8
Add new configuration option "tls_cipher_blacklist" to octavia.conf.
Blacklisted ciphers are blocked from being used in listeners, pools, or
default cipher strings.
Change-Id: I44fd4da1b47faee9cc01b9426898a28b6f13f223
Story: 2006627
Task: 37168
These classes were deprecated in Stein and marked for removal in Ussuri.
By removing these classes, we fix pep8 issues (catching-non-exception)
we started seeing at the gate with the release of astroid 2.4.0.
Change-Id: I66b2d0687f4edfbcbff99c29c9c5d539e4e7ea7f
When disable a loadbalancer have listener, the Heath Manager not update
amphora heath make it keep failover after heartbeat_timeout end time
Story: 2007587
Task: 39521
Change-Id: Ia6d3f40ae1b9b352492162513c9262748ee67e6f
* Make sure the user has access to the subnet in the request for
creating or updating pool member.
* Make sure the user has access to port or subnet or network for
creating load balancer
Story: 2007531
Task: 39339
Change-Id: I479019a911b5a1acfc1951d1cbbc2a351089cb4d
The python 3.8 unit test jobs are failing on a
mock.Mock(spec=secrets.Secret) call. Since this is legacy code and
we don't really need to be testing python-barbicanclient code here,
I'm removing the spec= for these tests to stop artificial failures
when running unit tests on python 3.8.
Change-Id: I9132e2d05bf67ddff4f2d7879d799c8dd25bd3c7
This patch adds a test skip for two tests that are impacted by the
recent sqlalchemy 1.3.16 release.
With this release, a patch[1], changes the default commit behavior
of a transaction. With this change we are seeing that the load
balancer created in the tree-create test disappears from the
transaction context during the test and the pool create call will
throw a foreign key error as the load balancer is not in the database.
It's not clear if this is purely a sqlalchemy, pysqlite, or sqlite3
bug at this time.
Given the requirements are already in freeze for the Ussuri release,
we are opting to disable the tests (we know only sqlite is impacted),
instead of attempt to blacklist 1.3.16 in requirements.
[1] 9ebbf8614a (diff-e9762e21a27d8e6c44db6f9dd4edc694R455)
Change-Id: I7910ebe4cff692bab67349bbf3e4ee4e24b5fa7a
This patch introduces 2 macros in lvs.
1. Support HTTP GET, allow users create HTTP healthmonitor for udp pool.
2. Support TCP check, allow users create TCP healthmonitor for udp pool.
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: I61c7d8d4df54710a92b8c055be84bba29bf3d7e6
Story: 2003200
Task: 23356
Story: 2003199
Task: 23355
Run taskflow jobboard conductor only if amphorav2 provider is
enabled.
Fixes devstack plugin.sh conditions for amphorav2 provider.
Change-Id: I49b587cf748996658859667485400307205d209b
Flask's stream always returns bytes, file write always takes string.
This causes py3 amps to return 500 on cert rotation AND wipe out the
certificate, so the amphora are no longer controllable and go to ERROR
state. Anyone running py3 amps prior to this patch will experience
amphorae breaking on a timer due to housekeeping cert rotation!
Change-Id: I831b0b48d719397c14d80f8ebcbad997c50c7795
Template was using timeout value instead of delay value.
Also clean up redundant values in the templates (things like retry and
delay_before_retry only need to be specified once at the top level).
Change-Id: I376917e40eb7a92f7f03e691ed9a0c23fd2ce8f8
Introduce TaskFlowServiceController which uses taskflow
jobboard feature and saves jobs info into persistence backend.
Jobboard could be operated via RedisTaskFlowDriver or
ZookeeperTaskFlowDriver, that could be set via the config.
RedisTaskFlowDriver is intoduced as default backend for jobboard.
Usage of jobboard allows to resume jobs in case of restart/stop
of Octavia controller services.
Persistence backend saves state of flow tasks that required in
case of resuming job. SQLAlchemy backend is used here.
Bump taskflow version to 3.7.1 and add dependency to
SQLAlchemy-Utils (required for taskflow sqlalchemy
backend support).
Story: 2005072
Task: 30806
Task: 30816
Task: 30817
Change-Id: I92ee4e879e98e4718d2e9aba56486341223a9157
Pools can now be each be assigned an OpenSSL cipher string with the
field tls_ciphers. A new configuration option, default_pool_ciphers,
specifies what cipher string to use for new tls-enabled pools
if one is not explicitly specified at time of creation.
Change-Id: Iedb7774bfb8d70ea307d6a513248e1fe2389fa34
Depends-On: I77da6f14063877af0077f2c12df1aab5d5ead187
Story: 2006627
Task: 37172
A recent change in oslo.policy has made it register its cli opts on
the global config object. This was done to fix a bug where the opts
passed to the oslo.policy cli tools would get lost once it called
into project code because it was previously using a private config
object.
Octavia had already fixed this bug in a different way by filtering
the args in the enforcer code, which should no longer be necessary
now that the oslo.policy fix has merged.
However, the use of the global config object by the policy cli has
introduced a new problem, which is that after the config object is
initialized you can't register more cli opts. Because Octavia was
registering cli opts on import, this means that when the policy
tools call the Octavia policy entrypoint those opts get registered
and cause a failure.
To fix that, this change moves the cli opt registration into a
function that gets called from config.init so they will only get
registered when running an actual Octavia service. A separate
function was needed because they also need to be registered in
unit tests, and we don't want to actually initialize the entire
config object there. This way they can be initialized properly
in both scenarios.
Change-Id: I48ae260335f67e8ab1a188a94e44a7f1968e6fe9
Listeners will now be able to each be assigned their own OpenSSL
cipher string with a new field: tls_ciphers. There is also a new
configuration option, default_listener_ciphers, which specifies the
cipher string to assign to new listeners when one is not explicitly
specified.
Change-Id: I77da6f14063877af0077f2c12df1aab5d5ead187
Depends-On: Id5f4c20abd40dd092558a711987953012d4ae67f
Story: 2006627
Task: 36839
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Fix problems found.
Update local hacking checks for new flake8.
Add test-requirements.txt to doc building, hacking is now imported and
needed for autodoc.
Change-Id: I06211ef20131c64deba135123e53d87f3b5356a2
Should have done "pad to 8 characters" on the hex conversion, but it was
instead hardcoded to pad a single `0`, which is right in a lot of cases
but not all.
For example:
>>> ip1 = ipaddress.ip_address('98.136.140.23')
>>> ip2 = ipaddress.ip_address('10.1.1.1')
>>> "%X" % ip1._ip
'62888C17'
>>> "%X" % ip2._ip
'A010101'
Change-Id: Ia9fec4e72c00f7086489b245d9dc50ed9c27f12a
healthcheck middleware adds a /healthcheck url that allows
unauthenticated access to provide a simple check when running
octavia-api behind a load balancer
https://docs.openstack.org/oslo.middleware/latest/reference/healthcheck_plugins.html
Co-authored-by: Michael Johnson <johnsomor@gmail.com>
Change-Id: I10db6226750f7b7c703067d2ab82eea3a9875112
Adam Harwell
Carlos Goncalves
Brian Haley
melissaml
Michael Johnson
Gregory Thiemonge
ramboman
zhaoleilc
Hervé Beraud
Yang JianFeng
Dawson Coleman
Bodo Petermann
chimeng
Lingxian Kong
archiephan
ZhaoBo
Ann Taraday
Ben Nemec
Andreas Jaeger
Sam Morrison