This patch changes the [haproxy_amphora] connection_max_retries and
build_active_retries default values from 300 to 120. This means load
balancer builds will wait for ten minutes instead of twenty-five minutes
for nova to boot the virtual machine.
We feel these are more reasonable default values for most production
deployments and provide a better user experience.
Only environments running in nested virtualization, without nested
virtualization enabled in the hypervisor could require a value as high as
300.
Depends-On: https://review.openstack.org/637074
Change-Id: I46be11062fb15ed21169fbec5dc8451a588273a5
This patch adds Cloud Auditing Data Federation (CADF) auditing support to the
Octavia API. This is implemented using the keystonemiddleware audit filter.
Change-Id: I87a7e15171dfaf28b6ed97ca71d4423d18fbdbea
This patch adds a few optimizations when using the amphora driver.
1. It increases the amp_active_retries from 10 to 30. This increases
the time we wait for nova to mark an instance "ACTIVE". The old default
of 10 was one minute forty seconds, but in some clouds it's been observed
that the nova schedule can get overloaded and take longer than a minute
forty to schedule the instance. Setting this to 30 means we will wait
five minutes for nova to schedule the instance.
2. It enables TCP kernel splicing in HAProxy. This has been shown to
reduce the CPU overhead for very high rate TCP load balancers.
3. Finally it enables "safe" HTTP keepalives on the backend member
connections [1]. This increases the request rate possible while using HTTP
protocol listeners and members.
[1] http://cbonte.github.io/haproxy-dconv/1.6/configuration.html#4-http-reuse
Change-Id: I3af009cac9a9edc8aef793b52c6a1488fde2c59b
When queue_event_streamer driver is used and RabbitMQ
is down, stats update processes occupy the thread pool
which is shared with health update processes. Then,
RabbitMQ down unexpectedly leads to delete all existing
amphorae. This commit separates the thread pool and aims
to keep the existing amphorae working even when RabbitMQ
is down.
Change-Id: I576687f5b646496ff3a00787cf5e8c27f36b9448
Task: 22929
Story: 2002937
This patch adds a configuration option for reserved IP addresses that
cannot be used for load balancer member addresses. By default, this will
include the nova metadata service address 169.254.169.254.
Change-Id: I25de5ed5f6f35afc55dd1154c3e02934fddb100a
Story: 2003413
Task: 24555
In Pike[1], we introduced a user_group auto detection for haproxy.
The default user group name is auto-detected for any OS distribution
we support as a base for Amphorae.
user_group remained as an option for admins but was also
marked deprecated in Pike[2].
This patch removes that option altogether.
Story: 2003323
Task: 24357
[1] Ia8fede9d7da4709a48661d1fc595a16d04fcbfa9
[2] https://review.openstack.org/#/c/429398/45/octavia/common/config.py@175
Change-Id: Iddd4162674f116705d2b47062cbf7ca88f2677a6
Add new types into db table.
Extending the existing API, including Listener, Pool, HealthMonitor for
UDP fields support.
For healthmonitor part, need to wait for other patch to fix the default
value.
Patch List:
[1] Finish keepalived LVS jinja template for UDP support
[2] Extend the ability of amp agent for upload/refresh the keepalived
process
[3] Extend the db model and db table with necessary fields for met the new
udp backend
[4] Add logic/workflow elements process in UDP cases
[5] Extend the existing API to access udp parameters in Listener API
[6] Extend the existing pool API to access the new option in
session_persistence fields
Story: 1657091
Task: 5484
Change-Id: If728705f142f4195fe624bd9ef17413722d54fe3
These files will split with the current Octavia repo, before other parts
are ok.
Patch List:
[1] Finish keepalived LVS jinja template for UDP support
[2] Extend the ability of amp agent for upload/refresh the keepalived
process
[3] Extend the db model and db table with necessary fields for met the new
udp backend
[4] Add logic/workflow elements process in UDP cases
[5] Extend the existing API to access udp parameters in Listener API
[6] Extend the existing pool API to access the new option in
session_persistence fields
Change-Id: Ib4924e602d450b1feadb29e830d715ae77f5bbfe
Currently when the command to generate a policy file is used, it will
generate a yaml based file, but call it json.sample.policy. This patch
forces it to generate a json file, and changes the default name to
policy.json.sample.
Change-Id: I4d04f4f37385bd4601f566238aaba43dcbe74780
If a load balancer loses more than one amphora at the same time
the failover process will fail and leave the load balancer in
provisioning status ERROR.
This patch resolves this by failing over one amphora at a time
marking any amphora that are also failed in status ERROR. The health
manager will then failover the other failed amphora in subsequent checks.
This patch will update multiple healthy amphora in parallel and will
timeout failed amphroa using the new "active_connection_max_retries"
configuration setting used for "fail-fast" connections.
The patch also updates the amphora failover flow documentation to
show the full flow and not just the spares failover flow.
It updates the amphora driver "get_diagnostics" method to pass instead
of error.
It also adds a AmphoraComputeConnectivityWait task to explicitly wait
for a compute instance to come up and be reachable. This allows a longer
timeout and clarifies this may fail due to compute (nova) failures.
Previously the first plug vip task would do this wait.
Change-Id: Ief97ddda8261b5bbc54c6824f90ae9c7a2d81701
Story: 2001481
Task: 6202
This patch addresses the following:
Fixes some unit tests.
Cleans up some code from the parent patches,
Adds a release note for the provider driver support.
Adds the "List providers" API.
Adds a document listing the know provider drivers.
Adds a provider driver development guide.
Change-Id: I90dc39e5e9d7d5839913dc2dbf187d935ee2b8b5
Story: 1655768
Task: 5165
This patch adds provider driver support to the Octavia v2 API, starting
with the load balancer API.
This patch also creates a provider driver for Octavia, initially fully
implementing the load balancer methods.
Follow on patches will implement the remain parts of the API.
Change-Id: Ia15280827799d1800c23ed76d2af0e3596b9d2f7
Story: 1655768
Task: 5165
PING is a trap. There is no real-world scenario where PING is the option
that makes the most sense, but people are familiar with it, and it seems
"simple", so they pick it. This needs to stop. Empower operators to
disable this!
Change-Id: Ifa80b7a5973361c13f2e6611789aa9798325ece0
Option auth_uri from group keystone_authtoken is deprecated[1].
Use option www_authenticate_uri from group keystone_authtoken.
[1]https://review.openstack.org/#/c/508522/
Change-Id: If6eee4ecfb4c6c607c9ee762cc535cf5d6180d88
* Switch to ProcessPool from ThreadPool (Python threads are horrible)
* Load the health/stats update drivers correctly with stevedore
* Add logging driver
Change-Id: Icda4ff218a6bbaea252c2d75c073754f4161a597
*NOT* deprecating the old way of storing these, as I believe that would
create a huge mess for anyone already using it.
Change-Id: I1fee174d8b8956f3d2053781a7f18c2940b21765
This patch is the initial implementation of a distributor driver for
Octavia Active/Active topology support.
This patch is a decompostion of the following patch:
https://review.openstack.org/#/c/313006
Story: 2001288
Task: 5836
Depends-On: I97b52b80efb33749647229a55147a08afa112dd2
Change-Id: I65e4a533caee692e1c98e8c6586c2e2132f2e34c
Co-Authored-By: Valeria Perelman <perelman@il.ibm.com>
This adds a way to configure the event streamer transport URL
so it can post to a different queue, e.g. Neutron's
Change-Id: I69d3d6d30e33878052f2c56b8c79a14cc4ec1b24
In large build situations, nova can be slow to build VMs, this means that the
default 100 second timeout may expire before the final status has been updated
in the neutron database. This patch will emit provisioning status to be sync
with neutron db
Change-Id: If6c0b81630fd1911518792d9947f8622f065ff4e
This patch makes developer debugging of Octavia easier. It adds
a configuration option that disables the controller worker taskflow
flows from reverting and cleaning up resources.
It also changes the amphora agent to keep a copy of a haproxy
configuration that failed validation.
Change-Id: Iaca070a0ab9589fb25513eb5fad7d1e99974d572
Also, create a section for API settings `api_settings` and move some
related settings there.
This patch also enables the configuration settings to be logged
when the api process is started if debug is True.
Change-Id: I31671789d186c4b8a775cc12a414acd2d439512d
This patch adds policies and enforcement to the Octavia v2 API for
load balancers and listeners. Child patches will add the rest of the API.
In this patch I also correct some improper functional tests.
Change-Id: Id8a2d15c117c54bd45fc8bb76bf71aff1b3c8fe9
Closes-Bug: #1690481
keystonemiddleware has deprecated the "signing_dir" configuration
option [1].
This patch also removes reference to it from octavia.
[1] https://review.openstack.org/#/c/391405
Change-Id: Idda46ab1459584eafd58097ec42b9f0fcea41759
This option was NEVER read, so there is no point in continuing to allow
it to be configured (it is pointless).
Change-Id: I147abdd8d3d95164168ec606f5b92401cb24d1fe
Closes-Bug: #1691286
Use glance sorting and pagination from inside the SQLAlchemy query
to handle the sorting and pagination for octavia.
Change-Id: I5489c5c89691b8871e32caf3f85ab1978bc3618c
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Co-Authored-By: Lubosz "diltram" Kosnik <lubosz.kosnik@intel.com>
Closes-Bug: #1596628
Closes-Bug: #1596625
The current default for auth_strategy is noauth, which is not how it is
expected to be set in production environments.
Note: Functional tests should be good with noauth.
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Change-Id: Ifc80fff06a1d793d7cee0b207af10061784e48db
As a followup to Id99948aec64656a0532afc68e146f0610bff1378, adding auto
detection to haproxy_amphora.user_group
haproxy is capable[1] handling a list of configuration files.
This patch leverages that capability by simply providing haproxy with an
additional configuration file, which is baked in the amphora image via a
diskimage-builder element.
The above-mentioned element will specify the following values for user group:
Ubuntu: 'nogroup'
RHEL/CentOS/Fedora: 'haproxy'
The amphora-agent will parse and remove any user_group configuration provided
by Octavia controller worker.
This is in order to maintain amphora-agent backward compatibility to old
Octavia workers, who still provide user_group to the amphora-agent.
Octavia Workers that include this patch will no longer provide user_group
configuration to the amphora-agent.
[1] https://cbonte.github.io/haproxy-dconv/1.7/management.html#3
Related-Bug #1548070
Change-Id: Ia8fede9d7da4709a48661d1fc595a16d04fcbfa9
Introduces a new config parameter to specify the anti-affinity
policy.
Bumps nova version.
Closes-Bug: 1677604
Change-Id: I8c50057bd43873182058097e802bc839d1be0554
This patch limits the number of Amphora build requests handled by the
controller worker at a given time.
Also, the amphora build requests are assigned priorities based on
whether it is a normal loadbalancer create, failover or spares pool
loadbalancer create request. Based on the priority and the order in
which the requests were made if there is an available build slot the
amphora will be built.
Co-Authored-By: Lubosz "diltram" Kosnik <lubosz.kosnik@intel.com>
Change-Id: I967cf0668f82fb3a63e18dc7a457c58b526b7e66
Closes-Bug: #1571802
oslo.log has deprecated the verbose option [1] so we should remove
mention of it as well.
[1] https://review.openstack.org/#/c/206437
Change-Id: I322e7b635f7337a6d399d7728bf55e581160b823