147 Commits

Author SHA1 Message Date
Michael Johnson
34edb58c12 Set the default retries down to 120
This patch changes the [haproxy_amphora] connection_max_retries and
build_active_retries default values from 300 to 120. This means load
balancer builds will wait for ten minutes instead of twenty-five minutes
for nova to boot the virtual machine.

We feel these are more reasonable default values for most production
deployments and provide a better user experience.

Only environments running in nested virtualization, without nested
virtualization enabled in the hypervisor could require a value as high as
300.

Depends-On: https://review.openstack.org/637074
Change-Id: I46be11062fb15ed21169fbec5dc8451a588273a5
2019-02-14 15:38:48 -08:00
Michael Johnson
e60cd189fb Add auditing support to the Octavia API
This patch adds Cloud Auditing Data Federation (CADF) auditing support to the
Octavia API. This is implemented using the keystonemiddleware audit filter.

Change-Id: I87a7e15171dfaf28b6ed97ca71d4423d18fbdbea
2018-12-11 23:32:58 +00:00
Zuul
687a0e8472 Merge "Separate the thread pool for health and stats update" 2018-10-01 19:41:57 +00:00
Michael Johnson
53772f5320 Set some amphora driver optimizations
This patch adds a few optimizations when using the amphora driver.

1. It increases the amp_active_retries from 10 to 30. This increases
the time we wait for nova to mark an instance "ACTIVE". The old default
of 10 was one minute forty seconds, but in some clouds it's been observed
that the nova schedule can get overloaded and take longer than a minute
forty to schedule the instance. Setting this to 30 means we will wait
five minutes for nova to schedule the instance.
2. It enables TCP kernel splicing in HAProxy. This has been shown to
reduce the CPU overhead for very high rate TCP load balancers.
3. Finally it enables "safe" HTTP keepalives on the backend member
connections [1]. This increases the request rate possible while using HTTP
protocol listeners and members.

[1] http://cbonte.github.io/haproxy-dconv/1.6/configuration.html#4-http-reuse

Change-Id: I3af009cac9a9edc8aef793b52c6a1488fde2c59b
2018-08-31 10:34:39 -07:00
Tatsuma Matsuki
ad69363fc7 Separate the thread pool for health and stats update
When queue_event_streamer driver is used and RabbitMQ
is down, stats update processes occupy the thread pool
which is shared with health update processes. Then,
RabbitMQ down unexpectedly leads to delete all existing
amphorae. This commit separates the thread pool and aims
to keep the existing amphorae working even when RabbitMQ
is down.

Change-Id: I576687f5b646496ff3a00787cf5e8c27f36b9448
Task: 22929
Story: 2002937
2018-08-22 09:07:01 +09:00
Zuul
4d867f623d Merge "Remove user_group option" 2018-08-21 15:51:12 +00:00
Michael Johnson
1ad9fd7272 Allow blocking IPs from member addresses
This patch adds a configuration option for reserved IP addresses that
cannot be used for load balancer member addresses. By default, this will
include the nova metadata service address 169.254.169.254.

Change-Id: I25de5ed5f6f35afc55dd1154c3e02934fddb100a
Story: 2003413
Task: 24555
2018-08-16 22:25:35 +00:00
Nir Magnezi
100858fa79 Remove user_group option
In Pike[1], we introduced a user_group auto detection for haproxy.
The default user group name is auto-detected for any OS distribution
we support as a base for Amphorae.

user_group remained as an option for admins but was also
marked deprecated in Pike[2].

This patch removes that option altogether.

Story: 2003323
Task: 24357

[1] Ia8fede9d7da4709a48661d1fc595a16d04fcbfa9
[2] https://review.openstack.org/#/c/429398/45/octavia/common/config.py@175

Change-Id: Iddd4162674f116705d2b47062cbf7ca88f2677a6
2018-08-08 23:45:29 +03:00
ZhaoBo
4c6846a568 UDP for [3][5][6]
Add new types into db table.
Extending the existing API, including Listener, Pool, HealthMonitor for
UDP fields support.

For healthmonitor part, need to wait for other patch to fix the default
value.

Patch List:

[1] Finish keepalived LVS jinja template for UDP support
[2] Extend the ability of amp agent for upload/refresh the keepalived
process
[3] Extend the db model and db table with necessary fields for met the new
udp backend
[4] Add logic/workflow elements process in UDP cases
[5] Extend the existing API to access udp parameters in Listener API
[6] Extend the existing pool API to access the new option in
session_persistence fields

Story: 1657091
Task: 5484
Change-Id: If728705f142f4195fe624bd9ef17413722d54fe3
2018-08-07 11:14:48 +00:00
ZhaoBo
a890f2ba35 UDP for [2]
These files will split with the current Octavia repo, before other parts
are ok.

Patch List:

[1] Finish keepalived LVS jinja template for UDP support
[2] Extend the ability of amp agent for upload/refresh the keepalived
process
[3] Extend the db model and db table with necessary fields for met the new
udp backend
[4] Add logic/workflow elements process in UDP cases
[5] Extend the existing API to access udp parameters in Listener API
[6] Extend the existing pool API to access the new option in
session_persistence fields

Change-Id: Ib4924e602d450b1feadb29e830d715ae77f5bbfe
2018-08-07 11:13:58 +00:00
Zuul
fbefbcd843 Merge "Fix failover when multiple amphora have failed" 2018-07-27 23:22:30 +00:00
Erik Olof Gunnar Andersson
cfae79cfee Fixing Octavia generating json named file containing yaml
Currently when the command to generate a policy file is used, it will
generate a yaml based file, but call it json.sample.policy. This patch
forces it to generate a json file, and changes the default name to
policy.json.sample.

Change-Id: I4d04f4f37385bd4601f566238aaba43dcbe74780
2018-07-24 13:06:55 +00:00
Michael Johnson
0139f12c2e Fix failover when multiple amphora have failed
If a load balancer loses more than one amphora at the same time
the failover process will fail and leave the load balancer in
provisioning status ERROR.

This patch resolves this by failing over one amphora at a time
marking any amphora that are also failed in status ERROR. The health
manager will then failover the other failed amphora in subsequent checks.

This patch will update multiple healthy amphora in parallel and will
timeout failed amphroa using the new "active_connection_max_retries"
configuration setting used for "fail-fast" connections.

The patch also updates the amphora failover flow documentation to
show the full flow and not just the spares failover flow.

It updates the amphora driver "get_diagnostics" method to pass instead
of error.

It also adds a AmphoraComputeConnectivityWait task to explicitly wait
for a compute instance to come up and be reachable. This allows a longer
timeout and clarifies this may fail due to compute (nova) failures.
Previously the first plug vip task would do this wait.

Change-Id: Ief97ddda8261b5bbc54c6824f90ae9c7a2d81701
Story: 2001481
Task: 6202
2018-07-22 16:08:45 -07:00
German Eichberger
1e4155f8ed Add a config to surpress amphora logging
Change-Id: I502e701acbaf0e92a64791a632eeb5ae61123bdd
2018-07-11 12:07:49 -07:00
Michael Johnson
6349976cde Implement provider drivers - Cleanup
This patch addresses the following:
Fixes some unit tests.
Cleans up some code from the parent patches,
Adds a release note for the provider driver support.
Adds the "List providers" API.
Adds a document listing the know provider drivers.
Adds a provider driver development guide.

Change-Id: I90dc39e5e9d7d5839913dc2dbf187d935ee2b8b5
Story: 1655768
Task: 5165
2018-06-04 19:34:42 -07:00
Michael Johnson
7b2621fe29 Implement provider drivers - Load Balancer
This patch adds provider driver support to the Octavia v2 API, starting
with the load balancer API.

This patch also creates a provider driver for Octavia, initially fully
implementing the load balancer methods.

Follow on patches will implement the remain parts of the API.

Change-Id: Ia15280827799d1800c23ed76d2af0e3596b9d2f7
Story: 1655768
Task: 5165
2018-05-22 17:26:08 -07:00
Adam Harwell
d28d311db1 Add config for disabling creation of PING type HMs
PING is a trap. There is no real-world scenario where PING is the option
that makes the most sense, but people are familiar with it, and it seems
"simple", so they pick it. This needs to stop. Empower operators to
disable this!

Change-Id: Ifa80b7a5973361c13f2e6611789aa9798325ece0
2018-05-22 14:02:00 +00:00
Vu Cong Tuan
bdd9c6f3e7 Replace port 35357 with 5000 for "auth_url"
Based on the change in Keystone Install Guide [1],
this patch replace port 35357 with 5000 for "auth_url".

For more details, please check similar changes which have been done
on other projects: Nova [2], Neutron [3], Cinder [4], Glance [5].

[1] https://review.openstack.org/#/c/541857
[2] https://review.openstack.org/#/c/562812
[3] https://review.openstack.org/#/c/566491
[4] https://review.openstack.org/#/c/565464
[5] https://review.openstack.org/#/c/558932

Change-Id: Iee308471a86c2d4ca5717bcb4dddfe61f1230380
2018-05-21 13:09:12 +07:00
caoyuan
cf296c2ade Update auth_uri option to www_authenticate_uri
Option auth_uri from group keystone_authtoken is deprecated[1].
Use option www_authenticate_uri from group keystone_authtoken.

[1]https://review.openstack.org/#/c/508522/

Change-Id: If6eee4ecfb4c6c607c9ee762cc535cf5d6180d88
2018-04-11 23:34:05 +08:00
Adam Harwell
f9dafb9a7a Overhaul HealthManager update threading
* Switch to ProcessPool from ThreadPool (Python threads are horrible)
* Load the health/stats update drivers correctly with stevedore
* Add logging driver

Change-Id: Icda4ff218a6bbaea252c2d75c073754f4161a597
2018-02-21 06:40:59 +00:00
Adam Harwell
8934a629df Switch to using PKCS12 for TLS Term certs
*NOT* deprecating the old way of storing these, as I believe that would
create a huge mess for anyone already using it.

Change-Id: I1fee174d8b8956f3d2053781a7f18c2940b21765
2018-01-23 07:17:14 +00:00
Michael Johnson
7b1621789c ACTIVE-ACTIVE: Initial distributor driver
This patch is the initial implementation of a distributor driver for
Octavia Active/Active topology support.

This patch is a decompostion of the following patch:
https://review.openstack.org/#/c/313006

Story: 2001288
Task: 5836

Depends-On: I97b52b80efb33749647229a55147a08afa112dd2
Change-Id: I65e4a533caee692e1c98e8c6586c2e2132f2e34c
Co-Authored-By: Valeria Perelman <perelman@il.ibm.com>
2017-11-22 05:53:48 +00:00
German Eichberger
060fcc1503 Make the event streamer transport URL configurable
This adds a way to configure the event streamer transport URL
so it can post to a different queue, e.g. Neutron's

Change-Id: I69d3d6d30e33878052f2c56b8c79a14cc4ec1b24
2017-10-25 10:52:48 -07:00
Adam Harwell
7bf8804177 Add flag to disable SSHD on the amphora image
Also deprecate the amp_ssh_access_allowed option.

Change-Id: Icb61a65fac57e74235fac904639c411b0fa2b495
2017-08-16 11:18:10 -07:00
Santhosh Fernandes
94a8d5715a Option to enable provisioning status sync with neutron db
In large build situations, nova can be slow to build VMs, this means that the
default 100 second timeout may expire before the final status has been updated
in the neutron database. This patch will emit provisioning status to be sync
with neutron db

Change-Id: If6c0b81630fd1911518792d9947f8622f065ff4e
2017-07-18 12:02:18 +05:30
Michael Johnson
767ef161fb Make developer debugging easier
This patch makes developer debugging of Octavia easier.  It adds
a configuration option that disables the controller worker taskflow
flows from reverting and cleaning up resources.
It also changes the amphora agent to keep a copy of a haproxy
configuration that failed validation.

Change-Id: Iaca070a0ab9589fb25513eb5fad7d1e99974d572
2017-06-28 13:51:03 -07:00
Adam Harwell
38a5563abc Allow operators to disallow creation of TLS Termination listeners
Change-Id: I93fbc26c775d1a7f6c69a0ab0b5f47a573cb125d
2017-06-26 18:47:15 -07:00
Adam Harwell
c764abc355 Allow operators to disable v1 or v2.0 api endpoints
Also, create a section for API settings `api_settings` and move some
related settings there.

This patch also enables the configuration settings to be logged
when the api process is started if debug is True.

Change-Id: I31671789d186c4b8a775cc12a414acd2d439512d
2017-06-26 14:37:27 -07:00
Michael Johnson
335c00ac18 Add RBAC enforcement to quotas v2 API
This patch adds policies and enforcement to the Octavia v2 API for quotas.

Change-Id: I5f2fa38973fce595ea3ec03cdff924336e0e71c8
Partial-Bug: #1690481
2017-06-20 18:52:05 -07:00
Michael Johnson
0ce46fe8d0 Add RBAC enforcement to Octavia v2 API
This patch adds policies and enforcement to the Octavia v2 API for
load balancers and listeners.  Child patches will add the rest of the API.

In this patch I also correct some improper functional tests.

Change-Id: Id8a2d15c117c54bd45fc8bb76bf71aff1b3c8fe9
Closes-Bug: #1690481
2017-06-20 13:43:47 +03:00
Jenkins
e5ac4a0426 Merge "Remove deprecated signing_dir config setting" 2017-06-20 02:01:36 +00:00
Adam Harwell
041d15a4b2 Allow operators to tune VIP creation parameters
Change-Id: Iff46479d530e5e3b09f27fd9d335651521f77a11
2017-06-16 14:44:58 -07:00
Michael Johnson
be27e4f0e8 Remove deprecated signing_dir config setting
keystonemiddleware has deprecated the "signing_dir" configuration
option [1].
This patch also removes reference to it from octavia.

[1] https://review.openstack.org/#/c/391405

Change-Id: Idda46ab1459584eafd58097ec42b9f0fcea41759
2017-06-12 17:04:53 -07:00
Jenkins
3c1b9ae360 Merge "Remove lb_network_name from config (it was bogus)" 2017-06-01 18:14:19 +00:00
Adam Harwell
da81984492 Remove lb_network_name from config (it was bogus)
This option was NEVER read, so there is no point in continuing to allow
it to be configured (it is pointless).

Change-Id: I147abdd8d3d95164168ec606f5b92401cb24d1fe
Closes-Bug: #1691286
2017-05-24 12:41:03 +00:00
Adam Harwell
104149d9d7 Update example config to have more correct keystone_authtoken example
Change-Id: Id00ed2b901ab2e1ab75f7ad3451147786349b6ac
2017-05-17 17:08:13 -07:00
Carlos D. Garza
9bfa58af9f Implement sorting and pagination for octavia
Use glance sorting and pagination from inside the SQLAlchemy query
to handle the sorting and pagination for octavia.

Change-Id: I5489c5c89691b8871e32caf3f85ab1978bc3618c
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Co-Authored-By: Lubosz "diltram" Kosnik <lubosz.kosnik@intel.com>
Closes-Bug: #1596628
Closes-Bug: #1596625
2017-05-05 21:08:46 -07:00
Nir Magnezi
75c1c5f22a Change auth_strategy default to keystone
The current default for auth_strategy is noauth, which is not how it is
expected to be set in production environments.

Note: Functional tests should be good with noauth.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Change-Id: Ifc80fff06a1d793d7cee0b207af10061784e48db
2017-05-01 11:33:37 +09:00
Jenkins
942afaacec Merge "Auto-detect haproxy user_group" 2017-04-23 23:24:29 +00:00
Nir Magnezi
26a55415ab Auto-detect haproxy user_group
As a followup to Id99948aec64656a0532afc68e146f0610bff1378, adding auto
detection to haproxy_amphora.user_group

haproxy is capable[1] handling a list of configuration files.
This patch leverages that capability by simply providing haproxy with an
additional configuration file, which is baked in the amphora image via a
diskimage-builder element.

The above-mentioned element will specify the following values for user group:
Ubuntu: 'nogroup'
RHEL/CentOS/Fedora: 'haproxy'

The amphora-agent will parse and remove any user_group configuration provided
by Octavia controller worker.
This is in order to maintain amphora-agent backward compatibility to old
Octavia workers, who still provide user_group to the amphora-agent.
Octavia Workers that include this patch will no longer provide user_group
configuration to the amphora-agent.

[1] https://cbonte.github.io/haproxy-dconv/1.7/management.html#3

Related-Bug #1548070

Change-Id: Ia8fede9d7da4709a48661d1fc595a16d04fcbfa9
2017-04-23 18:24:23 +03:00
Adam Harwell
9027154a5a Removing dependency on eventlet and oslo.service
Change-Id: I453e9b86d4edfedd63cc59e47bf745e166ff836f
2017-04-21 07:07:12 +09:00
German Eichberger
e58721c4e0 Adds a new config for soft-anti-affinity
Introduces a new config parameter to specify the anti-affinity
policy.

Bumps nova version.

Closes-Bug: 1677604

Change-Id: I8c50057bd43873182058097e802bc839d1be0554
2017-03-31 14:30:53 -04:00
Aishwarya Thangappa
e94ff2681f Adds a new feature to limit the amphora build rate
This patch limits the number of Amphora build requests handled by the
controller worker at a given time.

Also, the amphora build requests are assigned priorities based on
whether it is a normal loadbalancer create, failover or spares pool
loadbalancer create request. Based on the priority and the order in
which the requests were made if there is an available build slot the
amphora will be built.

Co-Authored-By: Lubosz "diltram" Kosnik <lubosz.kosnik@intel.com>
Change-Id: I967cf0668f82fb3a63e18dc7a457c58b526b7e66
Closes-Bug: #1571802
2017-03-16 16:50:31 +00:00
Jenkins
46252d0f5c Merge "Remove config option "amp_network"" 2017-02-15 01:42:31 +00:00
Jenkins
7c9baeb9d1 Merge "Add option to choose an availability_zone for amps" 2017-02-14 00:23:19 +00:00
Jenkins
bcd6f592d3 Merge "Allow a configuration option for random Amphora name" 2017-02-13 22:39:53 +00:00
Adam Harwell
e713f05b2a Add option to choose an availability_zone for amps
Change-Id: Id91c15da6caa656925184dda43d063aede89989e
2017-02-13 14:01:25 -08:00
johnsom
43949908e1 Remove mention of deprecated verbose option
oslo.log has deprecated the verbose option [1] so we should remove
mention of it as well.

[1] https://review.openstack.org/#/c/206437

Change-Id: I322e7b635f7337a6d399d7728bf55e581160b823
2017-02-13 08:50:55 -08:00
ZhaoBo
dc7c2967aa Remove config option "amp_network"
This option had been deprecated in N release. We could remove it from
master.

Change-Id: Ibd498f32a3c4f34621f3d814f1186a5bf9c5b75e
2017-02-13 09:59:41 +08:00
Adam Harwell
f345b4273b Allow a configuration option for random Amphora name
Also do a minor config file cleanup (looks like a bad previous merge).

Change-Id: I66e71795a4910c91cc2af4107fc60cc5aae72c79
Closes-Bug: #1663037
2017-02-08 16:26:29 -08:00