3365 Commits

Author SHA1 Message Date
Gregory Thiemonge
64a301d4ec Fix nf_conntrack_buckets sysctl in Amphora
Setting nf_conntrack_buckets in the amphora namespace fails because this
sysctl can only be set in the initial namespace (cf kernel doc at
https://www.kernel.org/doc/Documentation/networking/nf_conntrack-sysctl.txt)

This commit allows to set nf_conntrack_buckets in the initial namespace,
the value is then inherited by other namespaces.

Conntrack is not enabled in the main namespace, the new default value
doesn't affect this namespace behavior.

Story: 2008028
Task: 40682

Change-Id: Ie6ccc4bf0017587df8e8e29d8ee3bf5c19e6d615
2020-09-10 11:51:32 +00:00
Michael Johnson
4260d8a74b Update grenade job to run one smoke test
With the addition of many new test permutations, the grenade job
is taking more time to run.
Previously grenade ran one smoke test to test basic traffic.
This patch updated the grenade job to run the basic http traffic
test.
This patch also restores the grenade job to voting status.

Change-Id: Ic0446994ffac130c632624a37482393f00fe8714
2020-09-09 22:52:55 -07:00
Michael Johnson
df3f58903b Fix the lower-constraints job for focal
The move to focal for the tox jobs has raised issues with package
versions supporting python 3.8.
This patch bumps the version for the following packages:
PyMySQL==0.8.0
pyroute2==0.5.13
SQLAlchemy==1.2.19
cffi==1.14.0
cryptography==3.0
pyOpenSSL==19.1.0
oslo.db==8.30
oslo.messaging==12.4.0

This patch makes the grenade job non-voting as there is a chain
of gate fix patches required to fully fix all of the issues.

Change-Id: I95cc182a2dbcb2ea872e822ce9372bf5d7df63bd
2020-09-09 22:48:42 -07:00
Zuul
9a732565e9 Merge "Refactoring amphora stats driver interface" 2020-09-09 02:10:53 +00:00
Zuul
49912974b7 Merge "Deprecate the Spares Pool feature for removal in X" 2020-09-09 00:44:19 +00:00
asingh12
5092597f6b Refactoring amphora stats driver interface
Previously the stats driver was responsible for parsing the health
message, which should have been done prior to passing the stats to the
driver interface.

Removed the driver interface for the health updater because it is core
Octavia functionality.

Stats drivers is now a singleton and can load multiple drivers.

Both the amphora health manager AND provider statistics should use the
new driver interface.

Co-Authored-By: Stephanie Djajadi <stephanie.djajadi@gmail.com>
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: I3a013aebd1eb89cd4f983fbf4f8ae8d6639548cd
2020-09-08 15:32:04 -07:00
Zuul
954cf8afb3 Merge "Add DIB element octavia-lib" 2020-09-08 21:42:20 +00:00
Adam Harwell
29a2ec7187 Deprecate the Spares Pool feature for removal in X
Use of the spares pool was originally recommended to increase provisioning
speed, but since Nova's server groups do not support adding existing VMs,
Octavia cannot support use of the spares pool with the Active-Standby
topology. Since this is our recommended topology for production deployments,
and speed is less essential in development/testing environments (the
only place we could recommend the use of Single topology), the overhead of
maintaining spares pool support exceeds its theoretical usefulness.

Change-Id: I7375e9758c7ae80e2370189117e8e63c79446490
2020-09-08 14:29:58 -07:00
Zuul
483eb62711 Merge "Add an upgrade check for YAML oslo policy file" 2020-09-08 21:00:27 +00:00
Zuul
7bcd59efe3 Merge "Preupgrade check for amphorav2 provider" 2020-09-08 19:45:11 +00:00
Zuul
ed1ff8fb4c Merge "Add proxy v2 protocol support" 2020-09-07 15:00:09 +00:00
Zuul
1500eebd94 Merge "Add HTTP/2 over TLS support to amphora provider" 2020-09-07 15:00:07 +00:00
Michael Johnson
7fe78c5943 Add proxy v2 protocol support
This patch adds support for the proxy protocol v2 on pools.

Depends-On: https://review.opendev.org/747296
Change-Id: Ic112c5e71ee9b6433b307fdf27059f217ba4136e
Story: 2005611
Task: 30858
2020-09-04 18:15:16 +00:00
Zuul
46de66b240 Merge "Update amphora v2 for the failover refactor" 2020-09-04 14:15:37 +00:00
Zuul
eef3c91bf2 Merge "Add amphora delete API" 2020-09-04 13:28:38 +00:00
Michael Johnson
781647ab60 Add an upgrade check for YAML oslo policy file
Oslo policy is deprecating[1] the JSON file format for oslo policy
files.
This patch adds an upgrade check that includes a warning if a JSON
policy file is being used with Octavia.

[1] https://review.opendev.org/733650

Depends-On: https://review.opendev.org/733650
Change-Id: I293a30ddef77c09808a204c87a594a91bb6992da
2020-09-03 15:49:48 -07:00
Ann Taraday
78d84154f6 Preupgrade check for amphorav2 provider
Added checks if amphorav2 provider enabled - persistence backend
and jobboard backend configured properly.

Also small fix of parameter name in ZookeeperTaskFlowDriver.

Change-Id: I9342801bbbc7ec2c99e8698504a2b4c28bc00526
2020-09-03 15:44:47 -07:00
Michael Johnson
59dcdd9a86 Add amphora delete API
This patch adds an amphora delete API. It can be used to delete
extra "spare" amphora after the feature has been disabled.

A followup patch will be required for the amphorav2 path as the
amphorav2 failover patch, which is required for the amphora delete
flow, has not yet merged.

Story: 2008014
Task: 40666

Change-Id: I32b6561c78c153a4b7e73b1a4b83e045fbe97fb6
2020-09-03 13:34:07 -07:00
Carlos Goncalves
ea0bbd2775 Add amphora image tag capability to Octavia flavors
This patch adds 'amp_image_tag' to the supported capabilities of the
amphora provider driver. One use case where operators could find this
relevant is on multi CPU architecture clouds.

Task: 40156
Story: 2007854

Change-Id: Id4141e820a8b34e379a8f60b53fc183680c55c79
2020-09-03 13:10:07 -07:00
Carlos Goncalves
a422e5a203 Introduce an image driver interface
With this image driver interface, we align our codebase with other
existing driver interfaces like compute, network and volume.

This interface also allows the amphora provider driver to check for
existence of tagged images at API level (e.g. amphora image tag
capability in Octavia flavors).

Change-Id: Id808c082808fafe1a1e004957ff47eca57f97ee8
2020-09-03 20:06:35 +00:00
Carlos Goncalves
de2fe22f9d Add HTTP/2 over TLS support to amphora provider
HTTP/2 requires HAProxy 2.0 or newer in amphora images. The operating
system versions in test as of today (Ubuntu Bionic and CentOS 8) do not
provide HAProxy 2.x in their official repositories. For this reason, the
default listener ALPN protocol list remains unchanged, defaulting to
ALPN protocol IDs "http/1.1" and "http/1.0".

Change-Id: Id29d0d2cd5ab1ae3c7d26194a235af25c2271d62
2020-09-03 10:43:07 +00:00
Michael Johnson
d5fe092a3e Update amphora v2 for the failover refactor
This patch will update the amphora v2 code for the failover refactor[1].

[1] https://review.opendev.org/705317

Change-Id: I43803d0b750e8ca4722ababe296f2725148da405
2020-09-03 11:48:14 +04:00
Zuul
4a4a2344de Merge "Add a message for the amphora image size" 2020-09-02 10:06:13 +00:00
Zuul
97faa9c2ef Merge "Switch to live from noop drivers" 2020-09-02 09:16:17 +00:00
Zuul
0a1e834123 Merge "Add some details on enable_anti_affinity option" 2020-09-02 03:12:32 +00:00
Zuul
6c135364fc Merge "trivial fix" 2020-09-02 02:06:41 +00:00
Zuul
80a30d26d1 Merge "Fix API sort key for complex columns" 2020-09-01 15:49:20 +00:00
Zuul
71e53a5b67 Merge "Add missing log line for finishing amp operations" 2020-09-01 15:49:18 +00:00
Adam Harwell
6b182b50cd Add missing log line for finishing amp operations
Both amp config and amp cert rotate were missing Finished log lines.

Change-Id: I86add89ed7ea474dcb24962547c78460bc2e260c
2020-09-01 11:56:46 +00:00
Paul Peereboom
6df1ce8e28 Fix user permission for WSGI configuration
Task: 35692
Story: 2006172

Change-Id: I477d2d12c64c1e38e3917333e5d47f612cab93b0
2020-09-01 11:56:18 +00:00
Michael Johnson
b6f6c14292 Fix API sort key for complex columns
The Octavia API sort function was not working correctly for some
columns that have different names in the API vs the data model.
This patch also handles the case where a column may be a compound
object in the data model, such as vip_address->vip.ip_address.

Change-Id: I0c124e80fec2ac9ad813e78df6ad25da6b8e9668
Story: 2007991
Task: 40626
2020-09-01 11:55:12 +00:00
Carlos Goncalves
f3b5e6e147 Switch to live from noop drivers
The switch to live drivers ease greenfield deployments by not requiring
deployers to explicitly set live drivers. The switch also helps
deployers identify production-ready from experimental drivers.
Experimental features like volume and distributor remain defaulted to
their noop drivers.

This patch also fixes some tests that were not mocked properly.

Change-Id: I1e4e3c4f0e4142fa0d0d1ac895b10e2349d79c20
2020-08-28 20:51:45 +00:00
Carlos Goncalves
a5f0524fd0 Add ALPN support for TLS-terminated HTTPS LBs
ALPN is a TLS extension for application-layer protocol negotiation
within the TLS handshake [1].

This patch extends the Listener API to include a new 'alpn_protocols'
parameter. With this parameter, users can set an ALPN preference list
(descending order of preference).

Presently, the amphora provider driver is limited to http/1.0 and
http/1.1 ALPN protocol IDs. Support for "h2" (HTTP/2 over TLS) depends
on HAProxy 2.0 or newer.

[1] https://tools.ietf.org/html/rfc7301

Change-Id: If08a8169498cdfaa75440e8971ba0caff45ac4c4
2020-08-27 13:19:52 +02:00
Zuul
0b1d8dd5e7 Merge "Update devstack plugin" 2020-08-26 23:58:54 +00:00
Zuul
d7b568f73d Merge "Set Grub timeout to 0 for fast boot times" 2020-08-26 22:53:42 +00:00
Zuul
36085827eb Merge "Add octavia-amphora-image-build-live jobs" 2020-08-26 22:42:00 +00:00
Zuul
4cb81a65e5 Merge "Correct a typo in the document" 2020-08-26 22:39:14 +00:00
Zuul
746a495ea2 Merge "Set CentOS amphora image size back to default value" 2020-08-26 22:39:12 +00:00
Gregory Thiemonge
9fb58eb9f4 Fix invalid DOWN status when updating a UDP pool
When adding a new UDP member or a UDP-CONNECT health-monitor to a UDP
pool, there can be a race condition in the first heartbeat message
sent to the health-manager service.
This message might contain a DOWN status for a working member that
hasn't been checked yet.

This commit introduces a new member status between the amphora-agent and
the health-manager: it indicates that the UDP pool has been updated and
that the status of a member is a transitional state, preventing an
incorrect ERROR status.

Story: 2007792
Task: 40042

Change-Id: Id9e19375ebca6a720e6a85006f5e8948d3aed760
2020-08-26 09:56:57 +02:00
Zuul
2d97ebcd24 Merge "Allow amphorav2 to run without jobboard" 2020-08-25 22:47:04 +00:00
Zuul
a2d53b41b0 Merge "Fix operational status for disabled UDP listeners" 2020-08-25 22:39:01 +00:00
Zuul
a55a6fd2cd Merge "Fix listener tls_versions and tls_ciphers clearing" 2020-08-25 21:36:06 +00:00
Zuul
703a07df21 Merge "Ensure xtrace is enabled in devstack plugin" 2020-08-24 14:32:58 +00:00
Carlos Goncalves
dcda721320 Add DIB element octavia-lib
This new diskimage-builder element installs octavia-lib from Git for
source install type image builds rather than from released versions.

To mention some advantages:

1. allow custom octavia-lib versions (useful for development)
2. test unreleased octavia-lib changes in CI
3. install latest changes from master and stable branches
   (aligns with approach taken with amphora-agent)

Branch checkout to stable/* from master for octavia-lib DIB element will
be automated on branch create. See https://review.opendev.org/#/c/745877

Change-Id: I6d87b6bd25c536b2bed1994427cd933bdcc091d6
2020-08-24 11:24:19 +02:00
Zuul
13faf288e4 Merge "Fix memory consumption issues with default connection_limit" 2020-08-24 07:50:16 +00:00
Carlos Goncalves
17c856dbcb Set CentOS amphora image size back to default value
DIB reduced the size of CentOS minimal images to the point CentOS-based
amphora images fit back in the default 2 GB disk.

Change-Id: Icd3ff0162def1820940d49e36308fc976bfde364
2020-08-21 15:48:46 +00:00
Carlos Goncalves
bf3d5372b9 Fix new pylint errors
Pylint 2.6.0 added two new checkers [1] that make pep8 validation to
fail:

- raise-missing-from (W0707)
- super-with-arguments (R1725)

[1] http://pylint.pycqa.org/en/latest/technical_reference/features.html

Change-Id: I15fd7082646d95414bdd8f0815ee44b5867b3ed0
2020-08-21 15:00:27 +02:00
Zuul
0a85de4c6d Merge "Switch back to using git for the agent ref" 2020-08-20 09:35:12 +00:00
suhaiming
d1da0c1cd7 Correct a typo in the document
Change-Id: I9cbf9c4cdf8d8b8bf8d896b2d59d45d0ee1d4fc5
2020-08-20 07:35:32 +00:00
Zuul
2640607309 Merge "Increase the devstack secuirty group rules quota" 2020-08-19 23:17:28 +00:00