3041 Commits

Author SHA1 Message Date
Michael Johnson
e795406343 Set neutron client logging to INFO
The neutron client will post debug messages with the word "Error"
which is misleading. In reality it is a simple 404 that an extension
is not enabled.
This patch raises the default logging level for the neutron client to
INFO to suppress these messages.

Change-Id: Iacee63120a0d60e312cc85c7fcb8e7351688af3d
2019-09-13 16:09:29 -07:00
Michael Johnson
8685bba5ce Fix the tips job for octavia-lib
Change-Id: Ic6c97e78f579612b4b9207a24624e7a2e1c06723
2019-09-13 13:03:08 -07:00
Maciej Józefczyk
2eac7a7862 Add new algorithm SOURCE_IP_PORT
LB_ALGORITHM_SOURCE_IP_PORT is an algorithm used by OVN
Load Balancer [0]. This patch adds its support to the API.

[0] https://review.opendev.org/#/c/660369
Depends-On: I605f44f0f50219aa003df477de9bae4062f3c308

Change-Id: I436a6e553065d1755d465d20ad36f7ba2cbb8eba
Task: 35952
Story: 2006264
2019-09-13 15:19:15 +00:00
Carlos Goncalves
f3b48bc2f7 Add VIP access control list
This patch extends the listener API to include the new parameter
'allowed_cidrs'. This parameter is a list of IPv4 or IPv6 CIDRs. Leaving
this list unset defaults to the traditional behavior of allowing all
ingress traffic to the listener. Setting it will deny all traffic but
all CIDRs set in the 'allowed_cidrs' list.

Note that the API will validate that all CIDRs match the same IP version
of the VIP. This may change later as part of work to allow multiple VIPs
per LB (Change-Id Id7153dbf33b9616d7af685fcf13ad9a79793c06b).

Task: 26210
Story: 2003686

Change-Id: Id2b560df1cde9ce9403afbd593bbaa6cae5f06d6
2019-09-13 10:09:25 +02:00
Michael Johnson
05f5257075 Fix a few nits from the main volume-based patch
I had a few minor nits on the volume-based patch. This patch
corrects those.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>

Change-Id: I5f9ce36c878973f4ed96527af6f1024a362421d8
2019-09-12 10:32:45 +00:00
Zuul
510525a92b Merge "Add additive_only parameter to Batch Member call" 2019-09-11 20:43:47 +00:00
Zuul
ff483f3457 Merge "Add long-running provider agent support" 2019-09-11 16:03:56 +00:00
Zuul
b7278ceab4 Merge "Support create amphora instance from volume based." 2019-09-11 11:08:26 +00:00
Zuul
6b4dad2b87 Merge "Move to using octavia-lib constants" 2019-09-11 09:40:46 +00:00
Michael Johnson
78b1263237 Add long-running provider agent support
This patch adds support for long-running provider driver agents to
the Octavia driver-agent.
It will fork a process for all of the enabled provider driver
agents at startup.

Change-Id: Ib7042bcc48b1dd5b37b671dd5e64728b71ab9542
Story: 2006250
Task: 35863
2019-09-10 22:25:50 +00:00
Gregory Thiemonge
a4da3ef220 Fix cleanup of expired load balancer entries
Fix a bad query filter, deleted load balancers have a DELETED
provisioning_status (instead of operating_status).

Add a functional test that checks that deleted and expired load
balancers are correctly selected by get_all_deleted_expiring function.

Fix a similar test for amphorae that was buggy and not enabled (because
of missing test_ prefix).

Change-Id: I0ce2eabfd4dd41210312ea3b7f6274c9a6d50e44
Story: 2006496
Task: 36458
2019-09-10 22:24:34 +00:00
Ann Kamyshnikova
42df031e89 Fix building configs for multiple listeners
Currently jinja_combo.build_config method expect to use single
tls cert, though with multiple listeners there could be multiple
certs. Also in case of HTTP and TERMINATED_HTTPS listeners on the
same loadbalancer - creation of the second listener will fail.

Change-Id: Iad3b55e5add4283256f7836c3d4a501aa57ffc2f
Story: 2006513
Task: 36510
2019-09-10 22:22:56 +00:00
Adam Harwell
4b907b0627 Add additive_only parameter to Batch Member call
If `additive_only` is set, don't do a complete delta -- skip delete and
only update and create members (making the call additive rather than a
full replacement). This will allow for adding members in batches without
wiping out existing members.

Change-Id: I5e47d64243667cfaa10430e12229099b508de40e
2019-09-10 22:22:13 +00:00
sapd
93b509cfe5 Support create amphora instance from volume based.
In some deploy production, using volume based instead of localdisk
to protect data and live migrate can perform.

This patch adds:
 - creation a cinder volume for amphora
 - boot amphora with cinder volume
 - config options for cinder client
 - unit tests for cinder functionality

Story: 2001594
Co-authored-by: Vadim Ponomarev <velizarx@gmail.com>
Co-authored-by: Margarita Shakhova <shakhova.margarita@gmail.com>
Change-Id: I8181ed696b9ab556e7741c08839d79167aff8350
2019-09-10 22:21:42 +00:00
Michael Johnson
1aeb416a7e Bump diskimage-builder minimum to 2.24.0
There was a bug[1] in diskimage-builder when using pypi mirrors
if the host doesn't have "python" available DIB will fail.
So, I am going to bump the requirements minimum in Octavia for
diskimage-builder to be the minimum version with this fix.

[1] https://bugs.launchpad.net/diskimage-builder/+bug/1577105

Change-Id: If0ff2a855ad5b9d9ef3742ad0596c97a6dbf81ed
2019-09-10 22:20:42 +00:00
Michael Johnson
8577b3af6f Move to using octavia-lib constants
Previously the samlple data models and driver agent tests used
strings for some of the tests. This patch corrects those to use
constants from octavia-lib.

Depends-On: https://review.opendev.org/#/c/673681/
Change-Id: I90c2fa386fab75742d07e89edd3fba8b56bb6aff
2019-09-10 21:02:36 +00:00
Michael Johnson
09efc2a423 Add get method support to the driver-agent
This patch adds support for the octavia-lib to get objects by ID.

Change-Id: I98b399891488e5972ea4d332c06b55b34f20fb11
Story: 2005870
Task: 33680
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
2019-09-09 22:47:10 +00:00
Zuul
fa70e9759d Merge "Fix template that generates vrrp check script" 2019-08-30 15:50:33 +00:00
Zuul
f785bb4d0c Merge "Fixed typos and bad markup style in maintenance guide" 2019-08-29 23:19:11 +00:00
Michael Johnson
f20dc15844 Fix base (VRRP) port abandoned on revert
If the base (VRRP) port fails to attach to the amphora instance, the
port would not be deleted as part of the revert cleanup.
This patch splits the two plug VIP phases and attempts to clean up the
base (VRRP) port, that was created in the first phase, should the port
attach fail.

Change-Id: Ieab13c1152fed64da7390891b315b5e67513ce3e
Story: 2006468
Task: 36399
2019-08-28 16:23:23 -07:00
Michael Johnson
9de37f3955 Update the load balancing cookbook
The "basic load balancing cookbook" still had a reference to
neutron-lbaas and an old DVR bug.
This patch corrects those issues.

Change-Id: I9299084402a0b09104e81f48e998b77a195745a0
Story: 2006470
Task: 36401
2019-08-28 12:32:57 -07:00
Zuul
60bd083ce6 Merge "Force DIB Python version for py2 in diskimage-create" 2019-08-27 23:49:22 +00:00
Michael Johnson
950faea6e9 Use dual intermediate CAs for devstack
This patch updates the devstack plugin to use a dual Certificate
Authority (CA) with intermediate CAs for the Octavia controller
deployment.
This is a more realistic deployment model for testing.
Note: This change uses weak security to save gate resources. Please
refer to the Octavia Certificate Configuration Guide for production
instructions.

Change-Id: I3ec135766c9a1ddb7ac6655c0ee1ccb1e78ead5c
2019-08-27 16:28:12 -07:00
Rene Luria
905499162b
Fix template that generates vrrp check script
Correct the inline comment to not include an empty new line at the start
of generated /var/lib/octavia/vrrp/check_script.sh that leads to this
kind of error:

>  Aug 26 11:49:32 amphora-12184e15-1ec3-4d80-98a7-c7d1ddb6716f
> Keepalived_vrrp[15265]: Error exec-ing command
> '/var/lib/octavia/vrrp/check_script.sh', error 8: Exec format error

Change-Id: Icddd2873abeb56a389a35356995df6dde70872b2
2019-08-26 13:50:42 +02:00
Zuul
9ec9859bbf Merge "worker: Re-add FailoverPreparationForAmphora" 2019-08-22 21:50:36 +00:00
Zuul
7bffc1391b Merge "Allow listeners for different protocols on the same port" 2019-08-22 19:44:55 +00:00
Zuul
0ee7a3cf22 Merge "Change amphora logging to dedicated socket" 2019-08-22 19:44:54 +00:00
Zuul
b2bbe81321 Merge "Fix provider driver utils" 2019-08-22 18:44:36 +00:00
Zuul
be86fcfc8d Merge "Fix L7 repository create methods" 2019-08-22 18:12:07 +00:00
Sven Wegener
c4c7387f89 worker: Re-add FailoverPreparationForAmphora
This tasks removes the dns_name option from ports, so that they can be
plugged into a new amphora, which has a different name. This was lost in
change I04d267bd3cdedca11f0350c5255086233cba14ec.

Story: 2006205
Task: 35791

Change-Id: I4230feae5b6f0778b738bddbc69c5a0cd3e6930c
Signed-off-by: Sven Wegener <sven.wegener@inovex.de>
2019-08-22 10:59:34 -07:00
Zuul
f6b91c180d Merge "Add Octavia tox "tips" jobs" 2019-08-22 17:40:57 +00:00
Zuul
da27d2a11c Merge "Use the infra pypi mirror for DIB" 2019-08-22 17:40:56 +00:00
Zuul
b43a61d00a Merge "Lookup interfaces by MAC directly" 2019-08-21 18:59:13 +00:00
Zuul
bc8fbd7a7b Merge "Remove amphora-agent build deps" 2019-08-21 14:18:30 +00:00
Zuul
055006596a Merge "Clarify that an example is not an actual list" 2019-08-20 23:36:57 +00:00
Zuul
19a327524b Merge "Add the DIB_REPO* variables to the README.rst" 2019-08-20 23:34:21 +00:00
Colin Gibbons
3b5a19c386 Standardizes terminology in Listener documentation
This addresses the potentially confusing use of two terms
(TERMINATED_TLS and TERMINATED_HTTPS) used to describe the same
behavior by standardizing on the term TERMINATED_HTTPS in the
parameters of the documentation.

Change-Id: I3f444ba8e68ba8fc692ba41eec1ad4672ba5a16b
Story: 2006405
Task: 36289
2019-08-15 14:03:38 -07:00
Zuul
b8cc5ae533 Merge "Set Libvirt hw_rng for devstack" 2019-08-15 11:07:24 +00:00
Carlos Goncalves
0978c776a2 Switch TaskFlow engine to parallel
The default TaskFlow engine is now set to 'parallel' instead of
'serial'. The parallel engine schedules tasks onto different threads to
allow for running non-dependent tasks simultaneously. This has the
benefit of accelerating the execution of some Octavia Amphora flows such
as provisioning of active-standby amphora loadbalancers.

Change-Id: I108b7f629d39c40b60ddf4a1878631f32e37b357
2019-08-14 11:27:49 +02:00
Michael Johnson
37b50c1fef Set Libvirt hw_rng for devstack
This patch makes sure that the amphora service VMs have an rng
device available to them that by default sources from
/dev/urandom on the compute host.

Depends-On: https://review.opendev.org/675745
Change-Id: Ifc9ce78755b33b6ad88b1a0c4f100c27f35bd77e
2019-08-13 08:59:19 -07:00
Michael Johnson
0720397d5c Work around strptime threading issue
There is an open bug[1] in python strptime when used in multi-threaded
applications. We have seen this occur in the Octavia test jobs[2].
This patch works around the bug by loading strptime early.

[1] https://bugs.python.org/issue7980
[2] https://logs.opendev.org/37/673337/12/check/ \
    octavia-v2-act-stdby-iptables-dsvm-py2-scenario/440c965/controller/logs \
    /screen-o-cw.txt.gz?level=ERROR#_Aug_09_23_54_29_426364

Change-Id: I932ad625595333e97b0ead074ce64a7341af338d
2019-08-09 19:44:28 -07:00
Michael Johnson
6af536ec29 Fix provider driver utils
The provider driver utils module converts Octavia data model objects
to provider driver objects.
This module had some bugs in the conversion that led to errors and
incomplete data.
Specifically the following conversions were fixed:
flavor_id in load balancer conversion
sni_refs and L7 policies in listener conversion
health monitors in pool conversion

This patch corrects those and updates the tests.

Story: 2006306
Task: 36022

Change-Id: Ia2570b929be1c9d89ca971fb37f036f104fb32f2
2019-08-09 23:08:50 +00:00
Michael Johnson
36afa82d0c Fix L7 repository create methods
SQLAlchemy can get confused if the parent database objects are not
linked when creating new records using the repository.
For example, when creating an L7 policy, even if you specify the
listener_id, sqlalchemy will drop the ID from the final SQL INSERT
parameter.
This patch corrects the L7 policy and rule create methods to have
the required parent objects linked.

Story: 2006305
Task: 36021

Change-Id: I4380605bdb346eee824b2ce05fa25175c4cc3034
2019-08-09 23:08:44 +00:00
Michael Johnson
2529fa33ab Lookup interfaces by MAC directly
Currently the amphora agent will lookup interfaces using the
interface name determined earlier in the plug method. This can
lead to a race condition with the udev interface renaming rule.
This patch changes the interface lookup to use the MAC address
directly and not rely on the interface name.

Story: 2006300
Task: 36013

Change-Id: I5bc21d5abdeb67a3a8ae88456735643463f15694
2019-08-09 23:08:33 +00:00
Gregory Thiemonge
b42a64a00e Allow listeners for different protocols on the same port
Added 'protocol' name in the unique constraint list for listeners,
updated conflicting/duplicate entries detection in API.
Added alembic migration script.

Story: 2005070
Task: 29643

Change-Id: If85b59bddb8d6dc9916c3fef5155e838f1af63b6
2019-08-09 10:11:36 +02:00
Michael Johnson
1a26ba4a05 Use the infra pypi mirror for DIB
This patch changes the devstack plugin to use the infra pypi
mirror if one is declared in the environment.

Depends-On: https://review.opendev.org/675468

Change-Id: Iae839b6e59c43bc4be1e12702b3e2197b038b4c2
2019-08-08 22:57:41 +00:00
Carlos Goncalves
a04e3e3741 Remove amphora-agent build deps
The amphora-agent element installs a few build packages when installing
the agent from source. This patch removes those large in size packages
after they are no longer needed. This will reduce the
image size significantly.

Co-Authored-By: Michael Johnson <johnsomor@gmail.com>

Change-Id: I5d12b7a987f65013daa5298f5062c1f30db23f41
2019-08-08 12:41:58 +00:00
Zuul
6e4da85064 Merge "Re-enable the py36 tox environment" 2019-08-08 03:03:03 +00:00
Zuul
a7a04c899f Merge "Prevent UDP LBs to use different IP protocol versions in amphora driver" 2019-08-07 19:53:00 +00:00
Zuul
c65329391a Merge "Fixed down server issue after reloading keepalived" 2019-08-07 17:59:23 +00:00