3365 Commits

Author SHA1 Message Date
Zuul
6a535c979b Merge "Support hacking 2.0.0" 2020-03-13 07:47:07 +00:00
Brian Haley
27350a7d65 Use sqlalchemy isnot(None)
Instead of using '!= None' and pylint directives use
isnot(None) which should be equivalent.

Change-Id: I63f5fcb3421001e9e7f91b54707f53d2f20281cb
2020-03-13 03:57:25 +00:00
Brian Haley
c6db3e4f17 Support hacking 2.0.0
Eventually hacking will move to 2.0.0 (diskimage-builder
is holding it back), and when it does there will be a few
errors to fix.  We can get ahead of it so it doesn't
break us with some small changes for these items:

  F601 dictionary key $item repeated with different values
  F632 use ==/!= to compare str, bytes, and int literals
  E501 line too long

While doing this noticed the lower-constraints.txt for
hacking was set at 0.12.0, when test-requirements.txt
had it at 1.1.0, so fixed that as well.

Change-Id: I80d2a5f97e7a4896a8fa765c1971c8bb7e72d211
2020-03-12 20:53:01 -07:00
Ann Kamyshnikova
0f7d03feca Pick stale amphora randomly
Avoid getting the same amphora all the time if its loadbalancer
stuck in PENDING state and could not be failovered.

Story: 2007340
Task: 38877

Change-Id: I2f680a00be695c3828166e6803882765af79b79e
2020-03-06 15:23:12 +04:00
Michael Johnson
801b99ed00 Update the available provider drivers list.
This patch updates the list of known Octavia provider drivers.

Change-Id: Ic5e1717603a815de8655188e118b1ce95c99d9c1
2020-03-03 13:20:21 -08:00
Carlos Goncalves
2cfcd71a86 Fix load balancer update with provider filtered params
When only setting tags to an existing load balancer, the amphora driver
would try to apply QoS policy on VRRP ports even if no policy is
defined. This raised an NetworkException that led load balancers to go
into ERROR.

Task: 37589
Story: 2006922

Change-Id: I48315f8f293811e1d99584ea36da05c4211cf275
2020-02-26 09:09:31 -08:00
Adam Harwell
4a360bfda7 Allow AZ to override valid_vip_networks config
Different AZs may have access to different vip networks.

Change-Id: I7169b34d93bc8a265fc74fedcbba67e980285a7e
2020-02-25 01:30:56 +00:00
Adam Harwell
741397f1a9 Network Delta calculations should respect AZs
The network delta calculations were all based on the static configured
amp_boot_network_list which is not correct if it's overridden by the AZ.

Change-Id: Ia930e17c76cd601ac005de10fb03231a19f1a776
2020-02-25 01:30:12 +00:00
Adam Harwell
2c76209003 Select the right lb_network_ip interface using AZ
If the AZ system is being used and there is also a default boot network
set in the main config, the nova driver would not pick the right
interface when translating the amphora info for amps in custom AZs.

This affects the step that fetches the lb_network_ip (the management
address) from nova, immediately before the step that updates the
database with the correct IP. That IP is then used in the rest of the
amphora creation flow, which pulls in that address and uses it to
connect to the amp.

Change-Id: Ie4869035a557ebcddea2fce693067c82fbd2d2a9
2020-02-25 01:30:06 +00:00
Zuul
e0ae8a3cac Merge "Fix uncaught DB exception when trying to get a spare amphora" 2020-02-22 14:19:33 +00:00
Zuul
65a2a242be Merge "Add diskimage-create arg for Octavia Git branch" 2020-02-22 02:25:00 +00:00
Gregory Thiemonge
3bbd32a2a5 Fix uncaught DB exception when trying to get a spare amphora
The MapLoadbalancerToAmphora task performs some database queries but
doesn't catch any exception from it.

In case something goes wrong with the database or the query, this commit
catchs any exception from the DB call, logs it and then returns None to
switch to the next decider branch (boot a new amphora).

Story 2007320
Task 38831

Change-Id: Ifb6c34426e0927534d332a8bbf2c66aac6c002c5
2020-02-21 10:48:47 +01:00
Brian Haley
20fdbbecef Fix TESTING.rst example
There is no api.v1 functional test directory, just api.v2.

Trivialfix

Change-Id: I1b1a77fc671194524250db6c2ec9e89505ef602b
2020-02-18 18:11:05 -05:00
Michael Johnson
49036845c1 Remove the dependency on the "mock" package
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.

This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.

Change-Id: I72520a2ca010c2c27315d9dff839a4f9d7540b6b
2020-02-17 14:55:19 -08:00
Michael Johnson
bb0efa2a5d Add docs warning for PING health monitors
This patch adds a warning box to the load balancer cookbook clarifying
that the health monitor type of 'PING' should only be used in specific
cases.
This was called out in release notes, but was not clear in the cookbook.

Change-Id: I6b95891bec82e01c44b288cbe9796b1f87a07c32
2020-02-12 08:36:30 -08:00
Gregory Thiemonge
e1751220bc Add UDP LBs to the basic cookbook
Add a section to the basic cookbook that explains how to setup a UDP
load balancer with UDP-CONNECT health monitor.

And fix typos

Change-Id: Ib67a5c9437e3190f640a953c30f791cb34690910
2020-02-12 11:10:01 +01:00
Michael Johnson
1f9915037a Add a periodic image build job
This patch adds a periodic job to build the amphora image using the
released version of diskimage-builder (our other jobs use Git master).
This job will not publish the produced image, it is only to test the
build.

Change-Id: I611d9cd5a519abc6a330ec2c46a2f5a1fdb89313
2020-02-06 13:52:24 -08:00
Zuul
af9edc30d7 Merge "Complete dropping py27 support goal" 2020-02-06 19:54:18 +00:00
Adam Harwell
d973322529 Add diskimage-create arg for Octavia Git branch
Use `-g` to select the current Git branch to use for the amphora
build. This has historically been confusing for people, and this should
make it a little easier.

Change-Id: Iab01faaa17507c7bea399db7ee8f27f88d8de183
2020-02-05 13:14:48 -08:00
Adam Harwell
005cd1e6a6 Update the lb_id on an amp earlier if we know it
When we create amphora for specific loadbalancer it would be
good to get info about mapping pair loadbalancer-amphora as
soon as possible.

For example, if admin needs to debug connectivity issues with amphora
VMs - loadbalancer_id won't be set until AmphoraComputeConnectivityWait
task succeeds (which is never for such case) so they have to go to worker
logs to understand which amphora is related to a currently creating
loadbalancer.

Co-Authored-By: Ann Taraday <akamyshnikova@mirantis.com>
Change-Id: I865445af34bc63b90d965ef5e2c8f9f49aa9c2f3
2020-02-03 10:38:45 -08:00
Zuul
084b832a41 Merge "Fix jobs not running and add new ones to the gate" 2020-02-02 09:01:44 +00:00
Zuul
3b3d502fb2 Merge "Stop supporting CentOS 7 controllers and images" 2020-02-02 08:56:40 +00:00
Zuul
9f89da5f22 Merge "Support haproxy development snapshot version parsing" 2020-02-01 11:38:17 +00:00
Zuul
4bd9c65e4a Merge "Fix house keeping graceful shutdown" 2020-01-30 22:39:22 +00:00
Zuul
73096a5da1 Merge "Fix unit test when run on CentOS 7" 2020-01-30 22:35:59 +00:00
Gregory Thiemonge
7dc54eb9c9 Support haproxy development snapshot version parsing
Fix haproxy version parser in amphora-agent for beta versions (i.e
2.2-dev0)

Change-Id: I5baf7d18105494259361be1f0f411412071f1d36
2020-01-27 15:07:05 +01:00
Zuul
fb37005661 Merge "Transition l7rule flows to dicts" 2020-01-26 18:19:27 +00:00
Zuul
789f5310ca Merge "Transition l7policy flows to dicts" 2020-01-26 18:19:22 +00:00
Zuul
bb775f876a Merge "Convert health_monitor flows to use provider models" 2020-01-26 18:19:18 +00:00
Zuul
741a47763d Merge "Fix the interface filenames for Red Hat amphora images" 2020-01-24 17:08:13 +00:00
Carlos Goncalves
106cfb7e1e Add listener allowed_cidrs to the feature matrix
Change-Id: I8c0fadbc47823d030dc232984988e47f86368410
2020-01-24 15:02:23 +00:00
Ann Kamyshnikova
ea6b065cb0 Transition l7rule flows to dicts
Change-Id: Id94a293a88d27546c0fe7b45367922fa4cca260a
Story: 2005072
Task: 30813
2020-01-24 16:58:07 +04:00
Ann Kamyshnikova
7d310c3985 Transition l7policy flows to dicts
Story: 2005072
Task: 30812

Change-Id: I44919db35570fda7cc81a194c8c26e22c145855c
2020-01-24 16:58:07 +04:00
Ann Kamyshnikova
657068a278 Convert health_monitor flows to use provider models
Change-Id: Ifab50d30dbaf197dd2ba3b5e11b94732eb2e0aaf
Story: 2005072
Task: 30811
2020-01-24 16:58:07 +04:00
Zuul
5f1adf4e2b Merge "Add logging filter for AmpConnectionRetry exception" 2020-01-24 09:54:23 +00:00
Zuul
7f96e56ec2 Merge "Convert Lb flows to use provider dicts" 2020-01-24 09:54:18 +00:00
Zuul
8ea97754e6 Merge "Transition amphora flows to dicts" 2020-01-24 09:51:15 +00:00
Zuul
dee66e3025 Merge "Use LUKS1 for certfs-ramfs for all distributions" 2020-01-24 01:14:25 +00:00
Carlos Goncalves
08fb5a0639 Fix jobs not running and add new ones to the gate
Jobs like the barbican and active-standby were not being run in CI
because of the irrelevant-files list set in octavia-tempest-plugin. We
need to override that list in the octavia project.

This patch also adds spare pool and active-standby (replacing the
iptables-based one) jobs to the check and gate queues.

Change-Id: I26cbf5d39dc8857026dda58eb705e6b1fc807544
2020-01-22 17:43:00 +01:00
Zuul
b6deb12b5d Merge "Allow the Octavia wsgi to accept argv parameters" 2020-01-21 08:07:24 +00:00
Gregory Thiemonge
c2ef7792fd Use LUKS1 for certfs-ramfs for all distributions
This commit forces the use of LUKS1 in cryptsetup.

Centos uses LUKS2 by default in cryptsetup, ubuntu uses LUKS1.
Formating a LUKS2 block device is way longer than formating a LUKS1
device (15 sec vs 8 sec in my env) and LUKS2 doesn't provide any
significant features for octavia (it only improves recovery and
metadata).

The commit aslo limits the creation of more than 1 block ram device
(amphora doesn't need 16 ram devices), which reduces startup time.

Change-Id: I5cdc0a9ccc01548f195eed80f2ee2848a1a93e17
2020-01-17 10:38:45 +01:00
Ann Kamyshnikova
adbfaecc37 Convert Lb flows to use provider dicts
Split _get_delete_load_balancer_flow, create
separate methods for store generation.

Change-Id: I71117717a7fdff349473c43975ad639c0448713e
Story: 2005072
Task: 30807
2020-01-17 12:21:52 +04:00
Ann Kamyshnikova
dad38f61f8 Transition amphora flows to dicts
Fixed endpoints logs for listener, pool and member as well.

Rework _get_create_amp_for_lb_subflow due to issue with taskflow
decider and retry subflow.
Retry subflow was not ignored properly for spare amphorae case,
so _get_create_amp_for_lb_subflow has been split to
3 separate subflows each of them linked to graph flow.
This is work around and can be removed when proper mechanism
implemeneted in taskflow library. (added several todos about it).

Change-Id: Ibd114fa14123e6de6c5d6f260e32cf7f2b28805a
Story: 2005072
Task: 30814
2020-01-17 12:18:08 +04:00
Brian Haley
4af42f837f Complete dropping py27 support goal
This takes care of the last details for dropping py27
support by adding a proper min version of python in setup.cfg.

Change-Id: I693db277d802b2a54084cc1be11d8ce04ad9be2e
2020-01-16 11:32:03 -05:00
Zuul
1b6efa43f1 Merge "Remove test calls to reset_mock()" 2020-01-16 14:10:32 +00:00
Ann Taraday
038b2d5d6c Add logging filter for AmpConnectionRetry exception
We do not need to log exception info about AmpConnectionRetry as
this is expected exception, which enables retry process for
AmphoraComputeConnectivityWait task.

Change-Id: Ib40345fc441cf916c3542bfddbf8ac812159cd2e
Story: 2005072
2020-01-14 11:17:14 +00:00
Paul Peereboom
93cd9fc075 Fix the interface filenames for Red Hat amphora images
Code was not using the correct filenames for the 'route',
'route6', 'rule' and 'rule6' files on Red Hat images.
Changed to use config option 'agent_server_network_file'
if it's specified, else the file of the correct name, and
added unit tests for each.

Change-Id: I335287da66524d026f0c42086d885b478c568bbd
Task: 37881
Story: 2007051
2020-01-10 16:09:27 +01:00
Yang JianFeng
47e0ef31bc Add listener and pool protocol validation
The pool and listener can't be combined arbitrarily. We need to add
some constraints in protocol side.

Story: 2003500
Tasks: 24777

Co-Authored-By: Carlos Goncalves <cgoncalves@redhat.com>
Change-Id: Ifed862639d3fc3de23ace4c7ceaea1a4eca62749
2020-01-08 15:38:48 -08:00
Michael Johnson
7c6fbe7efe Allow the Octavia wsgi to accept argv parameters
Currently we only allow argv parameters to be passed into our wsgi
app. This means that some wsgi wrappers (such as pbr's script)
may not allow the octavia API to accept parameters such as
--config-file.
This patch attempts to pull in the sys.argv if none was passed into
the wsgi setup_app method.

Change-Id: I67cf32aac8aa8d8123ec8ca1ab433ff23fb1ffa2
2020-01-08 10:10:47 -08:00
Carlos Goncalves
dc2dd711e6 Stop supporting CentOS 7 controllers and images
The diskimage-create.sh tool will now default to CentOS 8 when building
CentOS-based amphora images.

This patch also removes leftover references to support for Ubuntu Trusty
and Xenial.

Change-Id: I3aba59c8dd86aeeee28cc6a67af93697912fb55b
2020-01-08 17:59:09 +01:00