3349 Commits

Author SHA1 Message Date
Michael Johnson
8394633635 Prioritize policy validation
This patch makes sure that we validate RBAC compliance before
other validation tasks.

Change-Id: I670087163b265e7098af35063572d6aa9d068bb9
2020-06-19 14:18:40 -07:00
Zuul
4a78b453d9 Merge "diskimage-create compatible with ShellCheck." 2020-06-19 21:12:46 +00:00
Thobias Salazar Trevisan
d8d722b54d diskimage-create compatible with ShellCheck.
Update diskimage-create script to pass in
ShellCheck (https://www.shellcheck.net/) tool.

Change-Id: Ia0dddac00155e11098fcbd3e95bcae12f2fc63d4
2020-06-19 10:22:07 -03:00
Michael Johnson
955bb88406 Refactor the failover flows
This patch refactors the failover flows to improve the performance
and reliability of failovers in Octavia.

Specific improvements are:
* More tasks and flows will retry when other OpenStack services are
  failing.
* Failover can now succeed even when all of the amphora are missing
  for a given load balancer.
* It will check and repair the load balancer VIP should the VIP
  port(s) become corrupted in neutron.
* It will cleanup extra resources that may be associated with a
  load balancer in the event of a cloud service failure.

This patch also removes some dead code.

Change-Id: I04cb2f1f10ec566298834f81df0cf8b100ca916c
Story: 2003084
Task: 23166
Story: 2004440
Task: 28108
2020-06-18 16:25:21 -07:00
Michael Johnson
e77355714b Update the API audit map
Recent additions to the Octavia API did not update the Octavia API
CADF audit map. This patch corrects that by adding the new API
paths.

Change-Id: I22107317837e68e54a29f8a4051c464120b29809
2020-06-18 21:43:34 +00:00
Michael Johnson
18887f5f60 Fix API audit CADF audit map for failover action
There was a bug in the CADF audit map file for the "failover" action.

This patch corrects the audit map file to handle "failover" correctly
and stop keystonemiddleware from raising an exception.

Change-Id: If3954ba34740e26937dba10bdd8061acde758c88
Story: 2007831
Task: 40116
2020-06-18 10:03:59 -07:00
Gregory Thiemonge
3ee9bc0d65 Fix listener API's test_create* assertions
Assertions were using the same expressions on both side: optionals and
lb_listener are both parameters to the API (and the lb_listener dict
contains all optionals items).
Those assertions should compare the parameters to the API results.

Change-Id: I6f372a3f82fdf4f41e661e640e4a983cf484ed6d
2020-06-18 14:17:33 +02:00
ramboman
f26ab8b97b add the verify for the session
We run the octavia scenario test failed when the OpenStack env
enable TLS. So we need add the verify for the session.

Story: 2007662
Task:  39754
Closes-Bug: #1877818
Change-Id: Ie71db27dc383c93496c1dfd69f486a4fd02b597e
2020-06-17 09:05:24 +00:00
Carlos Goncalves
e5951ced5f Use uwsgi binary from path
In-line with devstack patch [1], switch invocations to find uwsgi in the
path.

[1] https://review.opendev.org/#/c/577779/

Change-Id: I5e6aee49f434820881051874c9ad2628b4fcada7
2020-06-17 10:59:29 +02:00
Zuul
24acbe099b Merge "Improve terminology in an old spec file" 2020-06-16 03:37:01 +00:00
Gregory Thiemonge
c176cf60d4 Fix error on devstack cleanup
Change I7ebf4137feb04827490dffc0dac3d6e4c8888075 added 'set -e' in
devstack/plugin.sh, but on devstack cleanup, some commands may fail
because of non-working services (i.e after a reboot).
This commit allows 'openstack keypair delete' to fail on clean up.

Change-Id: Ic782faba3eb907d29b6735ac0a6d6a8a2e104e00
2020-06-15 08:44:32 +02:00
Hervé Beraud
8119b21452 Cap jsonschema 3.2.0 as the minimal version
Previous versions of jsonschema (<3.2.0) doesn't support python 3.8 [1].
Python 3.8 is part of the victoria supported runtimes [2] so we now force
to use jsonschema version 3.2.0 to avoid issues, remove ambiguity and ensure
that everything works with python 3 in general.

[1] https://github.com/Julian/jsonschema/pull/627
[2] https://governance.openstack.org/tc/reference/runtimes/victoria.html#python-runtimes-for-victoria

Change-Id: I2816360b59f82c996038929ed6e90b744feb3f75
2020-06-12 19:23:10 +02:00
Zuul
6418ae00c9 Merge "Fix netcat option in udp_check.sh for CentOS/RHEL" 2020-06-11 23:24:53 +00:00
Zuul
0a697a352c Merge "Switch oslo.policy over to yaml" 2020-06-11 22:11:11 +00:00
Zuul
18918267e3 Merge "Fix batch member create for v1 amphora driver" 2020-06-11 22:06:04 +00:00
Zuul
647fae2822 Merge "Fix some typos in the explanatory notes" 2020-06-11 22:02:49 +00:00
Zuul
b6e0221ca8 Merge "Remove all deprecated driver code that moved to octavia-lib" 2020-06-11 22:02:48 +00:00
Michael Johnson
66a67ba5af Clarify health monitor behavior
The basic cookbook document implied that pools without a health
monitor would eventually remove a failed member from the pool.
This will not happen as if there is not health monitor, the members
are assumed as ONLINE.

Change-Id: I6c52f163d8ac0456b4faf7d9bf5cc4a19ee6eeb7
2020-06-11 09:10:20 -07:00
Michael Johnson
958c3a18bd Improve terminology in an old spec file
Change-Id: I20bd0070c7eb24e981becbd24e8a98ca5eaff929
2020-06-11 07:37:51 -07:00
zhaoleilc
76616f35e7 Fix some typos in the explanatory notes
This patch changes 'defiend' to 'defined'
in the explanatory notes in octavia/
tests/functional/db/test_repositories.py

Change-Id: Ibb7f0f416a013b98edf72a5803aada71015cfade
2020-06-11 14:53:39 +08:00
Zuul
aee9cd6fe4 Merge "Use unittest.mock instead of mock" 2020-06-11 03:16:30 +00:00
Hervé Beraud
6cce3a72ae Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Also added and enabled a hacking check that would have caught this.

Change-Id: Idb10f84fd32c50db24f844352cb85de452181439
2020-06-09 11:25:00 -04:00
Zuul
be2acaeb36 Merge "Fix test_barbican_legacy for Python 3.8" 2020-06-09 12:18:23 +00:00
Brian Haley
de69b2c7ff Remove all deprecated driver code that moved to octavia-lib
In https://review.opendev.org/#/c/613709/ octavia was
changed to use octavia-lib for a lot of API driver-related
code and deprecation warnings put in place. Now that
we're in Victoria remove all the deprecation shims and
use octavia-lib exclusively.

Change-Id: If92988150479a7daf465af5f8df22818664a0fce
2020-06-08 14:41:00 -04:00
Michael Johnson
fafabad042 Switch oslo.policy over to yaml
Oslo.policy is moving away from using json format policy files[1].

This patch updates the Octavia documentation, policy configuration file, and
legacy admin-or-owner policy file to be in yaml format.

Octavia will continue to honor and support the json format file as long
as oslo.policy does, but this patch will encourage new deployments
to use the yaml format.

[1] https://docs.openstack.org/oslo.policy/latest/admin/policy-json-file.html

Change-Id: I925cc05981e677c0552b18f845fdbc512d2af22c
2020-06-08 08:54:07 -07:00
Zuul
c9e9fd9335 Merge "Change default tox envs from python37 to python3" 2020-06-08 06:45:25 +00:00
Carlos Goncalves
f97a89ded9 Fix UDP pool transform in LVS driver
Even though the API and database schema enforce a pool to have a load
balancing algorithm set and validate its value, the auxiliary transform
pool method had an invalid default value 'roundrobin'. Round robin in
LVS is 'rr'.

Change-Id: I72b669e7755c0851867453977946891d7074d92b
2020-06-08 08:25:02 +02:00
Carlos Goncalves
9e070e6e6d Fix listener update with SNI certificates
SNI certificates were not being set in the database on listener update.
A listener GET would not show the certificates in the sni_container_refs
attribute. Also, the API was allowing set of SNI certs on non
TERMINATED_HTTPS listeners.

Task: 39042
Story: 2007421
Story: 2007430

Change-Id: If5b6411a0b7c75441a406234c2792ea68d35d0fe
2020-06-08 06:13:37 +00:00
Zuul
409b89f141 Merge "Update the feature matrix for new features" 2020-06-08 05:20:17 +00:00
Zuul
daa28ffe21 Merge "Fixed typo in upgrade section" 2020-06-08 05:17:48 +00:00
Yang JianFeng
5d91913136 Add quota support to octavia's l7policy and l7rule
Current octavia has no l7policy and l7rule quota definitions. But
they are necessary for some scenarios. For example, implement
product design compatible with Neutron Lbaas.

Story: 2003382
Task: 24457
Change-Id: I09ee23dcb83f5f08a56e25cc05ff77caa3ad4230
2020-06-08 02:28:51 +00:00
Ross Martyn
611880cd15 Fixed typo in upgrade section
Change-Id: I9e7052bfbb990d526893e641236cbdb6e6203d67
2020-06-06 09:38:02 +01:00
Michael Johnson
c2ff9bce99 Update the feature matrix for new features
We missed updating the provider driver feature matrix for a few
new Octavia features. This patch updates the matrix.

Change-Id: I328830df19fb8df6ea93cee2ad2f0dbda03279a1
2020-06-05 12:39:15 -07:00
Michael Johnson
630a4e6a3c Fix batch member create for v1 amphora driver
A previous patch[1] missed batch_member_update when adding database
repository "get" method retries for new object creation actions.
This patch fixes batch member create to retry the database get call
when new members are being created via batch member update.
This issue only impacts the v1 amphora driver as the v2 driver
does not need to get these objects from the database.

Story: 2007581
Task: 39503

[1] 48e85569f7

Change-Id: Ia3476ab7b24dc3fd6e29ff2abe6eb6bacd9908ed
2020-06-05 09:08:48 -07:00
Zuul
3980c90403 Merge "Remove Babel requirement" 2020-06-05 11:33:24 +00:00
Zuul
e8db961ba4 Merge "Update cirros image to cirros-0.5.1-x86_64" 2020-06-05 11:33:22 +00:00
Dawson Coleman
270b973bf9 Add minimum TLS version option in octavia.conf
Add new configuration option "minimum_tls_versions" to octavia.conf.
Listeners, pools, or the default values for either will be blocked from
using lower versions.

Change-Id: Ifa0d695c2227772d6b37987a7857fe58ca660dc8
Story: 2006733
Task: 37171
Depends-On: I480b7fb9756d98ba9dbcdfd1d4b193ce6868e291
2020-06-04 13:11:01 -07:00
Carlos Goncalves
d031596b9f Add a periodic fast forward upgrade job
Fast forward upgrade (FFU) is an upgrade strategy that does in-place
upgrade and skips intermediary releases (e.g. N -> N+3).

This new periodic job fast forwards a cloud from Stein to master.

Please note that ideally a distro upgrade should also be performed but
Grenade does not support that today. For example, Bionic is a tested
Linux distribution on Stein, whereas on Victoria it is Focal.

Change-Id: I792f42468c37b6849c8fc085172841434d6a1376
2020-06-04 09:37:22 +02:00
Zuul
0321c28588 Merge "Add TLS version configuration for pools" 2020-06-04 07:04:51 +00:00
Zuul
16b96baff9 Merge "Add TLS version configuration for listeners" 2020-06-04 06:27:40 +00:00
Dawson Coleman
9a6da86481 Add TLS version configuration for pools
Add field tls_versions to pools for restricing TLS versions used.
This is a colon-separated string of versions to be used.
Available values (as defined in octavia-lib):
SSLv3, TLSv1, TLSv1.1, TLSv1.2, TLSv1.3

Add default_pool_tls_versions in octavia.conf

Note: TLSv1.3 connections will use haproxy's default ciphers
instead of the listener's tls_ciphers field

Change-Id: I480b7fb9756d98ba9dbcdfd1d4b193ce6868e291
Story: 2006733
Task: 37173
Depends-On: Ic33d9b9a256490ae1b048cdfd2475d6340509fdb
2020-06-03 21:58:47 +00:00
Dawson Coleman
6aad5d8b9f Add TLS version configuration for listeners
Add field tls_versions to listeners for restricting TLS versions used.
This is a list of versions to be used.
Available values (as defined in octavia-lib):
SSLv3, TLSv1, TLSv1.1, TLSv1.2, TLSv1.3

Add default_listener_tls_versions in octavia.conf.

Note that at this time TLS 1.3 ciphersuites are not impelemented,
so any TLS 1.3 connections will use haproxy's default ciphers
instead of what's specified by tls_ciphers.

Change-Id: Ic33d9b9a256490ae1b048cdfd2475d6340509fdb
Story: 2006733
Task: 37170
Task: 37169
2020-06-03 14:57:47 -07:00
Sean McGinnis
0deff25667 Change default tox envs from python37 to python3
Python versions supported by OpenStack change over time, and for minor
versions of Python 3 it is tedious to keep this file updated.

Since this does not impact zuul jobs in any way, nor prevent local
tests against py37, it should be safe to simply make this more easily
compatible for users that don't care about the specific Python versions
and just need basic tests to run.

The *only* thing this does is changes the default versions tested if none
are explicitly provided with `-e`.

Change-Id: I2372178351e961eeed5a819f39e75d54ba148798
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-06-03 07:28:27 -07:00
Hervé Beraud
bf460618bc Stop to use the __future__ module.
The __future__ module [1] was used in this context to ensure compatibility
between python 2 and python 3.

We previously dropped the support of python 2.7 [2] and now we only support
python 3 so we don't need to continue to use this module and the imports
listed below.

Imports commonly used and their related PEPs:
- `division` is related to PEP 238 [3]
- `print_function` is related to PEP 3105 [4]
- `unicode_literals` is related to PEP 3112 [5]
- `with_statement` is related to PEP 343 [6]
- `absolute_import` is related to PEP 328 [7]

[1] https://docs.python.org/3/library/__future__.html
[2] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html
[3] https://www.python.org/dev/peps/pep-0238
[4] https://www.python.org/dev/peps/pep-3105
[5] https://www.python.org/dev/peps/pep-3112
[6] https://www.python.org/dev/peps/pep-0343
[7] https://www.python.org/dev/peps/pep-0328

Change-Id: I193ba2193f24e6deac473423310b81a029e15e22
2020-06-02 20:32:07 +02:00
wu.chunyang
9bee93b201 trivial fix
Change-Id: I59d94bc9743b04674caca3ffcb8fc91ba803298c
2020-06-01 22:07:38 +08:00
Zuul
2c7c7747b7 Merge "Workaround peer name starting with hyphen" 2020-05-31 14:55:45 +00:00
Zuul
8a38c1a82b Merge "Fixed a bug: replace 'startwith' with 'startswith'" 2020-05-30 11:36:57 +00:00
Bodo Petermann
a84bf7d843 Fix test_barbican_legacy for Python 3.8
Fixes failing unit tests in
octavia.tests.unit.certificates.manager.test_barbican_legacy.TestBarbicanManager
for Python 3.8

Some of the tests fail setting up a mock.Mock(spec=secrets.Secret)
because a ValueError exception is raised unexpectedly.

The reason is that test_get_cert_no_registration_raise_on_secret_access_failure
patches the `payload` property of barbicanclient.v1.secrets.Secret to
raise a ValueError.
When a subsequent test tries to set up a mock.Mock(spec=secrets.Secret)
in Python 3.8 the Mock class will try to look at the properties of the spec
class and accessing `payload` doesn't behave normally anymore: it raises
ValueError now.

Fixed by using a different approach of mocking `payload` in
test_get_cert_no_registration_raise_on_secret_access_failure
so that it does not influence subsequent tests.

Change-Id: Ic534a4715c85c2216c7251209507acf74a999153
Story: 2007490
Task: 39212
2020-05-29 16:58:33 +02:00
Zuul
7e851d3f6e Merge "Fix the grenade plugin to also upgrade octavia-lib" 2020-05-29 13:48:39 +00:00
chimeng
7c96e455a9 Fixed a bug: replace 'startwith' with 'startswith'
story: 2007734
Change-Id: I6543fe5caf539c9b1167a7c58984b3474879e1de
2020-05-29 17:08:25 +08:00