268 Commits

Author SHA1 Message Date
Takashi Kajinami
abf7b9efcf Add conf file for oslo-config-generator
This change introduces the config file for the oslo-config-generator
command, so that users can easily generate octavia.conf.example without
tox.

Note this change adds parameters of oslo.policy and oslo.middleware
which were missing previously.

Change-Id: I5ea921cf8d63b28c5143f95dbb47802d5018d7a4
2022-09-04 05:14:16 +09:00
Tom Weininger
829e44acbd Make amphora timezone configurable
The new amp_timezone option allows to adjust the timezone of
amphora using cloud-init.

Change-Id: I2f49cbb7f8d99ba2da878bbfc7081a3cc3b3aa07
2022-05-06 12:13:41 +02:00
Zuul
d62d32e078 Merge "Ensure clean redis state in devstack env" 2022-03-28 11:12:21 +00:00
Gregory Thiemonge
74a7cbe122 Use centos amphora image in the FIPS jobs
The FIPS jobs use centos-8-stream controllers but the image is still
based on ubuntu, this commit updates the amphora images to
centos-8-stream and enable FIPS inside the amphora.

Change-Id: I8916796ed6727a103907a33d3c14e99e1d3734e6
2022-02-11 17:21:48 +00:00
Tom Weininger
64f404ac28 Ensure clean redis state in devstack env
Change-Id: I3b59f3ccceca3291b426110ad9c893bec5adcd2d
2022-02-10 15:53:31 +01:00
Gregory Thiemonge
791fbfce6c Fix devstack on centos-9-stream
debootstrap doesn't exist on centos-9-stream, disable it for
rhel9-based distribution.
Also remove a previous unsuccessful attempt to remove it.

Change-Id: I8fac328cdda6e1015a145a1f8a497e007ac6ac02
2022-02-07 21:20:53 +01:00
Gregory Thiemonge
237d443649 Mock log_opt_values in API tests & fix auth_url in devstack
A recent commit [0] sets log_opt_values to INFO in order to display
config options even if DEBUG is not enabled, but it makes our functional
tests 3 times slower because each api test dumps the configuration.

Mock log_opt_values in our API functional tests to speed up the
execution of the tests.

$OS_AUTH_URL was previously used to configure the service_auth.auth_url
setting in octavia.conf but this env var was removed from devstack in
[1]. Switch to $KEYSTONE_SERVICE_URI which was used to set OS_AUTH_URL.

[0] https://review.opendev.org/c/openstack/octavia/+/792697
[1] I86ffa9cd52454f1c1c72d29b3a0e0caa3e44b829

Change-Id: Ib9123365079502093b355db3430a98928a3a16d1
2021-11-18 21:16:08 +01:00
Zuul
a456b2cc8a Merge "Spare pool removal" 2021-05-05 12:33:01 +00:00
Gregory Thiemonge
815a283823 Spare pool removal
Spare pool feature was deprecated in Victoria, we decided to remove it
during the Xena release cycle.

Change-Id: I830c6a4c49fa47105f788cf99a0f775e5dbdcaea
2021-04-28 09:10:09 +02:00
Gregory Thiemonge
190d6a33dc Fix jobboard_enabled setting in devstack
When OCTAVIA_ENABLE_AMPHORAV2_JOBBOARD is True in devstack,
jobboard_enabled must be set to True.

Change-Id: I27b54b5b2e93665aad771438e618ac02900ab123
2021-04-23 16:29:10 +02:00
Gregory Thiemonge
5d470357ec Fix devstack cleanup when using amphorav2
When cleaning up a devstack environment with amphorav2 enabled,
./clean.sh might fail because redis has already been stopped and
uninstalled.
This commit move the stop_redis step to the octavia_stop step and
ensures that stopping redis doesn't fail.

Change-Id: I987535e90cbf13917c50c5905c64b614188928ec
2021-03-31 13:45:40 +02:00
Carlos Goncalves
e99d7b9389 Add aarch64/amd64 support to amphora and devstack
The disk image create tool can now build aarch64/arm64 amphora images.
The devstack plugin will facilitate image builds and upload to Glance
for this CPU architecture.

Change-Id: I1cebd8a3da58dc56ebbfac22f7802ab7f52585e1
2021-03-10 18:54:50 +01:00
Zuul
651f095a81 Merge "Remove Neutron SDN-specific code" 2021-03-10 11:07:11 +00:00
Michael Johnson
d17b23282c Fix grenade resource waiters
The grenade job resource.sh script is waiting for the created object
status but not the overall load balancer status to go ACTIVE.
This can lead to the script failing with a 409 error.

This patch adds a check for the load balancer to be unlocked before
advancing to the next create call.

Change-Id: I6505243ddbf1eab7d110e9bfa03bffda840f07ae
2021-03-05 01:40:25 +00:00
Gregory Thiemonge
f45092a876 Configure rsyslog on Octavia service nodes in devstack
{admin,tenant}_log_targets options are configured with
MGMT_PORT_IP in devstack, which contains the IP address
of the local management interface. In multinode setup,
it means that the second node should run a rsyslog
service to receive logs from amphorae that have been
spawned by its worker.

Change-Id: If2841720009c2e402127e2e0080efdd56b68f6c9
2021-02-09 12:06:19 +01:00
Ann Taraday
9a6a90000f Add experimental amphorav2 jobs
The following jobs with default amphorav2 added as experimental:
* octavia-v2-dsvm-noop-api
* octavia-v2-dsvm-tls-barbican
* octavia-v2-dsvm-spare-pool
* octavia-v2-act-stdby-dsvm-scenario
* octavia-grenade
* octavia-v2-dsvm-cinder-amphora
* octavia-v2-dsvm-scenario-two-node

Note: octavia-grenade-amphorav2 will show valid result
with renamed alias amphorav2 -> amphora.

Change-Id: I03385d93575db4e44a72335c0af4fde490a93b0f
2020-11-03 09:02:09 +00:00
Brian Haley
dcdf6315b3 Remove Neutron SDN-specific code
Migrated to Neutron repository.

Depends-on: https://review.opendev.org/#/c/741719
Depends-on: https://review.opendev.org/#/c/742027
Change-Id: I814d4e1abf3cee686b9f4511373edd46d1f5f4e2
2020-09-29 14:39:40 -04:00
Carlos Goncalves
cb7d5133e7 Ensure install of diskimage-create.sh requirements
Change I612ea1c583090897bd44453b867d75929a01b7fc [1] removed
diskimage-builder from the root requirements.txt file. Devstack
deployments not setting LIBS_FROM_GIT+=diskimage-builder will not
install diskimage-builder. This means our diskimage-create.sh tool will
fail to build amphora images and abort the deployment.

[1] https://review.opendev.org/#/c/741960/

Change-Id: Id535d2d5ea6c23c9646c568a17d7695f82c0c1a9
2020-09-24 12:06:00 +02:00
Zuul
f306ec7f46 Merge "Add requirements for jobboard drivers" 2020-09-10 22:02:29 +00:00
Zuul
87e55d6350 Merge "Add support for nftables" 2020-09-10 18:29:38 +00:00
Zuul
954cf8afb3 Merge "Add DIB element octavia-lib" 2020-09-08 21:42:20 +00:00
Michael Johnson
1a3b56a0d5 Add support for nftables
This patch adds support for nftables (an iptables replacement) to
the devstack plugin and the amphora agent.

Change-Id: I9e2c4d6e68da67d68c6dfeb3b47edd600d1ba397
2020-09-03 13:15:27 -07:00
Carlos Goncalves
a422e5a203 Introduce an image driver interface
With this image driver interface, we align our codebase with other
existing driver interfaces like compute, network and volume.

This interface also allows the amphora provider driver to check for
existence of tagged images at API level (e.g. amphora image tag
capability in Octavia flavors).

Change-Id: Id808c082808fafe1a1e004957ff47eca57f97ee8
2020-09-03 20:06:35 +00:00
Ann Taraday
460566e328 Add requirements for jobboard drivers
As Octavia allows to use RedisTaskFlowDriver or
ZookeeperTaskFlowDriver we should install python clients that
allows to work with redis and zookeeper backends.

Story: 2007892

Change-Id: I7312c8c1057618e909339aa7a4dfeb836f4b8f33
2020-09-03 08:02:01 +00:00
Zuul
0b1d8dd5e7 Merge "Update devstack plugin" 2020-08-26 23:58:54 +00:00
Zuul
2d97ebcd24 Merge "Allow amphorav2 to run without jobboard" 2020-08-25 22:47:04 +00:00
Zuul
703a07df21 Merge "Ensure xtrace is enabled in devstack plugin" 2020-08-24 14:32:58 +00:00
Carlos Goncalves
dcda721320 Add DIB element octavia-lib
This new diskimage-builder element installs octavia-lib from Git for
source install type image builds rather than from released versions.

To mention some advantages:

1. allow custom octavia-lib versions (useful for development)
2. test unreleased octavia-lib changes in CI
3. install latest changes from master and stable branches
   (aligns with approach taken with amphora-agent)

Branch checkout to stable/* from master for octavia-lib DIB element will
be automated on branch create. See https://review.opendev.org/#/c/745877

Change-Id: I6d87b6bd25c536b2bed1994427cd933bdcc091d6
2020-08-24 11:24:19 +02:00
Michael Johnson
2e2464d778 Increase the devstack secuirty group rules quota
Recent changes to the Octavia tempest tests has caused our test
runs to exceed the default neutron security group rule quota.
This patch increases that quota for the Octavia project in devstack.

Change-Id: I49e92f81b23e0b306c62c406a45029b96dce20df
2020-08-18 16:09:11 -07:00
Michael Johnson
e45937a47c Ensure xtrace is enabled in devstack plugin
Currently some phases of the devstack plugin do not have xtrace
enabled, which can make it hard to debug issues in the devstack
plugin.
This patch makes sure that xtrace is enabled while the Octavia
devstack plugin is running so that we can see the commands being
executed. It will restore the previous setting after our plugin
is done running.

Change-Id: Id6828b86779f1daca6a00a03f43c78fe26828f4f
2020-08-18 15:07:02 -07:00
Carlos Goncalves
bb9b30be7e Allow amphorav2 to run without jobboard
This patch adds a new configuration setting to enable/disable jobboard
functionality in the amphorav2 provider. When disabled, the amphorav2
provider behaves similarly to the amphora v1 provider.

The default setting is jobboard disabled while jobboard remains an
experimental feature.

Change-Id: I063d832f5a049d7ae38378766200c7f82a35996d
2020-08-17 13:57:33 +00:00
Jens Harbott
dac24cd7ec Update devstack plugin
There is no reason to create internal and admin endpoints in DevStack,
most other services have stopped doing so a long time ago.

Also use the global SERVICE_PROTOCOL variable as default for
OCTAVIA_PROTOCOL instead of "http", this will make us automatically
use the secure API access when the tls-proxy service is enabled.

Fix the instructions for using the devstack plugin, too.

Change-Id: I0154b83cb64952844a28895721694d3e2ff82be2
2020-08-11 10:22:58 +02:00
Michael Johnson
77f5c3893c Add a router to the lb-mgmt-net for slaac
Neutron now needs to have a router attached to the subnet to provide
the router advertisement messages needed for slaac address
configuration.
This patch adds this router to the lb-mgmt-network, allowing
the amphora instances to configure an IPv6 address.

Change-Id: I638c5c8baf1d76365fff2c99ded9c6b310348710
2020-07-07 09:16:21 -07:00
Zuul
f399bd83c2 Merge "Fix error on devstack cleanup" 2020-06-26 21:38:42 +00:00
Carlos Goncalves
e5951ced5f Use uwsgi binary from path
In-line with devstack patch [1], switch invocations to find uwsgi in the
path.

[1] https://review.opendev.org/#/c/577779/

Change-Id: I5e6aee49f434820881051874c9ad2628b4fcada7
2020-06-17 10:59:29 +02:00
Gregory Thiemonge
c176cf60d4 Fix error on devstack cleanup
Change I7ebf4137feb04827490dffc0dac3d6e4c8888075 added 'set -e' in
devstack/plugin.sh, but on devstack cleanup, some commands may fail
because of non-working services (i.e after a reboot).
This commit allows 'openstack keypair delete' to fail on clean up.

Change-Id: Ic782faba3eb907d29b6735ac0a6d6a8a2e104e00
2020-06-15 08:44:32 +02:00
Zuul
0a697a352c Merge "Switch oslo.policy over to yaml" 2020-06-11 22:11:11 +00:00
Michael Johnson
fafabad042 Switch oslo.policy over to yaml
Oslo.policy is moving away from using json format policy files[1].

This patch updates the Octavia documentation, policy configuration file, and
legacy admin-or-owner policy file to be in yaml format.

Octavia will continue to honor and support the json format file as long
as oslo.policy does, but this patch will encourage new deployments
to use the yaml format.

[1] https://docs.openstack.org/oslo.policy/latest/admin/policy-json-file.html

Change-Id: I925cc05981e677c0552b18f845fdbc512d2af22c
2020-06-08 08:54:07 -07:00
Zuul
e8db961ba4 Merge "Update cirros image to cirros-0.5.1-x86_64" 2020-06-05 11:33:22 +00:00
Carlos Goncalves
59831f46f7 Fix the grenade plugin to also upgrade octavia-lib
Upgrade octavia-lib in target environment to make sure we are testing
the latest code.

Change-Id: I8003de0f71a5dee7438ab2f6a2c497386018c1ac
2020-05-22 19:46:31 +02:00
Zuul
04ea9bf4dd Merge "Make sure devstack aborts if DIB fails" 2020-05-21 22:41:47 +00:00
Michael Johnson
d0f0233061 Make sure devstack aborts if DIB fails
There are cases where DIB can fail to create an image but devstack
does not abort. This leads the gate job to run all the way down to
starting the tempest test before the job will fail out.
This adds a simple check for the image file and will abort early
if the image is not present.

Change-Id: I7ebf4137feb04827490dffc0dac3d6e4c8888075
2020-05-19 16:26:57 +00:00
Carlos Goncalves
8b8965bd7b Migrate grenade job to native Zuul v3
This patch also switches the job back to voting.

Change-Id: Iedc1d5c5b603753b171fe17816b4d9c3aff1a16c
2020-05-04 14:01:40 +02:00
Ann Taraday
fb70e3b3c1 Run taskflow jobboard conductor conditionally
Run taskflow jobboard conductor only if amphorav2 provider is
enabled.

Fixes devstack plugin.sh conditions for amphorav2 provider.

Change-Id: I49b587cf748996658859667485400307205d209b
2020-04-15 20:11:07 +04:00
Ann Taraday
9c13499d21 Jobboard based controller
Introduce TaskFlowServiceController which uses taskflow
jobboard feature and saves jobs info into persistence backend.

Jobboard could be operated via RedisTaskFlowDriver or
ZookeeperTaskFlowDriver, that could be set via the config.
RedisTaskFlowDriver is intoduced as default backend for jobboard.
Usage of jobboard allows to resume jobs in case of restart/stop
of Octavia controller services.

Persistence backend saves state of flow tasks that required in
case of resuming job. SQLAlchemy backend is used here.

Bump taskflow version to 3.7.1 and add dependency to
SQLAlchemy-Utils (required for taskflow sqlalchemy
backend support).

Story: 2005072
Task: 30806
Task: 30816
Task: 30817

Change-Id: I92ee4e879e98e4718d2e9aba56486341223a9157
2020-04-08 19:53:09 +04:00
Brian Haley
70f5df1e17 Update cirros image to cirros-0.5.1-x86_64
Change example string and depend on devstack change to
make sure we won't break.

Depends-on: https://review.opendev.org/#/c/711492
Change-Id: I5ca700b27c3c5be26dc5132fafd9b6cffade2b6b
2020-03-18 12:21:44 -04:00
Carlos Goncalves
dc2dd711e6 Stop supporting CentOS 7 controllers and images
The diskimage-create.sh tool will now default to CentOS 8 when building
CentOS-based amphora images.

This patch also removes leftover references to support for Ubuntu Trusty
and Xenial.

Change-Id: I3aba59c8dd86aeeee28cc6a67af93697912fb55b
2020-01-08 17:59:09 +01:00
Zuul
2a60c13863 Merge "Fix controller worker graceful shutdown" 2019-11-18 17:53:52 +00:00
Gregory Thiemonge
215c4c2284 Fix controller worker graceful shutdown
Fix an issue that prevents graceful shutdown of controller workers.

cotyledon.Service.terminate function is by definition the graceful
termination function and doesn't have any 'graceful' optional boolean
argument (https://cotyledon.readthedocs.io/en/latest/api.html).

Because of this error, message_listener.wait() was never called in the
consumers' termination functions, so flows could be interrupted before
completion and could leave resources such as load balancer in a
PENDING_* provisioning state.

By default cotyledon.Service terminates the server after a timeout if
the worker could not shutdown itself gracefully. The default value
for the timeout is 300 seconds (set in devstack plugin) and can be
overriden using the graceful_shutdown_timeout setting in octavia.conf
The default value will be updated to a lower value when work on
persistant taskflow will be merged.

Story: 2006603
Task: 36770

Change-Id: I3f776bd018246897c9a889699a2d0ecbbfbb7098
2019-10-24 14:27:35 +02:00
Adam Harwell
2c6d8fc5ec Allow IPv6 health network in devstack
This allows the Octavia devstack plugin to use IPv6 for the service
endpoint as well as for the lb-mgmt-net.

Co-Authored-By: Michael Johnson <johnsomor@gmail.com>

Change-Id: I87917440565ea953ec0d762a7e219d912c56c418
2019-10-21 22:47:28 +00:00