2842 Commits

Author SHA1 Message Date
Michael Johnson
a84352af56 Add missing test requirement "fixtures"
Change-Id: I1cbf4dff958a36d2af2883cb2596fab53741cef1
2019-07-23 12:52:59 -07:00
Zuul
ba88441935 Merge "Update api-ref location" 2019-07-23 08:25:38 +00:00
Zuul
2363a929f3 Merge "Add cloud-init-datasources DIB element" 2019-07-23 00:01:05 +00:00
Michael Johnson
45cf2f328f Add project_id to all of the provider objects
This patch adds a project_id field to all of the provider driver data
model objects.

This is useful for drivers to track/associate objects on the backend.

Change-Id: I8f509677da463bc5d0a7649f2f609045cf9b2dd7
2019-07-22 20:49:19 +02:00
Andreas Jaeger
bc19e07923 Update api-ref location
The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.

Note that redirects will be set up as well but let's point now to the
new location.

For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html

Change-Id: I9f617159eddc8af532403e47c325048d92cc9ade
2019-07-22 19:18:40 +02:00
Michael Johnson
2bb2f918ee Fix cryptsetup --pbkdf-memory failures
A recent patch[1] added --pbkdf-memory to the cryptsetup command line
to limit the memory cryptsetup is using. However, some distros use
an older version of cryptsetup that does not need this setting.
This patch adds logic to detect this and run the commands without
--pbkdf-memory.

[1] https://review.opendev.org/663784

Change-Id: I9e0debcbfe6ceeff0012c827d70d80d938b5a2fb
Story: 2006066
Task: 34782
2019-07-17 21:32:45 +00:00
Carlos Goncalves
b12e5bb25e Add cloud-init-datasources DIB element
The cloud-init-datasources DIB element configures cloud-init to only use
an explicit list of data sources. Until the switch from ubuntu element
to ubuntu-minimal, cloud-init was being configured to only use
ConfigDrive datasource because the ubuntu element had
cloud-init-datasources as element dependency. The ubuntu-minimal and
other OS elements (e.g. RHEL-based) do not have it, so the export
DIB_CLOUD_INIT_DATASOURCES was not being used.

Change-Id: I74ecc826596c5f739a49defe6588bcc741b03594
2019-07-16 14:34:26 +02:00
Zuul
58a4490bbf Merge "Add warning log if auth_strategy is not keystone" 2019-07-16 05:31:44 +00:00
Zuul
5f38432c2f Merge "Fix dhclient enter hook path on RHEL-based amps" 2019-07-16 05:31:40 +00:00
Frode Nordahl
5114a9a4f9
elements: add arch property for `open-vm-tools`
The ``open-vm-tools`` package only exists for x86-platforms.

Task: 35814
Story: 2006219

Change-Id: I9f1b6e58004d7c2024f8d60d50ac998240abb6ee
2019-07-15 08:06:26 +02:00
Zuul
b49334bd2a Merge "Add Python 3 Train unit tests" 2019-07-13 10:20:46 +00:00
Michael Johnson
48371c98ee Add warning log if auth_strategy is not keystone
A user came to the IRC channel with CLI errors:
"Client-side error: Validation failure: Missing project ID in
request where one is required."

The root cause was the [api_settings] auth_strategy was set to
"noauth" instead of "keystone".

This patch adds a warning log message to the API process that
warns users that typically the auth_strategy should be set to
keystone.
It also points the user to have an administrator check the keystone
settings in the octavia.conf.

Change-Id: I7793d7a9113b23ac88e7c53d5dc292a70b9453b5
2019-07-11 09:15:00 -07:00
Carlos Goncalves
6c0756864f Fix dhclient enter hook path on RHEL-based amps
dhclient-script(8) defines the enter hook path is
/etc/dhcp/dhclient-enter-hooks:

"On  after defining the make_resolv_conf function, the client script
checks for the presence of an executable /etc/dhcp/dhclient-enter-hooks
script, and if present, it invokes the script inline, using the Bourne
shell command."

This was confirmed to be valid on RHEL and CentOS 7-8, and Fedora 28-30.

Change-Id: I473f1e5c6862ebf0d691a8191d17649ccf51e7f4
Task: 35725
Story: 2006190
2019-07-09 12:34:16 +02:00
Michael Johnson
e3aacb67ad Switch the default Ubuntu kernel to -virtual
A previous patch changed the default Ubuntu kernel for the amphora
images to the -kvm kernel[1]. This kernel is missing a module required
for cryptsetup.
This patch switches the default to the -virtual kernel, which is
slightly larger than the -kvm kernel, but includes the required
kernel module for cryptsetup.

[1] https://review.opendev.org/665861

Change-Id: Ia05d7e9f8ac2692e5769648255ef0e4e10a18c03
2019-07-08 10:30:43 -07:00
Corey Bryant
f8a2560ed9 Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html

Change-Id: I76cdc49e482458a2beefb26b9301023559d16029
Story: #2005924
Task: #34227
2019-07-05 14:03:42 -04:00
Zuul
31428bf25a Merge "only rollback DB when we have a connection to the DB" 2019-07-03 05:52:32 +00:00
tonybrad
de30dbb62f Update tox.ini for new upper constraints strategy
The requirements team has defined a new strategy for handling
upper constraints[1]. This patch applies those strategies to
Octavia.
This also corrects a lower constraint for oslo.messaging.

[1] http://lists.openstack.org/pipermail/openstack-discuss/ \
    2019-May/006478.html

Change-Id: I4cf0ad5d14c4378e33423a4d639ec4d0fcc2a614
2019-06-28 17:07:06 -07:00
Chuck Wilson
feaa1ce021 only rollback DB when we have a connection to the DB
Story: 2006062
Task: 34775

Change-Id: I728a9aa82986edef8128907a48c9df87d119fa8f
2019-06-27 13:06:50 -07:00
Zuul
520f1d0c7d Merge "Switch python3 versions of test jobs to match Train PTI" 2019-06-27 12:15:07 +00:00
Zuul
fbe8368779 Merge "Workaround pyroute2 platform-dependent imports" 2019-06-27 08:05:27 +00:00
Michael Johnson
cd50ae5c0c Specify the linux-image-kvm kernel for ubuntu
This patch switches to using the linux-image-kvm kernel by default.
It produces images that are about 200MB smaller than ubuntu-minimal with
linux-image-generic by using the linux-image-kvm instead of the
linux-image-generic kernel.

Depends-On: https://review.opendev.org/666063
Change-Id: I4103bcbaab65ed5d7683d35aaf04955e59ed2471
2019-06-26 23:32:12 +00:00
Michael Johnson
09733a0c39 Add bindep.txt for Octavia
Some base packages are no longer automatically included in zuul
job runs. This is good as we need to call out our platform package
requirements.
This patch adds a bindep.txt to Octavia to capture those
package requirements.

Change-Id: I48e82f39e3980d6d50e3b710326b8e437eddb08b
2019-06-24 12:59:12 -07:00
AjayKumar333
a144e512cf Switch python3 versions of test jobs to match Train PTI
In Train, we will use python 3.6 and 3.7 for python3 runtime
in our gate jobs [1]. This commit also adds python 3.7.

[1] https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: I8acde313e7e295cdecaa593dbeb41e8ee49afed5
2019-06-24 16:32:21 +05:30
Zuul
3a9df30a9a Merge "Fix pool API handling of None/null updates" 2019-06-24 04:53:43 +00:00
Zuul
b2d40c1120 Merge "Fix health monitor API handling of None updates" 2019-06-24 04:32:53 +00:00
Zuul
77dc9fbd95 Merge "Install DIB binary dependencies from bindep.txt" 2019-06-21 12:36:57 +00:00
Zuul
a485934de3 Merge "Clarify that the certificate guide should be used" 2019-06-18 16:36:30 +00:00
Zuul
3f033afbdc Merge "Add a note about nova hardware architectures" 2019-06-18 16:36:28 +00:00
Adam Harwell
624a63dce0 Workaround pyroute2 platform-dependent imports
Pyroute2 recently changed the way they import their base namespace
ojects. Now they are somewhat platform dependent, so on OSX unit tests
will fail for lack of linux-specific libs. Using `create=true` allows
the mocks to be created even if the library doesn't think they exist.
b69999ec82

Change-Id: I8f9c25e979a7e86ff0b8e19267e4525817802ba2
2019-06-18 01:34:32 -07:00
Zuul
954025cbc3 Merge "Fix a python3 issue in the amphora-agent" 2019-06-18 07:42:58 +00:00
Michael Johnson
f1542afba0 Add a note about nova hardware architectures
This patch adds a note to the diskimage-create README that
reminds users to set the hardware architecture property on the
amphora image if the cloud has multiple architectures available
in nova.
It also sets this property in our devstack plugin to provide an
example/reminder.

Change-Id: I15c2cabb1bced0a5646a3e1aed50bece78afcdce
2019-06-17 10:37:57 -07:00
Michael Johnson
dc459e2213 Fix a python3 issue in the amphora-agent
An exception handler in the amphora-agent has a python3 string
comparison bug that will cause a TypeError.
This patch fixes that bug and adds test coverage for the
start_stop_listener.

Change-Id: I6f5d95c5f875edda530f54ae72386d6495235ca6
Story: 2005898
Task: 33760
2019-06-17 15:21:35 +03:00
Zuul
6e6f3a1f1c Merge "Fix TCP listener logging bug" 2019-06-17 10:48:29 +00:00
Zuul
ff73592cae Merge "Amphora logging" 2019-06-15 05:12:46 +00:00
Zuul
acf551c0bd Merge "Fix allocate_and_associate DB deadlock" 2019-06-15 02:08:29 +00:00
Zuul
41e7cfa09f Merge "Align logging in the amphora" 2019-06-14 21:07:06 +00:00
Michael Johnson
83af850ec2 Fix TCP listener logging bug
HAProxy is not handling two of the HTTP log format variables correct
when the load balancer has a TCP listener.
This patch corrects that problem.

Change-Id: I2eb8a0b5de46ee56321bc0009b6ca2b3ad4caebf
2019-06-14 13:38:55 -07:00
Michael Johnson
0eaab450c1 Fix allocate_and_associate DB deadlock
In testing we saw a DB deadlock in allocate_and_associate.
This patch corrects that with a DB retry wrapper.

Change-Id: I20572505e448f9b7ff6314b5167cc4c519c445fb
Story: 2005893
Task: 33751
2019-06-14 09:41:09 -07:00
German Eichberger
686303e79d Amphora logging
Configure rsyslog to forward logs to a target host

Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Story: 1665069
Task: 33646

Change-Id: I00703f86555cbb574b943794b14a36fbc644f1b2
2019-06-14 09:02:26 -07:00
Michael Johnson
80ddbaeef4 Align logging in the amphora
This patch configures the primary components of the amphora to log
to syslog using consistent logging facilities.
By default, user traffic logs will go to LOG_LOCAL0 and the amphora
processes (haproxy, keepalived, etc.) will log to LOG_LOCAL1.

This is a patch supporting log offloading.

Change-Id: Ifda91e0310e812e34f1e398dd3176af8a9c58f89
Story: 1665069
Task: 5486
2019-06-13 12:42:18 -07:00
Zuul
620c19804c Merge "Add RHEL 8 amphora support" 2019-06-09 13:56:08 +00:00
Carlos Goncalves
8821737359 Add RHEL 8 amphora support
Network scripts are deprecated in RHEL 8. This patch makes sure the
package is installed and the service enabled. Sometime in the future
(Train release or newer), support for Network Manager will be added as
it became the default network configuration tool in RHEL/CentOS 8 and
Fedora 28+.

This patch also reflects a change to diskimage-builder that will now
have a version-less 'rhel' element as opposed to a separate 'rhel7'
and 'rhel8' elements.

Change-Id: Id11459ea70479aa0145059f88af847dddcd93553
2019-06-07 18:34:29 +02:00
Michael Johnson
4007d56b3e Limit cryptsetup key RAM usage
This patch limits cryptsetup to 256MB of RAM during the amphora
startup. Recent distros have changed to LUKS2 with Argon2
key derivation which defaults to using up to 1GB of RAM.
Typically our amphora are built with only 1GB of RAM for the whole
system.

Change-Id: I018e36f69a9c0b48a6651a01cc9a64abfc04d4de
Story: 2005837
Task: 33606
2019-06-06 15:10:36 -07:00
Zuul
ff4680eb71 Merge "Create Amphora V2 provider driver" 2019-06-06 05:21:31 +00:00
Michael Johnson
0ab16921ae Create Amphora V2 provider driver
This patch creates an Amphora v2 provider driver as well as a
V2 controller worker.
This is in preparation for having the amphora driver use the new
provider driver data models and rely less on native Octavia database
access.
It is also a prepartion step for enabling TaskFlow JobBoard as
this work will move to storing dictionaries in the flows instead
of database models.

Change-Id: Ia65539a8c39560e2276750d8e79a637be4c0f265
Story: 2005072
Task: 30806
2019-06-05 14:30:29 -07:00
Michael Johnson
6d2e2be86a Remove references to OpenStack Anchor
The OpenStack Anchor project is now officially retired[1].

This patch removes the references to Anchor from Octavia.
These old references were confusing new users.

[1] https://review.opendev.org/#/c/611187/

Change-Id: Idfe90aa69b497e8270118174dde00567d7fab4ab
2019-06-03 14:58:44 -07:00
Zuul
59660fb365 Merge "Force amp-agent communication to TLSv1.2" 2019-06-03 13:56:23 +00:00
Michael Johnson
d3c3f148f4 Clarify that the certificate guide should be used
The quick start guide was pointing users to the old create_certificate.sh
script. This patch clarifies that the "Octavia Certificate Configuration
Guide" should be used when configurting certificates for the Octavia
controllers.
It also adds a warning to the create_certificate.sh script.

Change-Id: I7a87430be5a1553ff760e31c26ca52090399d00a
2019-05-31 09:13:07 -07:00
Michael Johnson
a95f748156 Fix health monitor API handling of None updates
The current health monitor API does not properly handle
clearing/reseting values on update. Some integer only fields,
such as max_retries_down, will accept null, but will store the
value as "None". These will will cause failures updating the
amphora configuration.

This patch corrects this to appropriately handle None/null updates
to the health monitor parameters.

Change-Id: Ida1d544933aec9e5cd556aef57a06e9f19f1b255
Story: 2005374
Task: 33533
2019-05-30 11:15:36 -07:00
Zuul
21a54d6b78 Merge "Rename review.openstack.org to review.opendev.org" 2019-05-29 04:38:27 +00:00