103 Commits

Author SHA1 Message Date
Jenkins
942afaacec Merge "Auto-detect haproxy user_group" 2017-04-23 23:24:29 +00:00
Nir Magnezi
26a55415ab Auto-detect haproxy user_group
As a followup to Id99948aec64656a0532afc68e146f0610bff1378, adding auto
detection to haproxy_amphora.user_group

haproxy is capable[1] handling a list of configuration files.
This patch leverages that capability by simply providing haproxy with an
additional configuration file, which is baked in the amphora image via a
diskimage-builder element.

The above-mentioned element will specify the following values for user group:
Ubuntu: 'nogroup'
RHEL/CentOS/Fedora: 'haproxy'

The amphora-agent will parse and remove any user_group configuration provided
by Octavia controller worker.
This is in order to maintain amphora-agent backward compatibility to old
Octavia workers, who still provide user_group to the amphora-agent.
Octavia Workers that include this patch will no longer provide user_group
configuration to the amphora-agent.

[1] https://cbonte.github.io/haproxy-dconv/1.7/management.html#3

Related-Bug #1548070

Change-Id: Ia8fede9d7da4709a48661d1fc595a16d04fcbfa9
2017-04-23 18:24:23 +03:00
Adam Harwell
9027154a5a Removing dependency on eventlet and oslo.service
Change-Id: I453e9b86d4edfedd63cc59e47bf745e166ff836f
2017-04-21 07:07:12 +09:00
German Eichberger
e58721c4e0 Adds a new config for soft-anti-affinity
Introduces a new config parameter to specify the anti-affinity
policy.

Bumps nova version.

Closes-Bug: 1677604

Change-Id: I8c50057bd43873182058097e802bc839d1be0554
2017-03-31 14:30:53 -04:00
Aishwarya Thangappa
e94ff2681f Adds a new feature to limit the amphora build rate
This patch limits the number of Amphora build requests handled by the
controller worker at a given time.

Also, the amphora build requests are assigned priorities based on
whether it is a normal loadbalancer create, failover or spares pool
loadbalancer create request. Based on the priority and the order in
which the requests were made if there is an available build slot the
amphora will be built.

Co-Authored-By: Lubosz "diltram" Kosnik <lubosz.kosnik@intel.com>
Change-Id: I967cf0668f82fb3a63e18dc7a457c58b526b7e66
Closes-Bug: #1571802
2017-03-16 16:50:31 +00:00
Jenkins
46252d0f5c Merge "Remove config option "amp_network"" 2017-02-15 01:42:31 +00:00
Jenkins
7c9baeb9d1 Merge "Add option to choose an availability_zone for amps" 2017-02-14 00:23:19 +00:00
Jenkins
bcd6f592d3 Merge "Allow a configuration option for random Amphora name" 2017-02-13 22:39:53 +00:00
Adam Harwell
e713f05b2a Add option to choose an availability_zone for amps
Change-Id: Id91c15da6caa656925184dda43d063aede89989e
2017-02-13 14:01:25 -08:00
johnsom
43949908e1 Remove mention of deprecated verbose option
oslo.log has deprecated the verbose option [1] so we should remove
mention of it as well.

[1] https://review.openstack.org/#/c/206437

Change-Id: I322e7b635f7337a6d399d7728bf55e581160b823
2017-02-13 08:50:55 -08:00
ZhaoBo
dc7c2967aa Remove config option "amp_network"
This option had been deprecated in N release. We could remove it from
master.

Change-Id: Ibd498f32a3c4f34621f3d814f1186a5bf9c5b75e
2017-02-13 09:59:41 +08:00
Adam Harwell
f345b4273b Allow a configuration option for random Amphora name
Also do a minor config file cleanup (looks like a bad previous merge).

Change-Id: I66e71795a4910c91cc2af4107fc60cc5aae72c79
Closes-Bug: #1663037
2017-02-08 16:26:29 -08:00
German Eichberger
bfb8195f16 Fixes misspelled amp_ssh_access_allowed config
The config is misspelled in the sample octavia.conf file
leading to confusion. This fixes that.

Change-Id: I0ed36f3e12a6a9f973918e3d233e54e8834be57f
2017-02-02 11:18:21 -05:00
Nir Magnezi
c00488143d Fix the amphora image support for RH Linux flavors
Not all Linux flavors accept the same type of configuration to manage
NICs. The amphora-agent must be able to distinguish between different
Linux flavors and choose the appropriate type of jinja2 NIC
configuration template for each one, respectively.

Up until now, The amphora-agent had no notion of the operating system
it is running on, therefore it used NIC configuration templates that
only match Debian based Linux flavors (mostly Ubuntu). Making it
unusable for flavors such as RHEL, Fedora and CentOS.

This fix enhances how the amphora-agent is handling NIC hot plugs.
It will use the appropriate jinja2 template by checking the Amphora
distribution name when needed.

Co-Authored-By: Michael Johnson <johnsomor@gmail.com>

Closes-Bug #1548070

Change-Id: Id99948aec64656a0532afc68e146f0610bff1378
2017-01-31 20:05:07 +02:00
He Qing
7468a2a6a4 Remove dumplicated config option 'cert_generator'
There are two 'cert_generator' options in config file. Remove one
of them.

Change-Id: I2daec1baf7832b14e6fe38ee73588bd86d55e060
Closes-Bug: #1629162
2017-01-18 19:21:35 +00:00
Jenkins
ae3204986e Merge "Add quota support to Octavia" 2017-01-13 18:11:08 +00:00
Trevor Vardeman
7d933da31e Add quota support to Octavia
Octavia has no quota definitions, but needs them for parity with Neutron LBaaS.
This will provide an endpoint and support for retrieving, updating, and deleting
quotas for projects, as well as adding enforcement of those those quotas.

Adds scenario test that simply validates quotas in a lb graph.

Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Co-Authored-By: Phillip Toohill <phillip.toohill@rackspace.com>
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>

Change-Id: Ia1d85dcd931a57a2fa3f6276d3fe6dabfeadd15e
Closes-Bug: #1596652
2017-01-13 02:45:14 +00:00
Nir Magnezi
e75c8ecc09 Adds user_group option under haproxy_amphora
This fix adds the user_group configuration option under the haproxy_amphora
section, which is currently available in the neutron-lbaas code base.

The incentive for the above-mentioned addition is that in CentOS, Fedora and
RHEL based amphora images, the 'nogroup' user group does not exist by default.
Therefore users who wish to use those type of amphora images should have the
option to configure a different user group such as 'haproxy'.

The default value for  user group is 'nogroup' so the default behavior
remains intact.

Related-Bug #1548070
Change-Id: Ifac59889fa8120d974840bae2913587292f474c1
2017-01-09 19:11:25 +00:00
Adam Harwell
e51a073614 oslo_messaging_rabbit settings are deprecated
Use [DEFAULT]/transport_url instead.

Change-Id: I6b8a44958570c970b6ce1a48e5d7c09cb2ec3c43
2016-12-19 16:23:40 -08:00
Jenkins
4f0571f64e Merge "Add keystone authentication of token" 2016-12-13 13:45:28 +00:00
Dustin Lundquist
6ce85349c9 Enable IPv6 load balancer networks
This patch addresses several places where IPv6 and IPv6 link-local
addresses where not considered for communication between amphora and the
controller worker.

In the devstack plugin we permit both IPv4 and IPv6 for health
monitoring and the amphora REST API.

In the amphora's UDP health sender we parse the IP port string in a
manner which permits IPv6 addresses by splitting on the last colon
rather than every colon.

In the controller REST API driver we append an interface scope if using
IPv6 link-local addresses. This interface can be specified by an
operator is they are using an interface other than o-hm0, this only is
required if using IPv6 link-local addresses.

Change-Id: I9d07bec4ac105e8876fadb72a83a590ffd4d2e66
2016-11-23 12:03:42 -08:00
Jenkins
8fef2f04a7 Merge "Run amphora agent with gunicorn" 2016-11-23 17:05:19 +00:00
Brandon Logan
1ace351fd8 Add keystone authentication of token
Closes-Bug: #1532075
Change-Id: Id45a0babc8e128d02bf648fedb7b66099bc3c7ae
Co-Authored-By: Lubosz "diltram" Kosnik <lubosz.kosnik@intel.com>
Depends-On: Id0deee2714040d271f43a537c27f410e2f4e3ef2
2016-11-22 12:57:07 -06:00
Jenkins
7f13dbc917 Merge "Backend Keystone authentication" 2016-11-18 13:57:44 +00:00
Lubosz "diltram" Kosnik
076e016bb2 Backend Keystone authentication
Change methods used in backend to authenticate with keystone.
Use autodetection mechanizm for API version and refactor config
options specified in Octavia.

Change-Id: Id0deee2714040d271f43a537c27f410e2f4e3ef2
Closes-Bug: #1620668
Closes-Bug: #1618691
2016-11-17 11:03:40 -06:00
Michael Johnson
c4408c4c78 Adds support for systemd amphora images
This patch enables auto-detection of the init system used in the
amphora image and adds support for systemd amphora.
This patch allows Ubuntu xenial amphora images to work.
It also merges two functional test files into one file to reduce
code duplication.

This is a scenario gate fix.

Change-Id: I5fec1680bd47719ae9f2fcb6abaaba8a78e2ae8b
Closes-Bug: #1640866
2016-11-16 00:42:37 +00:00
Adam Harwell
48a1e7cbe9 Run amphora agent with gunicorn
Flask's default runner (werkzeug) is plagued with bugs.
If we use gunicorn instead, we should have many less problems!

Depends-On: I211dc771aa95147c0f1d9e6ac1a65a7e164b33c2
Change-Id: I59897167f9285bf013f8a155dd2ea4f799ac1d3f
2016-10-26 17:42:49 +02:00
zhangyanxian
7feb462ea3 Fix typos in amphora_flows.py & octavia.conf
trivial fix

Change-Id: I8417740f03e4003714a5bbd9cde7ba520c5e36c1
2016-09-23 00:36:34 +00:00
Michael Johnson
d7d062a47a Option to restrict amp glance image owner
This patch adds an optional configuration setting that allows an
operator to restrict the amphora glance image selection to a specific
owner id.  This is a recommended security setting for clouds that
allow user uploadable images.

Change-Id: I73347b5b3e868d13974cd6ca6bada9cdf75773fe
Closes-Bug: #1620629
2016-09-15 19:46:46 +00:00
Paul Glass
e291a88210 Stop using bandit-baseline
bandit-baseline finds *new* issues introduced in a commit, by comparing
results between two git commits. If the git repository has uncommitted
changes, bandit-baseline refuses to run.

This switches over to using plain bandit instead of bandit-baseline,
and resolves or stifles existing bandit errors so we have a clean run.
These updates apply to bandit running as part of `tox -e pep8` or `tox
-e bandit`.

* Have bandit runs from tox ignore the octavia/tests directory
* Resolve several instances of `B701 jinja2_autoescape_false`
* Stifle several instances of `B303 md5`
* Resolve two instances of `B104 hardcoded_bind_all_interfaces`
* Stifle one instance of `B104 hardcoded_bind_all_interfaces` (see
https://bugs.launchpad.net/octavia/+bug/1489963)

Closes-Bug: #1621251
Related-Bug: #1489963

Change-Id: Iad3cbe5762949a6311bdd361b1f12c5a24c40633
2016-09-09 14:50:20 +00:00
Michael Johnson
7ba33e6ee4 Fixes failover flow with namespace driver
This patch updates the haproxy service scripts to handle the case
where the network interfaces have not yet been plugged.  This can
occur in a failover situation.
This patch also makes sure we don't move the management lan interface
into the network namespace.

Closes-Bug: #1509706
Closes-Bug: #1577963
Change-Id: I04d267bd3cdedca11f0350c5255086233cba14ec
2016-08-19 17:48:46 +00:00
Elena Ezhova
d73df70d85 Cleanup deleted load balancers in housekeeper's db_cleanup
When load balancer is deleted the corresponding DB entry is marked
as DELETED and is never actually removed along with a VIP
associated whit this load balancer.

This adds a new method to db_cleanup routine that scans the DB for
load balancers with DELETED provisioning_status and deletes them
from db if they are older than load_balancer_expiry_age. Corresponding
VIP entries are deleted in cascade.

Added new config option `load_balancer_expiry_age` to the `house_keeping`
config section.

Also changed the default value of exp_age argument to
CONF.house_keeping.amphora_expiry_age in check_amphora_expiry_age
method.

DocImpact
Closes-Bug #1573725

Change-Id: I4f99d38f44f218ac55a76ef062ed9ea401c0a02d
2016-07-07 03:03:03 +00:00
ptoohill1
6c000c2069 Amphora fails to build
Bug in devstack plugin caused network option to be corrupted
causing failures during amphora build.

Change-Id: I9585f22e3bb3a53ae70a5ddb8b76a3a930b10b73
Closes-Bug: #1574784
2016-04-25 13:42:25 -05:00
Jenkins
81fc37c2fe Merge "Update amp_network to allow multiple networks" 2016-04-21 22:19:45 +00:00
Michael Johnson
b89abe1871 Run amphora haproxy in a network namespace
In the current Octavia there is the possibility of an address
space conflict between the Octavia load balancer management
network and a tenant network.
This patch puts the haproxy processes inside the amphora into
a network namespace to provide isolation from the load balancer
management network.

A new file /var/lib/octavia/plugged_interfaces is created and
interfaces are writted to it on every plugVIP or plugNetwork call.
Interfaces in this file are created under the network namespace.

Change-Id: I75472885fe45226a5315867369eaef9b001a112b
Co-Authored-By: Bharath M <bharath.stacker@gmail.com>
Closes-Bug: #1458920
2016-04-16 00:24:16 +00:00
ptoohill1
4230e00a2c Update amp_network to allow multiple networks
Currently the amp_network is a stringOpt and in code it
translates that to a list for processing. It may
be required to deploy with multiple networks,
a listOpt for amp_network option allows this.

Change-Id: I4364c1d03d4c7b560f0d8030b7d66412583a31ae
2016-04-14 14:29:29 -05:00
minwang
c358e1b99b Remove the default anchor usrename and password value
Remove anchor's default value for username and  password
for the sake of a security perspective.

Closes-Bug: #1548555

Change-Id: I14f1b84f5161308fc23ef3776a796636ba61154d
2016-03-30 15:16:12 -07:00
Jenkins
c4acd4fd30 Merge "Add CA Cert file config option to validate against SSL endpoints" 2016-03-09 21:14:16 +00:00
Bharath M
f4da51c27d Add CA Cert file config option to validate against SSL endpoints
Currently Octavia cannot validate against SSL service endpoints,
which would be keystone, neutron, nova and glance in this case.

This patch adds a config option under nova, neutron and glance
sections to read the specified CA certificate files
for validation. It's slightly different in the case of glance,
because glance session method invocations depend on the endpoint
URL whether it starts with HTTP or HTTPS.

Also added is the "insecure" option for these services in case
the cert validation needs to be skipped.

For keystone, we read config params from keystone middleware. Thus,
instead of defining a new config option, we can make use of it's
pre-defined "cafile".

Barbican is not added because we do not yet have a barbican endpoint
override in it's config. This could be added in the future as a
separate patch, if needed.

Lastly, unrelated to the above, fixes the amphora REST api default
bind_port in octavia.conf

Change-Id: Id57672a3dc7c962b8ee07db0cb7a743041082c66
Closes-Bug: #1552987
2016-03-09 10:15:44 -08:00
Michael Johnson
a9fde42f6d Remove an unused configuration option
This patch removes an unused configuration option
"haproxy_cert_dir" that was flagged as a potential security
risk.

Change-Id: I31af43e8265431767544802451d9b5c297d83d28
Closes-Bug: #1548556
2016-03-09 01:28:15 +00:00
Jenkins
9fe1056aa7 Merge "glance: support relying on tags to extract image id" 2016-03-01 23:06:06 +00:00
Ihar Hrachyshka
fb53fe2340 glance: support relying on tags to extract image id
Deprecated amp_image_id option with the new amp_image_tag option.

Also switched devstack plugin to rely on the tag to update the image
used for new load balancers.

Implements: blueprint use-glance-tags-to-manage-image
Change-Id: Ibc28b2220565667e15ca2b2674e55074d6126ec3
2016-03-01 20:43:33 +01:00
Jenkins
667bb92381 Merge "Remove old SSH specific config options from sample config" 2016-02-28 01:06:03 +00:00
minwang
07a608f681 Implements: blueprint anti-affinity server group
https://blueprints.launchpad.net/octavia/+spec/anti-affinity
Added a new column in lb table for server group id;
Added a new task in compute tasks for creating server group;
Added a new task in dtabase tasks to update server
group id info for lb;
Add server group id in create method in nova driver to support
anti-affinity when creating compute instance

Change-Id: If0d3a9ba1012651937a2bda9bc95ab4f4c8852d5
2016-02-24 10:42:06 -08:00
Adam Harwell
42c12d7e8c Remove old SSH specific config options from sample config
Change-Id: I771ef200cd53d48abe08266ca2cb52aca711ec77
2016-02-24 11:36:00 -06:00
Michael Johnson
90c465ebb3 Add a request timeout to the REST API driver
The amphora REST driver does not have reasonable timeouts on the
python "requests" library calls. For example, the PUT call to
load a certificate into an amphora can hang forever.
This patch adds a request timeout that defaults to 5 seconds.

Change-Id: I75bed9fa1d590ce206aa0947d35552fc80907c0a
Closes-Bug: #1496634
2016-02-24 04:07:41 +00:00
Stephen Balukoff
2a0a0944bf Delete SSH amphora driver
The old SSH amphora driver is not being used by anyone
anymore, nor is it being maintained. This patch removes it from
the Octavia code tree.

Closes-Bug: 1534218
Change-Id: I006f1c794e1ab0483886d06495ca6649f0afe479
2016-02-17 13:51:11 -08:00
ptoohill1
025ec0024b Allow user-data on amphora creation
Currently, Amphora configuration data is being sent as personality
files as considered by Nova and some providers have limitations
and must use cloud-init user-data.

This patch introduces a new config option to enable user-data.
If enabled the files that were built, such as the amphora config
and certificates for the agent, will be templated into a cloud-init
user-data script that loads the files as expected. After this we
need to restart the agent as cloud-init happens at a higher level
than service scripts. This does increase the boot time.

This is configurable so there is no impact if it's not needed.

Change-Id: I60fa87722302eee9d3d1fd6ff1b5b5b697a2406e
Closes-Bug: #1541231
2016-02-09 16:34:16 -06:00
Jenkins
16719f4400 Merge "Adding "region and endpoint_type" parameters to barbican_acl.py" 2016-02-09 00:35:06 +00:00
Aishwarya Thangappa
c887461f61 Adding "region and endpoint_type" parameters to barbican_acl.py
When requesting for a barbican client, this change lets you filter based on
region and endpoint_type.

Conflicts:
	etc/octavia.conf

Change-Id: Ib4b9b75027443177c039f60f99822b9b3d021b8a
2016-02-08 13:53:18 -08:00