74 Commits

Author SHA1 Message Date
Elena Ezhova
d73df70d85 Cleanup deleted load balancers in housekeeper's db_cleanup
When load balancer is deleted the corresponding DB entry is marked
as DELETED and is never actually removed along with a VIP
associated whit this load balancer.

This adds a new method to db_cleanup routine that scans the DB for
load balancers with DELETED provisioning_status and deletes them
from db if they are older than load_balancer_expiry_age. Corresponding
VIP entries are deleted in cascade.

Added new config option `load_balancer_expiry_age` to the `house_keeping`
config section.

Also changed the default value of exp_age argument to
CONF.house_keeping.amphora_expiry_age in check_amphora_expiry_age
method.

DocImpact
Closes-Bug #1573725

Change-Id: I4f99d38f44f218ac55a76ef062ed9ea401c0a02d
2016-07-07 03:03:03 +00:00
chen-li
a345559d9b Disable dhclient requests the default route info
Closes-Bug: #1555235

Change-Id: I7a7e3a9361f7ce45f92ff52e98c547026f44cc0c
2016-06-08 00:14:16 -05:00
ptoohill1
6c000c2069 Amphora fails to build
Bug in devstack plugin caused network option to be corrupted
causing failures during amphora build.

Change-Id: I9585f22e3bb3a53ae70a5ddb8b76a3a930b10b73
Closes-Bug: #1574784
2016-04-25 13:42:25 -05:00
Jenkins
81fc37c2fe Merge "Update amp_network to allow multiple networks" 2016-04-21 22:19:45 +00:00
Michael Johnson
b89abe1871 Run amphora haproxy in a network namespace
In the current Octavia there is the possibility of an address
space conflict between the Octavia load balancer management
network and a tenant network.
This patch puts the haproxy processes inside the amphora into
a network namespace to provide isolation from the load balancer
management network.

A new file /var/lib/octavia/plugged_interfaces is created and
interfaces are writted to it on every plugVIP or plugNetwork call.
Interfaces in this file are created under the network namespace.

Change-Id: I75472885fe45226a5315867369eaef9b001a112b
Co-Authored-By: Bharath M <bharath.stacker@gmail.com>
Closes-Bug: #1458920
2016-04-16 00:24:16 +00:00
ptoohill1
4230e00a2c Update amp_network to allow multiple networks
Currently the amp_network is a stringOpt and in code it
translates that to a list for processing. It may
be required to deploy with multiple networks,
a listOpt for amp_network option allows this.

Change-Id: I4364c1d03d4c7b560f0d8030b7d66412583a31ae
2016-04-14 14:29:29 -05:00
minwang
c358e1b99b Remove the default anchor usrename and password value
Remove anchor's default value for username and  password
for the sake of a security perspective.

Closes-Bug: #1548555

Change-Id: I14f1b84f5161308fc23ef3776a796636ba61154d
2016-03-30 15:16:12 -07:00
Jenkins
c4acd4fd30 Merge "Add CA Cert file config option to validate against SSL endpoints" 2016-03-09 21:14:16 +00:00
Bharath M
f4da51c27d Add CA Cert file config option to validate against SSL endpoints
Currently Octavia cannot validate against SSL service endpoints,
which would be keystone, neutron, nova and glance in this case.

This patch adds a config option under nova, neutron and glance
sections to read the specified CA certificate files
for validation. It's slightly different in the case of glance,
because glance session method invocations depend on the endpoint
URL whether it starts with HTTP or HTTPS.

Also added is the "insecure" option for these services in case
the cert validation needs to be skipped.

For keystone, we read config params from keystone middleware. Thus,
instead of defining a new config option, we can make use of it's
pre-defined "cafile".

Barbican is not added because we do not yet have a barbican endpoint
override in it's config. This could be added in the future as a
separate patch, if needed.

Lastly, unrelated to the above, fixes the amphora REST api default
bind_port in octavia.conf

Change-Id: Id57672a3dc7c962b8ee07db0cb7a743041082c66
Closes-Bug: #1552987
2016-03-09 10:15:44 -08:00
Michael Johnson
a9fde42f6d Remove an unused configuration option
This patch removes an unused configuration option
"haproxy_cert_dir" that was flagged as a potential security
risk.

Change-Id: I31af43e8265431767544802451d9b5c297d83d28
Closes-Bug: #1548556
2016-03-09 01:28:15 +00:00
Jenkins
9fe1056aa7 Merge "glance: support relying on tags to extract image id" 2016-03-01 23:06:06 +00:00
Ihar Hrachyshka
fb53fe2340 glance: support relying on tags to extract image id
Deprecated amp_image_id option with the new amp_image_tag option.

Also switched devstack plugin to rely on the tag to update the image
used for new load balancers.

Implements: blueprint use-glance-tags-to-manage-image
Change-Id: Ibc28b2220565667e15ca2b2674e55074d6126ec3
2016-03-01 20:43:33 +01:00
Jenkins
667bb92381 Merge "Remove old SSH specific config options from sample config" 2016-02-28 01:06:03 +00:00
minwang
07a608f681 Implements: blueprint anti-affinity server group
https://blueprints.launchpad.net/octavia/+spec/anti-affinity
Added a new column in lb table for server group id;
Added a new task in compute tasks for creating server group;
Added a new task in dtabase tasks to update server
group id info for lb;
Add server group id in create method in nova driver to support
anti-affinity when creating compute instance

Change-Id: If0d3a9ba1012651937a2bda9bc95ab4f4c8852d5
2016-02-24 10:42:06 -08:00
Adam Harwell
42c12d7e8c Remove old SSH specific config options from sample config
Change-Id: I771ef200cd53d48abe08266ca2cb52aca711ec77
2016-02-24 11:36:00 -06:00
Michael Johnson
90c465ebb3 Add a request timeout to the REST API driver
The amphora REST driver does not have reasonable timeouts on the
python "requests" library calls. For example, the PUT call to
load a certificate into an amphora can hang forever.
This patch adds a request timeout that defaults to 5 seconds.

Change-Id: I75bed9fa1d590ce206aa0947d35552fc80907c0a
Closes-Bug: #1496634
2016-02-24 04:07:41 +00:00
Stephen Balukoff
2a0a0944bf Delete SSH amphora driver
The old SSH amphora driver is not being used by anyone
anymore, nor is it being maintained. This patch removes it from
the Octavia code tree.

Closes-Bug: 1534218
Change-Id: I006f1c794e1ab0483886d06495ca6649f0afe479
2016-02-17 13:51:11 -08:00
ptoohill1
025ec0024b Allow user-data on amphora creation
Currently, Amphora configuration data is being sent as personality
files as considered by Nova and some providers have limitations
and must use cloud-init user-data.

This patch introduces a new config option to enable user-data.
If enabled the files that were built, such as the amphora config
and certificates for the agent, will be templated into a cloud-init
user-data script that loads the files as expected. After this we
need to restart the agent as cloud-init happens at a higher level
than service scripts. This does increase the boot time.

This is configurable so there is no impact if it's not needed.

Change-Id: I60fa87722302eee9d3d1fd6ff1b5b5b697a2406e
Closes-Bug: #1541231
2016-02-09 16:34:16 -06:00
Jenkins
16719f4400 Merge "Adding "region and endpoint_type" parameters to barbican_acl.py" 2016-02-09 00:35:06 +00:00
Aishwarya Thangappa
c887461f61 Adding "region and endpoint_type" parameters to barbican_acl.py
When requesting for a barbican client, this change lets you filter based on
region and endpoint_type.

Conflicts:
	etc/octavia.conf

Change-Id: Ib4b9b75027443177c039f60f99822b9b3d021b8a
2016-02-08 13:53:18 -08:00
German Eichberger
22097dc88f Adds a parameter to specify endpoint type
When getting a service catalog from keystone there might be
multiple endpoint types. Adds new parameters to specify
the endpoint type to use in each of the new neutron/nova
groups.

Nova config attributes are removed as the endpoint would be retrieved
from the service catalog with respective endpoint_type and corresponding
roles would define nova access.

CONFIG is added as needed and cfg being removed.

Change-Id: Ie01bd6967eb2003dbe4f7a11ffe8e20a16aa83f5
2016-02-08 12:45:55 -08:00
Jenkins
25bcbc1cc1 Merge "Add endpoint override conf option for neutron and nova" 2016-02-06 09:52:58 +00:00
Jenkins
d1e31e266b Merge "Default api_handler to queue_producer" 2016-02-05 18:29:59 +00:00
Brandon Logan
be8fa11a6e Default api_handler to queue_producer
No reason to have the default be the simulated handler anymore.

Change-Id: I37a06c2afd14f442090a0ae1b36111699057f375
2016-02-05 02:24:05 -06:00
Jenkins
a73aae6117 Merge "Implementing EventStreamer" 2016-02-04 18:50:51 +00:00
Trevor Vardeman
240931610f Add endpoint override conf option for neutron and nova
In some implementations overriding the neutron and/or nova endpoint returned in
the keystone catalog will be necessary.  Adding this will allow that to happen.

Change-Id: I66a9ed82a895b9fe282ef7b2c4bdfb954af0cc0c
2016-02-03 16:18:16 -06:00
Brandon Logan
4c96356030 Add service name conf option for neutron and nova
This will be especially used in testing environments where there are multiple
nova and neutron services in the same region.

Change-Id: I4ff2ce0143d66f857629e3220952552a3d0fc632
Closes-Bug: 1539440
2016-01-29 01:44:22 -06:00
Carlos D. Garza
c84021ac27 Implementing EventStreamer
EvenStream will be used to serialize messages from the octavia
database to neutron-lbaas database via oslo_messaging. Also
renaming update mixin class since its not really a mixin. The
health manager will make changes to the octavia database when
members etc are marked as down and up etc which would result
in databases that were not in sync between neutron-lbaas and
octavia. A mechanism to communicate database changes from
octavia back to neutron is required so this CR attempts
to use a oslo_messaging system to communicate those changes,

Docimpact - /etc/octavia.conf the user can set the option
            event_streamer_driver = neutron_event_streamer
            to setup a queue to connect to neutron-lbaas.
            if this option is left blank it will default to
            the noop_event_streamer which will do nothing
            effectively turning the Queue off.

Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Change-Id: I77a049dcc21e3ee6287e661e82365ab7b9c44562
2016-01-19 00:48:14 -06:00
Adam Harwell
9fc9eeaf03 Make Barbican the default certificate manager
Local certificate manager is broken and should probably be removed,
since the effort was never taken to make it functional (adding
additional fields for taking raw certificate data in the API).
Barbican should be the default as it is the functional option.

Also taking this opportunity to clean up the config and local.conf
defaults to match reality.

Change-Id: Icf1e89838b761e7d2926c8eb7532404b4c2f19dd
2016-01-14 16:19:54 -06:00
Jenkins
a3b251d72b Merge "Refactor BarbicanAuth to allow for configurable auth method" 2015-12-10 06:47:20 +00:00
Adam Harwell
52351a5698 Refactor BarbicanAuth to allow for configurable auth method
There is now a new configuration option "barbican_auth" in the
certificates section, to specify which auth plugin to use when
communicating with Barbican. This is because the default option (using
ACLs inside Barbican to control access) should be ok as a default
workflow, but it might be required to use other methods depending on
your deployment. For example, another possible auth method would be
BarbicanTrustAuth, utilizing Keystone Trusts.

Some deployers may need custom auth methods that do not exist in
upstream Keystone, and will need their own Auth plugin. This should be in line
with the way Octavia's network and compute drivers work already.

While we're in this file, prune the unused (and really bad) method that
would *actually* delete certs from Barbican (not in our scope).
Also do the tenant_id -> project_id rename.

Change-Id: Ic9aef68924bb5c216734afd25403e59476c576e7
2015-12-08 14:56:32 -06:00
ptoohill1
8e31a1d044 Allow choice of upstart or sysvinit
upstart was hardcoded as the means of start, stopping and reloading
of haproxy. Allow for sysvinit scripts and paths to handle haproxy.

This patch provides a configuration option to switch
between the defaultl upstart init scripts or sysvinit.

Change-Id: I9efe51c5a08d8e2268150d69ac25725c708dfb8e
2015-12-08 00:06:45 -06:00
Jenkins
86b4f8dcec Merge "Amphora Flows and Drivers for Active Standby" 2015-12-04 16:26:08 +00:00
Sherif Abdelwahab
58cda714ba Amphora Flows and Drivers for Active Standby
This patch implements the Active/Standby blueprint in
https://blueprints.launchpad.net/octavia/+spec/activepassiveamphora

The following points describe the main changes:

1. The patch introduces new flows and subflows to create M amphorae. The
controller worker parses the loadbalancer_topology configuration. If the
loadbalancer_topology value is ACTIVE_STANDBY, the controller invokes a new flow
independent from the SINGLE topology case, which is left untouched. The new
flow uses conditional taskflows to check for spare amphorae at runtime. This
removes the need for the exception workaround we earlier had. The controller
creates the amphorae in parallel using an unordered flow. A new database task
alter an amphora role as either MASTER or BACKUP and assigns a VRRP priority to
each amphora. After the amphorae are created, the controller invokes a separate
flow for post amphora configuration including plug_vip methods, vrrp
configuration upload, and keepalived service start.

2. The patch introduces new data models that include a new table for VRRP group
configuration per loadbalancer, and update the amphora, loadbalancer, and
listener tables to support the new active/standby capability. The VRRPGroup
table hides authentication data, and makes future extensions of VRRP
capabilities easy.

3. This patch updates the existing Haproxy configuration templates  to include
peer synchronization. In case of ACTIVE_STANDBY configuration, the jinja
configuration renders the peer section in the Haproxy configuration and assigns
short names to the amphorae as listener peers. As listeners implies different
Haproxy process, each listener synchronizes on a different port evaluated as
BASE_PORT (1024) + NUMBER_OF_LISTENERS accounting for ports in use.

4. This patch introduces a new Jinja configuration templater and a REST driver
for Keepalived (developed as a Mixin). By default, Keepalived runs "all" check
scripts found in a predefined directory. The keepalived driver is a Mixin that
can be plugged in other services' drivers. It is the responsibility of these
services drivers to introduce their own check scripts. In this patch a
lightweight check script for Haproxy was introduced along with changes in the
amphora agent installation script.

5. The VRRP requires enabling protocol 112 for Master/Backup advertisements,
and enabling protocol 51 for authentication header. This patch enables these
protocols as needed in the loadbalancer security group.

Note: Updates to the failover flow to support active/standby will come in
a dependent patch.
Note: The amphora-agent is pinned to this patch in this patch set.  This
is required so the scenario tests will pass.  It will be removed in a
follow up patch.

Co-Authored-By: Sherif Abdelwahab <sherif.abdelwahab@hp.com>
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Implements: blueprint activepassiveamphora
Depends-On: Ifdf20378b26cdd13e0a3ff87cec8990fe89c0661
Change-Id: Ic4e04594e114ba682088d68d5f1af3f8f376db83
2015-12-03 23:27:20 +00:00
bharath
d2072ae0ae Anchor support to Octavia
Use Anchor for certificate signing to make the octavia communication
more secure. Anchor Ref url: https://github.com/openstack/anchor

Co-Authored-By: bharath <bharath.stacker@gmail.com>
Co-Authored-By: German Eichberger <german.eichberger@hp.com>

Change-Id: Id77b2b1540377db661f15d4eeafc4922f446d987
2015-12-03 10:06:04 -08:00
Brandon Logan
21058fae59 REST Amp Agent: Handle interfaces file too
The amphora agent should be able to add interfaces by using the
/etc/network/interfaces file as well as the /etc/network/interfaces.d/*.cfg
files.

Closes-Bug: #1507889
Change-Id: I7840931fc426a0c74386512dfae3666d223049f8
2015-11-14 02:31:14 +09:00
ptoohill1
0d0853f4b9 Make amphora ssh access configurable
It some instances it is required to not allow access to the Amphora.
In order to solve both cases make the option configurable.

DocImpact
Closes-Bug: #1515078

Change-Id: I8a04ca86e4111c44a0bf44c8993b87a7acc334a2
2015-11-11 19:26:49 -06:00
German Eichberger
6ff8907f2d Adds cert_manager option to octavia.conf
Adds the cert manager option the conf file to make it easier
for users to enable Barbican for TLS termination

Change-Id: I38b9fa9b74fd3fc2046dc7e2eb403932e4573479
2015-10-19 14:23:37 -07:00
ptoohill1
811f7a594e Adding init.d script
For booting on systems where upstart isnt used, such as
Debian, an init.d script can be used.

Change-Id: I2fadc32dc73f31d733c424591d64448059de89e3
Closes-Bug: #1502542
2015-10-04 16:21:06 -05:00
Michael Johnson
833d633975 Increase the number of network call retries
On hosts without nested virtualization enabled, network calls
can timeout.  This triples the number of retries Octavia attempts
for networking calls.

Change-Id: I31ff9cbbdb073d0ee92971894fac86a2245c93f8
2015-09-21 19:13:13 +00:00
ptoohill1
2494c1f920 Make keystone verify configurable
In some cases, such as test environements, it is required
to run with an insecure keystone connection. Allow for a
configuration option to set the verify flag.

Change-Id: I4b0658322b5d737b116d5659551fb2fe4f9e68a1
2015-09-17 11:30:51 -05:00
Brandon Logan
d14ebc2cdf Fix issues uncovered by neutron lbaas tempest tests
Updating a listener caused issues because SNI was not being appropriately
handles on a listener update.  This has been "fixed" but another patch will be
needed to make sure that every listener update does not clear out the list. I
think it is best to leave that for another review though.

This also fixes the bug where security groups were not getting cleaned up when
a load balancer is deleted.  Since neutron does not synchronously remove a
security group from a port, a retry mechanism had to be used and thus more
config options to tune the interval and max attempts.

Change-Id: I0434b8ced144ab08413b91569bd008295ef1784e
Closes-Bug: #1464953
2015-09-15 18:02:40 -05:00
Jenkins
9cf4fe4b75 Merge "Implement UDP heartbeat sender and receiver" 2015-09-10 16:01:09 +00:00
Jenkins
9d1e5defd0 Merge "health manager service" 2015-09-10 02:49:59 +00:00
minwang
c511b2f195 Increase connection_max_retries to be 300
We also set request_poll_timeout to be 3000 in neutron.conf;
This can help to resolve the error for could not ssh to instance for octavia gate-setting.


This patch is needed by https://review.openstack.org/#/c/209675/


Change-Id: I90002a8913dd7db375ff5740a3f9cd5566b82054
2015-09-09 18:16:22 +00:00
Carlos D. Garza
ccd7865350 Implement UDP heartbeat sender and receiver
Used binary compressed encoding of json dumped object. To reduce
the size needed to send heart beats incase some stats objects
start getting sent later on. Also used sha256 instead of sha1
with hmac.

Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Co-Authored-By: German Eichberger <german.eichbeger@hp.com>
Co-Authored-By: Carlos Garza <carlos.garza@rackspace.com>
Partially implements: health-manager
Change-Id: I932c693101b94c9132e1741291610508876eab43
2015-09-09 06:48:24 +00:00
minwang
f849f55e5e health manager service
This model is used to check amphora health
Add a column 'busy' and primary key  for data table amphora health
Add mutiprocessing code in cmd/health_manager, one for health check, the other is for UDP pacakge listening,

Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Co-Authored-By: min wang <swiftwangster@gmail.com>
Implements: blueprint health-manager
Change-Id: I8aeb6b82b58b59951a414e7c2e4c2c58c33a5d15
2015-09-09 06:32:19 +00:00
Jenkins
72dbe50126 Merge "Added support of multi-region environment" 2015-09-02 21:52:10 +00:00
Jenkins
11d682870b Merge "Increase the amp active retries to be 500 times" 2015-08-29 18:28:37 +00:00
minwang
1271d5653f Increase the amp active retries to be 500 times
By increasing te times of amphora active retries,we are targeting for octavia gate setting to work.

Change-Id: If81cc13966a0a24368225b89c9678f6b0fdc1a7b
2015-08-28 13:41:52 -07:00