When load balancer is deleted the corresponding DB entry is marked
as DELETED and is never actually removed along with a VIP
associated whit this load balancer.
This adds a new method to db_cleanup routine that scans the DB for
load balancers with DELETED provisioning_status and deletes them
from db if they are older than load_balancer_expiry_age. Corresponding
VIP entries are deleted in cascade.
Added new config option `load_balancer_expiry_age` to the `house_keeping`
config section.
Also changed the default value of exp_age argument to
CONF.house_keeping.amphora_expiry_age in check_amphora_expiry_age
method.
DocImpact
Closes-Bug #1573725
Change-Id: I4f99d38f44f218ac55a76ef062ed9ea401c0a02d
Bug in devstack plugin caused network option to be corrupted
causing failures during amphora build.
Change-Id: I9585f22e3bb3a53ae70a5ddb8b76a3a930b10b73
Closes-Bug: #1574784
In the current Octavia there is the possibility of an address
space conflict between the Octavia load balancer management
network and a tenant network.
This patch puts the haproxy processes inside the amphora into
a network namespace to provide isolation from the load balancer
management network.
A new file /var/lib/octavia/plugged_interfaces is created and
interfaces are writted to it on every plugVIP or plugNetwork call.
Interfaces in this file are created under the network namespace.
Change-Id: I75472885fe45226a5315867369eaef9b001a112b
Co-Authored-By: Bharath M <bharath.stacker@gmail.com>
Closes-Bug: #1458920
Currently the amp_network is a stringOpt and in code it
translates that to a list for processing. It may
be required to deploy with multiple networks,
a listOpt for amp_network option allows this.
Change-Id: I4364c1d03d4c7b560f0d8030b7d66412583a31ae
Remove anchor's default value for username and password
for the sake of a security perspective.
Closes-Bug: #1548555
Change-Id: I14f1b84f5161308fc23ef3776a796636ba61154d
Currently Octavia cannot validate against SSL service endpoints,
which would be keystone, neutron, nova and glance in this case.
This patch adds a config option under nova, neutron and glance
sections to read the specified CA certificate files
for validation. It's slightly different in the case of glance,
because glance session method invocations depend on the endpoint
URL whether it starts with HTTP or HTTPS.
Also added is the "insecure" option for these services in case
the cert validation needs to be skipped.
For keystone, we read config params from keystone middleware. Thus,
instead of defining a new config option, we can make use of it's
pre-defined "cafile".
Barbican is not added because we do not yet have a barbican endpoint
override in it's config. This could be added in the future as a
separate patch, if needed.
Lastly, unrelated to the above, fixes the amphora REST api default
bind_port in octavia.conf
Change-Id: Id57672a3dc7c962b8ee07db0cb7a743041082c66
Closes-Bug: #1552987
This patch removes an unused configuration option
"haproxy_cert_dir" that was flagged as a potential security
risk.
Change-Id: I31af43e8265431767544802451d9b5c297d83d28
Closes-Bug: #1548556
Deprecated amp_image_id option with the new amp_image_tag option.
Also switched devstack plugin to rely on the tag to update the image
used for new load balancers.
Implements: blueprint use-glance-tags-to-manage-image
Change-Id: Ibc28b2220565667e15ca2b2674e55074d6126ec3
https://blueprints.launchpad.net/octavia/+spec/anti-affinity
Added a new column in lb table for server group id;
Added a new task in compute tasks for creating server group;
Added a new task in dtabase tasks to update server
group id info for lb;
Add server group id in create method in nova driver to support
anti-affinity when creating compute instance
Change-Id: If0d3a9ba1012651937a2bda9bc95ab4f4c8852d5
The amphora REST driver does not have reasonable timeouts on the
python "requests" library calls. For example, the PUT call to
load a certificate into an amphora can hang forever.
This patch adds a request timeout that defaults to 5 seconds.
Change-Id: I75bed9fa1d590ce206aa0947d35552fc80907c0a
Closes-Bug: #1496634
The old SSH amphora driver is not being used by anyone
anymore, nor is it being maintained. This patch removes it from
the Octavia code tree.
Closes-Bug: 1534218
Change-Id: I006f1c794e1ab0483886d06495ca6649f0afe479
Currently, Amphora configuration data is being sent as personality
files as considered by Nova and some providers have limitations
and must use cloud-init user-data.
This patch introduces a new config option to enable user-data.
If enabled the files that were built, such as the amphora config
and certificates for the agent, will be templated into a cloud-init
user-data script that loads the files as expected. After this we
need to restart the agent as cloud-init happens at a higher level
than service scripts. This does increase the boot time.
This is configurable so there is no impact if it's not needed.
Change-Id: I60fa87722302eee9d3d1fd6ff1b5b5b697a2406e
Closes-Bug: #1541231
When requesting for a barbican client, this change lets you filter based on
region and endpoint_type.
Conflicts:
etc/octavia.conf
Change-Id: Ib4b9b75027443177c039f60f99822b9b3d021b8a
When getting a service catalog from keystone there might be
multiple endpoint types. Adds new parameters to specify
the endpoint type to use in each of the new neutron/nova
groups.
Nova config attributes are removed as the endpoint would be retrieved
from the service catalog with respective endpoint_type and corresponding
roles would define nova access.
CONFIG is added as needed and cfg being removed.
Change-Id: Ie01bd6967eb2003dbe4f7a11ffe8e20a16aa83f5
In some implementations overriding the neutron and/or nova endpoint returned in
the keystone catalog will be necessary. Adding this will allow that to happen.
Change-Id: I66a9ed82a895b9fe282ef7b2c4bdfb954af0cc0c
This will be especially used in testing environments where there are multiple
nova and neutron services in the same region.
Change-Id: I4ff2ce0143d66f857629e3220952552a3d0fc632
Closes-Bug: 1539440
EvenStream will be used to serialize messages from the octavia
database to neutron-lbaas database via oslo_messaging. Also
renaming update mixin class since its not really a mixin. The
health manager will make changes to the octavia database when
members etc are marked as down and up etc which would result
in databases that were not in sync between neutron-lbaas and
octavia. A mechanism to communicate database changes from
octavia back to neutron is required so this CR attempts
to use a oslo_messaging system to communicate those changes,
Docimpact - /etc/octavia.conf the user can set the option
event_streamer_driver = neutron_event_streamer
to setup a queue to connect to neutron-lbaas.
if this option is left blank it will default to
the noop_event_streamer which will do nothing
effectively turning the Queue off.
Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Change-Id: I77a049dcc21e3ee6287e661e82365ab7b9c44562
Local certificate manager is broken and should probably be removed,
since the effort was never taken to make it functional (adding
additional fields for taking raw certificate data in the API).
Barbican should be the default as it is the functional option.
Also taking this opportunity to clean up the config and local.conf
defaults to match reality.
Change-Id: Icf1e89838b761e7d2926c8eb7532404b4c2f19dd
There is now a new configuration option "barbican_auth" in the
certificates section, to specify which auth plugin to use when
communicating with Barbican. This is because the default option (using
ACLs inside Barbican to control access) should be ok as a default
workflow, but it might be required to use other methods depending on
your deployment. For example, another possible auth method would be
BarbicanTrustAuth, utilizing Keystone Trusts.
Some deployers may need custom auth methods that do not exist in
upstream Keystone, and will need their own Auth plugin. This should be in line
with the way Octavia's network and compute drivers work already.
While we're in this file, prune the unused (and really bad) method that
would *actually* delete certs from Barbican (not in our scope).
Also do the tenant_id -> project_id rename.
Change-Id: Ic9aef68924bb5c216734afd25403e59476c576e7
upstart was hardcoded as the means of start, stopping and reloading
of haproxy. Allow for sysvinit scripts and paths to handle haproxy.
This patch provides a configuration option to switch
between the defaultl upstart init scripts or sysvinit.
Change-Id: I9efe51c5a08d8e2268150d69ac25725c708dfb8e
This patch implements the Active/Standby blueprint in
https://blueprints.launchpad.net/octavia/+spec/activepassiveamphora
The following points describe the main changes:
1. The patch introduces new flows and subflows to create M amphorae. The
controller worker parses the loadbalancer_topology configuration. If the
loadbalancer_topology value is ACTIVE_STANDBY, the controller invokes a new flow
independent from the SINGLE topology case, which is left untouched. The new
flow uses conditional taskflows to check for spare amphorae at runtime. This
removes the need for the exception workaround we earlier had. The controller
creates the amphorae in parallel using an unordered flow. A new database task
alter an amphora role as either MASTER or BACKUP and assigns a VRRP priority to
each amphora. After the amphorae are created, the controller invokes a separate
flow for post amphora configuration including plug_vip methods, vrrp
configuration upload, and keepalived service start.
2. The patch introduces new data models that include a new table for VRRP group
configuration per loadbalancer, and update the amphora, loadbalancer, and
listener tables to support the new active/standby capability. The VRRPGroup
table hides authentication data, and makes future extensions of VRRP
capabilities easy.
3. This patch updates the existing Haproxy configuration templates to include
peer synchronization. In case of ACTIVE_STANDBY configuration, the jinja
configuration renders the peer section in the Haproxy configuration and assigns
short names to the amphorae as listener peers. As listeners implies different
Haproxy process, each listener synchronizes on a different port evaluated as
BASE_PORT (1024) + NUMBER_OF_LISTENERS accounting for ports in use.
4. This patch introduces a new Jinja configuration templater and a REST driver
for Keepalived (developed as a Mixin). By default, Keepalived runs "all" check
scripts found in a predefined directory. The keepalived driver is a Mixin that
can be plugged in other services' drivers. It is the responsibility of these
services drivers to introduce their own check scripts. In this patch a
lightweight check script for Haproxy was introduced along with changes in the
amphora agent installation script.
5. The VRRP requires enabling protocol 112 for Master/Backup advertisements,
and enabling protocol 51 for authentication header. This patch enables these
protocols as needed in the loadbalancer security group.
Note: Updates to the failover flow to support active/standby will come in
a dependent patch.
Note: The amphora-agent is pinned to this patch in this patch set. This
is required so the scenario tests will pass. It will be removed in a
follow up patch.
Co-Authored-By: Sherif Abdelwahab <sherif.abdelwahab@hp.com>
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Implements: blueprint activepassiveamphora
Depends-On: Ifdf20378b26cdd13e0a3ff87cec8990fe89c0661
Change-Id: Ic4e04594e114ba682088d68d5f1af3f8f376db83
Use Anchor for certificate signing to make the octavia communication
more secure. Anchor Ref url: https://github.com/openstack/anchor
Co-Authored-By: bharath <bharath.stacker@gmail.com>
Co-Authored-By: German Eichberger <german.eichberger@hp.com>
Change-Id: Id77b2b1540377db661f15d4eeafc4922f446d987
The amphora agent should be able to add interfaces by using the
/etc/network/interfaces file as well as the /etc/network/interfaces.d/*.cfg
files.
Closes-Bug: #1507889
Change-Id: I7840931fc426a0c74386512dfae3666d223049f8
It some instances it is required to not allow access to the Amphora.
In order to solve both cases make the option configurable.
DocImpact
Closes-Bug: #1515078
Change-Id: I8a04ca86e4111c44a0bf44c8993b87a7acc334a2
Adds the cert manager option the conf file to make it easier
for users to enable Barbican for TLS termination
Change-Id: I38b9fa9b74fd3fc2046dc7e2eb403932e4573479
For booting on systems where upstart isnt used, such as
Debian, an init.d script can be used.
Change-Id: I2fadc32dc73f31d733c424591d64448059de89e3
Closes-Bug: #1502542
On hosts without nested virtualization enabled, network calls
can timeout. This triples the number of retries Octavia attempts
for networking calls.
Change-Id: I31ff9cbbdb073d0ee92971894fac86a2245c93f8
In some cases, such as test environements, it is required
to run with an insecure keystone connection. Allow for a
configuration option to set the verify flag.
Change-Id: I4b0658322b5d737b116d5659551fb2fe4f9e68a1
Updating a listener caused issues because SNI was not being appropriately
handles on a listener update. This has been "fixed" but another patch will be
needed to make sure that every listener update does not clear out the list. I
think it is best to leave that for another review though.
This also fixes the bug where security groups were not getting cleaned up when
a load balancer is deleted. Since neutron does not synchronously remove a
security group from a port, a retry mechanism had to be used and thus more
config options to tune the interval and max attempts.
Change-Id: I0434b8ced144ab08413b91569bd008295ef1784e
Closes-Bug: #1464953
We also set request_poll_timeout to be 3000 in neutron.conf;
This can help to resolve the error for could not ssh to instance for octavia gate-setting.
This patch is needed by https://review.openstack.org/#/c/209675/
Change-Id: I90002a8913dd7db375ff5740a3f9cd5566b82054
Used binary compressed encoding of json dumped object. To reduce
the size needed to send heart beats incase some stats objects
start getting sent later on. Also used sha256 instead of sha1
with hmac.
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Co-Authored-By: German Eichberger <german.eichbeger@hp.com>
Co-Authored-By: Carlos Garza <carlos.garza@rackspace.com>
Partially implements: health-manager
Change-Id: I932c693101b94c9132e1741291610508876eab43
This model is used to check amphora health
Add a column 'busy' and primary key for data table amphora health
Add mutiprocessing code in cmd/health_manager, one for health check, the other is for UDP pacakge listening,
Co-Authored-By: Michael Johnson <johnsomor@gmail.com>
Co-Authored-By: min wang <swiftwangster@gmail.com>
Implements: blueprint health-manager
Change-Id: I8aeb6b82b58b59951a414e7c2e4c2c58c33a5d15
By increasing te times of amphora active retries,we are targeting for octavia gate setting to work.
Change-Id: If81cc13966a0a24368225b89c9678f6b0fdc1a7b