When purely using relation-set from a leader, updates after
the leader has changed can lead to old data being persisted
on a relation in addition to newer data being set by the new
leader. When this happens, there can be issues with services
using old data to talk to other related services.
This change introduces the use of the application data bag
to ensure that all units related to keystone get the same
data from the leader, regardless of leadership changes.
While this change enables the application data bag for these
relations, it still sends the per-unit relation data as well
to maintain backwards compatibility. Charms that consume the
identity-service and identity-notification relations will
need an update to use the application data bag to complete
this change.
Partial-Bug: #1902264
Change-Id: Iadd795fec605e7704e5a6673906452279bbecb34
The related bug indicated that the Fernet keys could get out of sync
between the leader and non-leader units. This patchset assumes that
hooks fail, or that units are off-line when the rotation occurs. Thus
it tries hard to ensure that the keys are in sync. It still uses juju
to 'send' the keys from the leader to the subordinate units, so in that
sense, it is not a fix to the related bug, but it does make it more
robust.
Change-Id: Id40a3ccbe565bd742e3fdbd5190deb6b21204a82
Related-Bug: #1849519
Sharing the admin password with peers over the 'cluster' relation
was needed in case the leader would die and the next leader would
then need that information. This was implemented years ago when
the leader DB didn't exist.
This led to a race condition described in the mentioned bug and
can now be safely removed.
Validated by deploying several keystone and glance units, then
removing the keystone leader, then adding a glance unit and
checking that this new unit gets its service credentials.
Also added useful traces, made linter happy and increased fernet
token expiration to avoid spurious test failures.
Closes-Bug: #1818113
Change-Id: I004903e50f51e190467d71691982de26518d7149
When the certificates relation is ready before the
HA relation is clustered, the VIP symlinks will not
be created pointing at the correct certificates. This
change updates the HA handlers to ensure that the
certificate relation is handled after clustering,
if there are any certificate relations.
Change-Id: Idfbdaf7919569983cdf159e44a6dad26eccfd195
Closes-Bug: #1886077
When Keystone tries to setup identity credentials before the
API is ready, there will be hook errors as the API service
is not available to communicate with. Adding in a gate on
API readiness before handing out credentials makes this safe.
Change-Id: I9f2700a391cfb664572a39e8db5b2d3e370cf170
Closes-Bug: #1886918
The checks that keystone was performing before emitting identity
data were applicable to any Openstack api charm so the check
definitions have been moved to charmhelpers so other charms can
use them. The checks as they were are encapsulated in
`check_api_unit_ready` *1
Bug 1818113 was caused by keystone emitting identity data
as soon as the leader was ready but ignoring the state of the
peer units. This is now covered by a new check
`check_api_application_ready` which performs all the local
unit checks and then checks that all peers have reported
as ready too.
In addition `check_api_unit_ready` is now used when
setting the units workload status and `check_api_application_ready`
is used when setting the application workload status.
*1 https://github.com/juju/charm-helpers/blob/master/charmhelpers/contrib/openstack/utils.py#L2289
*2 https://github.com/juju/charm-helpers/blob/master/charmhelpers/contrib/openstack/utils.py#L2330
Change-Id: I99830ab2c2482e8beef174424820907ce96fd5d7
Closes-Bug: #1818113
Keystone does not reliably pick up policy changes (observed on
Queens) unless apache is restarted. This change triggers a restart
when policy is changed.
Change-Id: Ia29312baa9c1b8113649fc2826f0464588c3ce11
Stop the use of the admin_token and use the bootstrap process
to initialize Keystone instead. Fortunately the implementation
of the bootstrap process is both idempotent when it needs to be
and it can be safely called on an existing deployment.
Subsequently we can migrate by just removing the admin_token
from the configuration and create new credentials for use by
the charm with a call to ``keystone-manage bootstrap``.
Remove configuration templates for versions prior to Mitaka, by
doing this we need to move any configuration initially defined
prior to Miataka forward to the ``templates/mitaka`` folder.
A side effect of this migration is that newly bootstrapped
deployments will get their ``default`` domain created with a
literal ID of ``default``. Prior to this change third party
software making assumptions about that being the case may have
had issues.
Closes-Bug: #1859844
Closes-Bug: #1837113
Related-Bug: #1774733
Closes-Bug: #1648719
Closes-Bug: #1578678
Func-Test-Pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/191
Change-Id: I23940720c24527ee34149f035c3bdf9ff54812c9
When a remote service requests notification about changes to
endpoints, provide the full detail on each endpoint rather than
just a checksum of the internal/admin/public URL's.
This allows consuming services which require explicit configuration
of service endpoint URL's to configure everything via their
relation to keystone rather than directly relating to all required
services.
Change-Id: I39b6e3df17e44c801f5f6bb122407623cbf1c937
This feature adds a "password-security-compliance" option to the
charm to enable setting of keys in the "[security_compliance]" section
of the keystone.conf file. This section was added in the Newton
release, and so this feature supports this from the Newton release.
It also protects the service accounts from two of the PCI-DSS options
but setting the user options 'ignore_password_expiry' and
'ignore_change_password_upon_first_use' to True to prevent the cloud
from being broken.
Change-Id: If7c54fae73188284bd9b03a53626cdf52158b994
Closes-Bug: #1776688
If the database is in maintenace mode do not attempt to access
it.
Depends-On: I5d8ed7d3935db5568c50f8d585e37a4d0cc6914f
Change-Id: I8cdb42364b7da03129bb8e2debebf6f6947d7ff3
Services that expose multiple endpoints use a prefix in their keys, this
patch refactors that code to put it in their own function to be reused
by the notifications functionality and make it notificate for changes in
those endpoints (e.g. neutron-api and nova-cloud-controller).
Change-Id: Ieecfc4ef7c85c7f716ceef0c2938ae0c7787953d
Closes-Bug: #1856419
This patchset implements policy overrides for keystone. It uses the
code in charmhelpers.
Closed-Bug: #1741723
Change-Id: I187f4493392178d87ef7dbd67de841bbeae0c65d
During series upgrade, the keystone packages get re-installed as the
underlying Linux has been upgraded and new package sets are updated and
then pulled in. For trusty->xenial this means that keystone.service
gets enabled which then breaks haproxy. On install, on xenial+, the
keystone.service is disabled in the install hook. This just replicates
this in the series-upgrade hook.
Change-Id: Ic5ed6cf354d5545b9e554e205a048955a381e0f5
Closed-Bug: #1839021
When the certificates endpoint has completed TLS configuration
via Vault, ensure that any federated identity backends are
updated for the switch to TLS, other the generated SP data
incorrectly used http:// instead of https://
Closes-Bug: 1834442
Change-Id: Ie160095789f5c34bc3509ffce4a7c5c0ec430632
When running an upgrade of the keystone charm in a large environment,
the upgrade process can take more than 30 minutes.
Prior to this change, the user would only see 'Installing apt packages'
during most of the process.
This change adds status_set and log messages to the upgrade charm hook
so that end users can see what is the status of the upgrade.
Change-Id: I1eade73fad551310ceec0bb8dcbccb777737faeb
Closes-Bug: #1782004
This patch ensures that any running manager.py script is stopped if the
keystone service (either standalone, or run via apache2) is restarted.
The ManagerServer will automatically (re)start the manager.py script
when it is needed.
Change-Id: I4e3dff340cce73b2de7d697f3685d602defb5179
Related-Bug: #1823743
When ``certificates-relation-changed`` hook is called before the
certificate data is present on the relation do not attempt to
configure apache.
Change-Id: If915451d4b0846023355edcf3a49f643e12c7522
Closes-Bug: #1822952
Before this change there was a race condition in which keystone would
hand out its local IP rather than the VIP when in HA mode and never
update clients other than on the identity-service relation.
This change guarantees the other relations are also updated with HA VIP
information.
Change-Id: Ib2ea3103c06a7a06bb1d04b6c7e872f36d12c1c4
The charm assumed the use of ssl_cert and ssl_key. The current best
practice is to deploy with vault and the tls-certificates relation.
Enable tls-certificates relation aware configuration for the
fid-service-provider relation.
Change-Id: I5441359b1d60b07c6b47ca58b45a09c4b7ac886b
The keystone charm sets db-initialised to true after initializing the
database the first time. However, if the database application is
removed, this flag is not unset.
This results in breakage on attempts to re-add a shared-db relation
with a new database application, as the charm will not attempt to
re-initialize the database prior to doing DB operations.
This fix simply ensures that we unset this flag prior to finalizing
removal of the shared-db relation.
Change-Id: I78ae12fda05ce006939b2d90a3d738bacb815915
Closes-Bug: #1797229
This patch add a new key/value parameter to
the middleware relation to notify subordinate middleware
charm with actual keystone release version
Change-Id: Ie53355946e895cefe813c9e539fc904949f9df0c
Closes-Bug: #1808597
This patch creates a new middleware context to retrieve data
from subordinate charm and update the kesytone configuration file.
It also allows integration with keystone-middleware interface:
https://github.com/openstack-charmers/interface-keystone-middleware
This patch uses the subordinate configuration approach
to retrieve data from the subordinate charm.
Every changes required for paste.ini file will be handled by
the subordinate charm. The latter should deal with keystone upgrades.
Closes-Bug: #1808597
Change-Id: I4897011fbc791abc97e34e75826579820e80a4f1
Use the generate_ha_relation_data helper from charmhelpers to
generate the data to send down the relation to the hacluster
charm.
This results in a few changes in behaviour:
1) The charm will no longer specify a nic name to bind the vip. This
is because Pacemaker VIP resources are able to automatically
detect and configure correct iface and netmask parameters based
on local configuration of the unit.
2) The original iface named VIP resource will be stopped and deleted
prior to the creation of the new short hash named VIP resource.
Change-Id: I906e96ad8cbcf2ca2d1cdbfd091070c21427214c
On charm upgrade the charm may switch to py3 packages. If so, ensure
the old py2 packages are purged after any new packages have been
installed. If packages where removed during the upgrade, then
restart all services to pickup any py2->py3 switches.
Change-Id: I8a17f679e85448680ab0b1f5ae362ef7a65b035c
Closes-Bug: 1803451
When keystone recieves certificates down the certificates relation
it needs to update the identity endpoint to https.
Change-Id: I9a423096b77a73f78a6cc9e3d250bdea2fb861b6
At present the Keystone charm frequently initiates updates to its
relations before it has reached a stable state.
Make use information from ``juju goal-state`` to predict scale and gate
initial update of relations on having reached expected scale.
Depends-On: https://github.com/juju/charm-helpers/pull/226
Change-Id: I96d4aff7c4ec9fb9ea160c7e294581bab3103df8
Major changes:
* decoupling the hooks/manager.py file from the charm. It is now a
script that is called from hooks/keystone_utils.py as it has to use
the same Python version/libraries as the installed keystone payload
software. keystone_utils.py and manager.py communicate via a Unix
Domain Socket using json, encoded to base64.
* As Python3 requires absolute imports, the charmhelpers symlink has
been removed from hooks, and the hooks and charmhelpers symlinks have
been removed from the actions directory. Instead, the path is
adjusted so that the modules can be found.
Change-Id: I18996e15d2d08b1dacf0533132eae880cbb9aa32
This patchset adds more Fernet token implementation:
1. Adds a cron job to rotate / sync keys to other units.
2. Adds additional tests around gating on config.
3. Adds rotation / syncing with more robust key handling.
Change-Id: Ied021ad83c241f241dbb5f9acdede9045e43a8a3
There are no relation-level concerns for this flag.
Any pre-existing deployments with recent charms (as in released
2015 or later) will have already migrated the peer storage to
leader storage, so this change can build on that work directly.
Ref: https://github.com/juju/charm-helpers/blame/master/charmhelpers/contrib/peerstorage/__init__.py
Change-Id: Ia7362d257428b102c452d0e91bcf9b3378a1e6e9
Closes-bug: #1783747
The `keystone-common` package no longer exist and using it for
OpenStack release comparison will not produce expected results.
Change-Id: I263c408679759971265d721cdd66ae9ea7d481d8
Starting OpenStack Rocky the currently used `uuid` token format
is no longer supported and we need to change to use `fernet` tokens.
This change provides basic functionalty to initialize fernet token
repository and distribute keys to non-leader units.
A configuration option is also added allowing change of token format
in a controlled manner prior to upgrading to OpenStack Rocky.
Further work is required to implement key rotation, actions etc. and
these topics will be addressed in separate commits.
The commit also fixes a instance of missing release check for writing
of `policy.json`, and a few places where writing of `policy.json`
previously was omitted.
Change-Id: I1d0ff22a5f091b02f5700412745572c246103e9e
Commit 855c954c305ac3a25d1cea1c7638c313ee698b38 revealed a bug
that occurrs some times when having multpile Keystone units.
Change-Id: I7bfce6db6b0dba855af32aef4c3f496f03a5a34b
Related-Bug: #1637453
On every call to update_all_identity_relation_units()
apache was reconfigured and all configuration files written.
This was used for the legacy management of self signed
certificates which now has been removed.
Change-Id: I7480575bc06287f6898ce469b420fd63206015e2
These features are disabled by default, a majority of our
users provide certificates through configuration.
At present the cluster relation carries information required
for these features even when they are not enabled. This makes
processing of cluster relation changes unnecessarily heavy
and vulnerable to bugs.
Notice of deprecation and removal in next release was given
as part of the 18.05 release notes.
Change-Id: I8b07c7e0d5c2c623c115c83dc8aff230b554a986
Closes-Bug: #1755897
Related-Bug: #1744990
* add support for relating with subordinate charms providing Service
Provider functionality via apache2 authentication modules;
* enable additional authentication methods on the keystone side to
accept parsed assertion data provided via apache2 authentication module
variables exported to WSGI environment;
* move https frontend and WSGI API apache config files to keystone
instead of relying on charm-helpers as modifications are needed there to
add IncludeOptional directives. openstack_https_frontend.conf is added
on purpose as ServerName cannot be correctly determined after ProxyPass
which results in TLS errors during SAML exchange process;
* add an additional relation to openstack-dashboard to provide URL
information necessary to trust 'origin' parameter in WebSSO URLs used by
horizon during the authentication process. Also add a context to render
the federation section that is used to render this information in
keystone.conf;
Subordinates can choose to use different apache2 authentication modules.
If those modules support vhost-level variables then multiple
subordinates for the same module can be used. For example,
mod_auth_mellon can be used multiple times in different vhosts to
protect federated token endpoints related to different identity provider
and protocol combinations).
Trusted dashboard relation could be used to provide dashboard origin URL
from a different site via cross-model relations.
NOTE: this functionality will be triggered only on Ocata+ (inclusive)
Change-Id: I1ef623b0b0e2a9f68cec4be550965c5e15e5f561
When keystone is deployed with multiple units but without hacluster one
off scenarios occur where one non-leader unit will fail to update its
client relations.
This change runs all identity client relations when the database
relation is complete thus guaranteeing all keystone units update there
identity relation data with clients.
Small timing fix to amulet tests.
Closes-Bug: #1761562
Change-Id: I338e500dbc155b75c75b9261a9b5b471bd73088a
