Commit Graph

124 Commits (master)

Author SHA1 Message Date
Takashi Kajinami 0f8158a399 Replace mocha by rspec-mocks
puppetlabs_spec_helper recommends rspec-mocks instead of mocha[1] and
it uses rspec-mocks by default instead of mocha since v 5.0.0[2]

This is the prep work to adapt to that migration.

[2] 493f0cbc1c

Closes-Bug: #2004135
Change-Id: Idfb4d8344af472aec0c6bbdd9b3600eedd17eb46
4 months ago
Takashi Kajinami aa394b1775 Remove keystone_puppet_config resource type
This is no longer used since we replaced the configuration file by
clouds.yaml. The resource was already deprecated by [1] in the past
cycle so can be removed.

[1] c140a44aeb

Change-Id: I631c0a14cc0ee5b56e7864980da9a651d6bf7d9b
7 months ago
Zuul 3c5f719ee6 Merge "Remove parse logic for openstacclient < 1.9.0" 10 months ago
Takashi Kajinami bae793888d Fix wrong test description
The value with a white space should be accepted instead of being
rejected. This change fixes the wrong test case description and makes
sure the description explains what is actually tested.

Change-Id: I51abf6af70438b418391146786eb217c148bfecb
11 months ago
Takashi Kajinami b8dfd90084 Remove parse logic for openstacclient < 1.9.0
... because that version is quite old and we don't expect it is still

This also updates the output string used in unit tests according to
the one in newer versions.

Change-Id: I18a8ced73197efe96964d760c3a97703f8d8b6b9
11 months ago
Tobias Urdin 9bb3834df4 Add description support to keystone_user
Change-Id: Ib5068b4e083e7673137e01564d52e203b379774e
1 year ago
Takashi Kajinami fc9cd07b6b keystone_user: Use un-scoped token to verify password
This change replaces project-scoped/domain-scoped token used to verify
password by un-scoped token. The current logic doesn't work if the user
has only system-scoped role. In addition, scope is not required to
generate a token thus un-scoped token is enough to validate a password.

Change-Id: I670f596ccb5e8b5543f4d7bb236ddaec1a2c961e
1 year ago
Takashi Kajinami c140a44aeb Use system scope credentials to request keystone
When SRBAC is enforced, Keystone allows only system admin to create
resources like user, role, role assignment and etc. With this change
now each provider uses system scope credential to create resources
like user, endpoint and etc.

This change also replaces /etc/keystone/puppet.conf by the yaml file
for openstackclient(/etc/openstack/puppet/admin-clouds.yaml)
This allows us to switch a system scope credential and a project
scope credential, and helps us implement a new provider which requires
project scope, in the future.

Change-Id: I27eb6b11df593581c94ef0affaf5abb8e333833b
1 year ago
Zuul c02a9e50ce Merge "Load libraries in a single place" 1 year ago
Takashi Kajinami ddfc0005b7 Load libraries in a single place
This change refactors how the dependent libraries are loaded during
unit tests, and load the libraries in the base spec_helper to avoid
duplicate and redundant implementations.

Change-Id: I59d8a5794a2f9c27ef231a22c38e507364e2e1a8
1 year ago
Takashi Kajinami 5457be773e Add support for system scope
This change is the initial work to support enforcing secure RBAC(SRBAC)
feature. The following two changes are made by this change.

- The keystone_user_role resource type now supports creating system
  roles in addition to project roles and domain roles. The following
  example shows how to assign the "admin" role to the "nova" user for
  the system scope "all".

    ensure => 'present',
    roles  => ['admin'],

- Some defined resource types were updated so that the other puppet
  modules can define keystone credentials for system scope access
  instead of project scope access.

Note that this change does not update the usage of project scope
credentials in each providers, and that should be fixed later to
enforce SRBAC completely.

Change-Id: Id43eeb31424f04d6969a993704e5a5c175eb1cb0
2 years ago
Takashi Kajinami 37f61f10e4 Fix unit test failure with recent puppetlabs-inifile
Change-Id: Idcef0958d6e68f94ce1690503456dbb1e86feac2
2 years ago
Takashi Kajinami 2b7a15eb91 Add support for MultiStrOpt
This replaces the provider implementation of keystone_config type so
that MultiStrOpt, which is used by several options like
 - oslo_messaging_notifications/driver
 - oslo_policy/policy_dirs
are handled correctly.

Change-Id: I84bb99e1ad746611d31f2e950614be73de4e8f0e
2 years ago
Thomas Goirand 4fc3eb01e5 Add support for keystone_uwsgi_config in Debian
This patch is adding the configuration of the number of workers,
threads, and the size of the listen queue in Debian, which uses
uwsgi to run Keystone. Therefore, this patch adds a new
keystone_uwsgi_config provider as well as a new keystone::wsgi::uwsgi

This is, hopefully, the first of a long series of patches for
the whole puppet-openstack to add uwsgi support.

Change-Id: I7df39cd3ab18967946afb2954cfa9afd8aa46d72
2 years ago
Takashi Kajinami 35e4cb2a7d Use anchor to require necessary packages
... so that correct packages are required without re-defining them in
resource implementations.

Change-Id: Iae4374a503cf46e3c774dad36f48f25526de765e
3 years ago
Tobias Urdin bc1ff1d7cb Add keystone::bootstrap class
This class combines the keystone-manage bootstrap command
from init, the keystone::endpoint functionality that manages
the keystone endpoints and the keystone::roles::admin class
that manages users and projects.

This is one of the steps to make sure we only have a single
point of entry for bootstrapping (keystone-manage bootstrap)
and then only managing resources after that.

This is especially required since we are getting rid of the
admin token and cannot manage resources before keystone-manage
bootstrap has created the user, project, service and endpoints
for us.

These resources should always be in the default domain and
deployments should manage domain specific configuration themselves
using the provider resources.

This class uses the default values from the keystone-manage
bootstrap command.

In the past puppet-keystone has always created a openstack project
that is assumed as a admin project even though the bootstrap command
creates the admin project. Since this uses the default values from
the bootstrap command we should move away from having an openstack
project, if we need that in testing it should be created there and
not in the default deployment.

Change-Id: I683fcdd743bddf6d4e989dd7e7c553db745934db
4 years ago
Takashi Kajinami d58fcfe75e Do not set public_bind_host and public_port in eventlet section
Remove public_bind_host and public_port configured under eventlet
section as they were alrady deprecated.
Set public_endpoint from public_bind_host and public_port so that
these information can be refered by provider code to get endpoint
even if public_endpoint isn't explicitly given.

Change-Id: Ic38e41b31155a7d3a4f1f5fc606421dd525c1025
4 years ago
Zuul 329ab549a2 Merge "Replace url with endpoint in credentials provider" 4 years ago
Javier Pena 9922e4196f Replace url with endpoint in credentials provider
python-openstackclient has removed the --os-url option in [1]. This
commit is part of the upcoming 4.0.0 release.

The openstack provider relies on that for Keystone initial configuration,
so we need to fix this or puppet-keystone will be broken.

[2] is a more solid long-term solution, but this can work as a stop-gap.

[1] -
[2] -

Change-Id: Id0bc07f352d0b545e60aabd4523536dfc7fc59a8
4 years ago
yatinkarel f16e26daa1 Use role assignment list instead of role list
openstack role list is deprecated since Newton and
is removed in openstackclient-4.0.0 with [1]. Use
role assignment list --names instead.


Change-Id: I7bb2d3573c4f7ba092e019a363116bab11cc7063
4 years ago
Takashi Kajinami a4f74ef5fa Migrate configuration options related to eventlet server from DEFAULT
This patch migrates configuration options related to eventlet server
from DEFAULT section to eventlet_server, as the ones in DEFAULT
section was deprecated[1]

[1] I6dd718c4d54056d0e29978f393ec45f7291f802d

Change-Id: I1a726c706f509f2a2be68098cda8431cddc0fe92
4 years ago
Tobias Urdin 553b03a132 Remove paste
Removes all the deprecated resources for paste
support in keystone which we notified about in Stein.

Keystone doesn't use paste so it doesn't work in Stein
either so we remove it for Train.

Change-Id: Id7dfdf2f5cb9453044d39812b7cbd216b7f3eb50
4 years ago
Nick Bertrand e17812f1ce Don't use proc in title patterns
Using a proc in type title patterns causes an error when generating
puppet types for environment isolation. For example, running `puppet
generate types` on a puppet master produces the following error:
title patterns that use procs are not supported.

This commit changes the `project_domain` parameter to always default to
`DEFAULT_DOMAIN` even when using a `<user>::<user_domain>@::<domain>` or
`<user>@::<domain>` title pattern. The only place I could find that
utilized `project_domain` being unset was determining `keystone_domain`
autorequires. The logic was updated to reflect that `project_domain` is
either set or should use the default unless `domain` is set.

Change-Id: Icd29c5aa89ae442ca69cad0411188162462bf6ff
4 years ago
Tobias Urdin c2456fcaa8 Keystone_user should not use disabled projects
When testing the password for a keystone_user
resource we need to ensure the project id that
is used for testing auth is not disabled causing
it to fail and puppet things the password should
be changed.

Change-Id: Ic4b17a2c750c3162cc609a9469d7422c2084b977
Closes-Bug: 1814906
4 years ago
Juan Antonio Osorio Robles 58dfc07b3a Use "public" endpoint for the authentication URL for the keystone provider
With the removal of the 35357 port in a recent commit [1], we ended up
with an inconsistent use of public/internal bits of the URL. This breaks
in TripleO, since we still configure the admin endpoint. So, the default
port that was used (5000), doesn't work in TripleO.

To address this, we then completely remove the usage of the admin
endpoint for the provider, relying instead on the "public" endpoint
that's configured in keystone.

Typically, it will be behind a load balancer, so it'll actually point to
the internal endpoint of keystone. Which is what we really want to use.

[1] I951e863e7e7c8f409a13398b397b82ef70d7c123

Change-Id: I64cf93ab0c4ade3ae71aa3cd4aea444aff699a17
Related-Bug: #1804426
4 years ago
Tobias Urdin ace7aeb3b7 Remove port 35357 deployment
The legacy admin and public ports for Keystone has since the
release of the v3 API not been required as keystone moved all
actions to the same API. [1]

This patch removes the deployment of port 35357 and remodels
puppet-keystone and more specifically the keystone::wsgi::apache
class to only deploy keystone on port 5000.

This has already been changed in the installation guides [2]
and is the recommend way to deploy keystone.

We have already prepared all our modules default values to use
port 5000 instead of 35357 a while ago and we also in the Rocky
release informed our users with a release note that this would
be performed [3]


Closes-Bug: 1804426
Change-Id: I726cd9408d20f868b2b5337ef2df4da458904e51
4 years ago
Tobias Urdin 697be3badb Deprecate all paste code
Keystone no longer uses it's paste ini file or
uses the paste deploy parts so we deprecate all
that now. [1]

Unfortunately this should probably have been done
in Rocky but got missed, in Stein keystone will
effectively remove the paste ini file. [1]


Change-Id: Ifefea626f754d247b376745fbb2791e725ebafbc
Closes-Bug: 1794045
5 years ago
zhulingjie d4479c1bd2 Replace port 35357 with 5000
Now that the v2.0 API has been removed, we don't have a reason to
include deployment instructions for two separate applications on
different ports.

Change-Id: Ifb06f5f66db45312c87b07aa77195c77d9a2668c
5 years ago
zhangyanxian 5b90e9f4b2 Fix typos in keystone_paste_ini_spec.rb & keystone_config_spec.rb

Change-Id: I4e40a96a60a53da36d2347475a58bb2d4fefd976
6 years ago
Mathieu Gagné 3010079b0a Use domain_id when testing keystone_user password
Older python-openstackclient crashes with this error if no project or
domain are passed to the openstack token issue command:

  Set a scope, such as a project or domain, with --os-project-name or OS_PROJECT_NAME

The @property_hash[:domain] value is not properly populated at this time,
making the "openstack token issue" command being invoked without proper scope.

The @property_hash[:domain_id] value is however available. Using the domain_id
instead makes both older and recent python-openstackclient happy.

Closes-bug: #1654379
Change-Id: Ie23bdb7e9dd77e6ec4358c68d77671ef3c0678a6
7 years ago
Sofer Athlan-Guyot 07cee48dfc Add retry to keystone_user.exists?
Put back exists? method in keystone_user in line with the usual
openstacklib mechanism.  This is done by adding the possibility for
request call to pass regexp messages that shouldn't be retried.

Now we can safely call fetch_user without worrying about having the call
retried by opentacklib.

Fetch_project has the same behavior, so I added it to the mix.  It may
be a performance killer somewhere.

Change-Id: I368cf6a06d21d018337af3e6d09cdabee839a563
Closes-Bug: 1597357
7 years ago
Sofer Athlan-Guyot fe0edef97d Fix endpoint update when one endpoint is missing.
Endpoint are created for admin, internal and public network by this
provider.  If only one of the endpoint is missing then all the endpoints
are recreated as puppet fails to match the resource with the remaining

This fix enable one to update the resource where update means "recreate
the missing endpoint".

Change-Id: Ic605725d1923680c6518ebadda36cb5d596c08fe
Closes-bug: 1559013
7 years ago
Jenkins acc4254e2e Merge "Add Unit Tests for keystone_config type" 7 years ago
MikeG451 ff3c84d3cc Add Unit Tests for keystone_config type
Change-Id: Ieb34b1d9ff0d94e196d0ae4e07465fb7dd719b62
Partial-bug: #1440401
7 years ago
Sofer Athlan-Guyot fef6467537 Add missing bit to the user prefetch removal.
Those are improvement based on last comment in the "Remove user/role
prefetch to support multi-domain.".  Namely:

  1. remove useless comments;
  2. improve the spec to reflect the actual error;

Change-Id: I1f622a14b41876be4b0b6d15cf78e28d76b6360e
7 years ago
Sofer Athlan-Guyot 64100bb284 Remove user/role prefetch to support multi-domain.
In keystone when the multi-domain configuration is enable, listing all
the user is no longer supported.  You have to specify the domain.  The
rational is that some domain will have LDAP backend (possibly AD) with
tons of users.  Listing them all would not be reliable.

The prefetch feature in puppet needs to know all users and create an
associated object.  This is not a good idea when the number of user is
too high.  Thus the removal of this is necessary.  The rational for
using prefetch is that checking all items in one go "cost" less than
fetching individual information.  As the number of user defined in the
catalog is likely to be less than the number of user in the keystone db,
this seems dubious that this would be case here, hence the removal.

As a consequence the keystone_user_role needs prefetch removal as well.
It actually greatly simplify the code.  A cache is made for user and
project id to minimize the number of requests to the minimum.

Closes-Bug: 1554555
Closes-Bug: 1485508

Depends-On: I5b334e3ffd26df4ba8584d77a5e41b56e73536c8
Change-Id: I8e117a9ddbd2ed5b3df739a0b27a66ad07a33e29
7 years ago
Emilien Macchi f710bed6aa sanitize IPv6 in keystone provider
In keystone.rb provider, we need to make sure the ipv6 returned in
clean_host method has brackets.
If brackets are missing, the provider will fail to manage Keystone
resources, because of parsing issue and the multiple ":" in the host

Change-Id: Ibdb340642270afae64b1055ef5fb97281b17066d
Closes-Bug: #1541512
7 years ago
Jenkins 4b0f5276d7 Merge "Fix region association for endpoints." 7 years ago
Matt Fischer 3e50ac09ee Fix region association for endpoints.
The code matching the existing endpoints did not take the region into
account.  This was giving random results and messing up the catalog

This code fix it this and add associated rspec checks.

Closes-Bug: #1535939

Change-Id: If1cdf30c37194b3a7b08bf85860cf7fb7266f6e1
7 years ago
Jenkins 714062347e Merge "Hash domains by name" 7 years ago
Denis Egorenko 367f812476 Hash domains by name
Currently every project and user resource generates a call to
fetch_domain, which is an unhashed query to the openstack domain show
command. This is very wasteful and does not scale well. This patch takes
the code that hashes domains by id and generates an inverse, a hash of
domains by name. This code is used in place of fetch_domain, which is
removed. Since the only thing used in this hash is the id, which should
be immutable, we should not have issues with the other fields changing
(like description and enabled).

This should provide a performance boost to users who have a lot of users
and projects, especially if everything is in the default domain.

Co-Authored-By: Denis Egorenko <>

Change-Id: I99d8fe272aedc0cbc6ad48561ea50a7b7d6cdb1e
7 years ago
Rich Megginson e2792e6b57 Testing some keystone types alone fails
Add the same spec_helper and LOAD_PATH hacks from keystone_paste_ini_spec
to keystone_config_spec
Closes-Bug: 1491812

Change-Id: Ic1c782ad8bfaa1ffef8a4df46f8b8e36024a3941
8 years ago
Jenkins 021c737e1c Merge "Resource keystone_identity_provider for Keystone." 8 years ago
Sofer Athlan-Guyot 07f19bd38a Add keystone domain specific configuration.
Implements blueprint keystone-domain-configuration

Adds a provider able to configure multiple domains and two parameters in
keystone class to setup a working multi-domains configuration.

The keystone_config type has been refactored into a mixin to be shared
by keystone_config and keystone_domain_config.

The provider, even though it is inheriting from openstack_config (and
not keystone_config because it hard code the path), has required more
new code.  The problem is that we have several configuration files to
work with (one per domain) which is unusual.

The self.prefetch method is required to check the current catalog.  If
it's changing the Keystone_config[identity/domain_config_dir] we take it
directly into account without the need for another run.

Keystone_config[identity/domain_config_dir] configuration and the
associated directory are autorequired.

Change-Id: I5e4b298460ee592640af59ac9dcbefa3daf98098
8 years ago
Sofer Athlan-Guyot 27e8c1e9c3 Adjust rspec tests for retries in openstacklib.
A patch in openstacklib[1] broke the some rspec test in keystone.  This
adjust the test to match the new behavior of the lib.

The request_timeout is fixed to 0 so that only one retry is done.
Failure to do that would make the number of retries unknown and the time
spend on the three failing tests around 60 seconds each.

[1] 701b6fe0ac

Change-Id: I6adc439a806cf209e2fae35446a15b07353a18c2
Closes-Bug: #1529371
8 years ago
Sofer Athlan-Guyot 1d1b65c30d Resource keystone_identity_provider for Keystone.
Used for federation, this implement [1]

The remote-id parameter is missing from openstack client kilo release on
most distribution so this provider will work from liberty on.

The hardest part of the implementation is the support for the remote-id
and remote-id-file which are parameters which match the openstack cli.

First they are mutually exclusive.  Second, a file full of remote-id is
passed to the remote-id-file.  This file must be parsed to check if it's
in sync.  If the current format change, ie one id by line, it will need
to be changed.  Third the parsing of the shell output of the remote-id
is rather challenging before openstack cli 1.9.0[2].  This code handles
pre and post 1.9.0.


Change-Id: I4003d1a16d8ac3694f350c82b7b990f374a4a7b6
8 years ago
Jenkins 8f10d5dfdf Merge "Keystone_endpoint match service by name/type." 8 years ago
Sofer Athlan-Guyot 0a4e06abb0 Keystone_endpoint match service by name/type.
This enable keystone_endpoint to specify the type of the service
matched.  This way one can match services which are different only by
type and not only by name, like services nova/compute and nova/computev3
for instance.  It does so by fetching the _id_ of the service when it
has the type information instead of just using the name.

This should be required, and deprecation has been added, as the current
code work only because of a convention.

Change-Id: I9ea20fbad274d583485bc09a52b9df8000eb1af5
Closes-Bug: #1506996
8 years ago
Sofer Athlan-Guyot 961c64e143 Fix default domain.
After the move to composite namevar a problem could occur if another
module was using indirection to find resource by name.

If the manifest didn't have any
keystone_user/keystone_tenant/keystone_user_role definition, then, the
'Default' domain would be appended to the name.

This patch, fix that by simplifying the rule for calculating the default

It now strictly follows what is described there

Change-Id: Ic2efb51fe76d055307c8c27fa79015764417160b
Closes-Bug: #1517187
8 years ago
Sofer Athlan-Guyot 5167a2c804 Refactor of keystone_domain provider.
This remove the @@default_domain_id from the code.  It is no longer
necessary to have a global class variable sharing this id.  All is now
encapsulated inside the keystone_domain class.

The change come mainly from the new policy regarding default domain[1]
which simplifies the logic, and the code.


Change-Id: I71ab37165db6b0fe8472e7dfc8abcf72e0caac4a
8 years ago