Commit Graph

187 Commits

Author SHA1 Message Date
Zuul
0d807bc375 Merge "Use clouds.yaml for puppet" 2022-03-03 12:37:52 +00:00
Takashi Kajinami
83f966a850 CentOS 9: Disable unit tests dependent on puppet-postgresql
The puppt-postgresql module does not support CentOS 9 yet and requires
some version parameters to be run on CentOS 9. This change disables
unit tests requiring that module, until the module supports CentOS 9.

Change-Id: I175dfd157fb3be842a3c1ed38cf3325ec9283f69
2022-02-16 00:26:49 +09:00
Takashi Kajinami
08bf393ee4 Use clouds.yaml for puppet
This change introduces the capability to load clouds.yaml file in
the base Puppet::Provider::Openstack::Auth module, so that each
providers can look up credentials from clouds.yaml instead of rc file.
When SRBAC is enforced, services require appropriate scope for each
operation and this makes it difficult to use rc files which can store
only one credential per file. Usage of clouds.yaml allows us to store
multiple credentials in a single file and switch scopes according to
the API request used.

The new implementation loads the clouds.yaml file for admin user, which
is created by puppet-keystoe. It also allows overriding the credential
by a user-created clouds.file.

We expect clouds.yaml file is created under /etc/openstack, which is
the location openstackclient searches to look up clouds.yaml. To avoid
unexpected conjunction with existing files, the files used by puppet
are located in an independent 'puppet' directory at this moment.

Change-Id: I7587f6e0c2486cbfaf2cbafeb64e9db56a817106
2022-02-11 20:23:53 +09:00
Zuul
47fe665d21 Merge "Support clouds.yaml to manage keystone user credentials" 2022-01-20 18:18:56 +00:00
Takashi Kajinami
522d06ba8b Support clouds.yaml to manage keystone user credentials
Recent openstack cli supports loading user credentials from clouds.yaml
instead of passing each parameters by environment variables or command
options.

This allows us to manage user credentials more flexibly. The biggest
benefit of the clouds.yaml file is that it supports managing multiple
credentials in a single file. When SRBAC is enforced, each API request
should be made with the proper scope credential, and we need to switch
credentials for different scopes(project, domain and system) according.
Usage of clouds.yaml helps this use case hugely because it allows us to
store credentials for each scope in a single file and switch them by
the single OS_CLOUD environment variable(or the --os-cloud option).

Change-Id: Ie8246aa18d90ba506fe708be13c9a5afa3e5d2fd
2022-01-18 09:06:01 +09:00
Zuul
003aef1282 Merge "Support system/domain scope credential" 2022-01-08 00:13:46 +00:00
Takashi Kajinami
e61fb1e4e2 apache: Add support for RequestHeader statements
This change introduces the new parameter to override
the apache::vhost::request_headers parameter, which is used to define
RequestHeader statements in vhost configuration.

Change-Id: I360b18acdf974bf3cdc9f8e817b66cd85f116afd
2021-12-09 01:19:11 +00:00
Takashi Kajinami
0027bf6893 Support system/domain scope credential
This change introduces the base implementation to use system scope
credential or domain scope credential to request OpenStack API in each
provider implementations.

Change-Id: If3781cd2ed828126ef1388553f4b85eed78196e7
2021-10-18 15:09:02 +00:00
Tobias Urdin
16ce2f30de Prevent --password from leaking in failed command output
There is cases when a command times out or when it fails
that we and Puppet [1] will output the raw command that
was executed.

For a user create command that output contains the
--password argument passed down to openstack CLI which
causes sensitive passwords to be leaked into log files
of the system executing Puppet, these can then be shipped
of from the system into a remote syslog and still be in
plain text.

This tries to use Ruby gsub with a regular expression
matching the two cases and instead output [redacted secret]
the same way we do with config provider.

[1] https://github.com/puppetlabs/puppet/blob/main/lib/puppet/util/execution.rb#L286

Change-Id: I4cad8f88fc7b67bb7aa4330832fc47bac41ae9df
2021-09-23 17:03:59 +00:00
Takashi Kajinami
a5cb8965d5 Fix unit tests incompatible with puppetlab-stdlib 8.0.0
Since puppetlabs-stdlib 8.0.0, ensure_packages automatically translates
'ensure => present'  to 'ensure => installed' and that translation
breaks existing assertions in unit tests.

Change-Id: I35500af08e39725bab61be036f780c74e38313b9
2021-09-23 17:03:52 +00:00
Zuul
ad47bad388 Merge "CredentialsV3: Ensure all attributes are cleared by unset" 2021-09-20 08:38:03 +00:00
Takashi Kajinami
d1eb834a20 CredentialsV3: Ensure all attributes are cleared by unset
Currently the unset method only clears attributes defined in the base
Crednetial class and ones specific to CredentialV3 are left set.
This change ensures the method clears all attributes.

Closes-Bug: #1942145
Change-Id: I4bddbf9bb3c6251aa8b68a8bc2ef8799f3c8065e
2021-08-31 13:19:29 +09:00
Takashi Kajinami
2a53c66ed3 Re-implement openstacklib::policy
openstacklib::policy has never been used in any other modules because
it was implemented as a class not reusable for each service.
This change re-implements openstacklib::policy as a defined resource
type so that we can use this implementation from each puppet modules.

The openstacklib::policy resource type provides the purge_config
parameter. When this parameter is set to true, a policy file is cleared
during configuration process. This allows users to remove any existing
rules before applying their own (no) rules.

Change-Id: I9bb486c9191c50c11717dcb9c6af00d17c3aa8f5
2021-08-10 10:25:12 +09:00
Thomas Goirand
508fc7adde Get rid of the $pyvers variable
Since everyone has switched to Python3, it's time for the removal of the
$pyvers variable.

Change-Id: I371ef3a2e257e6d197238e7e9cbfd9d445405be5
2021-06-29 08:06:57 +09:00
Zuul
de68e1eace Merge "Add support for iscsid configuration" 2021-06-28 10:47:27 +00:00
Takashi Kajinami
6e67434c27 Add support for iscsid configuration
The iscsid service is used in sevral components like nova, cinder,
glance and so on to connect to iscsi devices. This change introduces
the new class to manage basic configuration of the iscsid service.

Co-authored-by: Alfredo Moralejo <amoralej@redhat.com>
Change-Id: I3fc6d1192632cc1458d00900508d548f522e9cdb
2021-06-04 20:40:50 +09:00
Zuul
20a4cbd1c4 Merge "Set fixture paths for unit tests" 2021-06-03 00:33:10 +00:00
Takashi Kajinami
55188f078a Set fixture paths for unit tests
This change defines manifest_dir and module_path expicitly in unit
tests so that modules installed under fixtures directory is properly
loaded.

Closes-Bug: #1930403
Change-Id: Id2e226593fa9005e8c051bb714e4c341114f640f
2021-06-02 14:12:26 +09:00
Takashi Kajinami
f0b36283fe Drop Fedora support
Fedora support is never tested, and has been unmaintained for a while.
Because we don't expect any actual user using OpenStack on Fedora, this
change drops support for Fedora directly.

Change-Id: I63c96cd92bad210c0a9527c59f8e1347967172a3
2021-05-26 23:06:00 +09:00
Takashi Kajinami
61aeda0b48 policy: Raise error when file_path and file_format are inconsistent
We have replaced policy.yaml with policy.json following the community
goal[1], but it might be possible that users are not aware of that
migration and still expect json files.
This change ensures that users are not expecting json file based on
the given file path.

[1] https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: Ie2e5a6798e8603585c20b947eb91bbad5453934b
2021-03-25 13:38:08 +00:00
Takashi Kajinami
0ffc8e1c4b Fix support for policy.yaml
This change fixes support for policy.yaml, which was broken because
of the following problems.
 - The default content was still formatted in json
 - Augeas doesn't support flat yaml contents required

Change-Id: Ie308a481eb70d5f930633b18d8044f9542a142af
2021-01-07 16:37:30 +09:00
Tobias Urdin
342f4f0b04 Add ssl_verify_client to wsgi::apache
Change-Id: Iee9f8e0d87310e099185044f1cc1939a03aa9977
2020-10-14 11:20:21 +02:00
Andy Botting
a2e9ebc745 Add the aliases param to pass through to apache
This commit adds a new 'aliases' parameter for passing through to
the Apache vhost config.

Change-Id: Ibe841397e064bbda54bddbb83f8885b5451cbd6d
2020-09-18 10:08:27 +10:00
Zuul
e5c24002f6 Merge "Add Puppet Litmus" 2020-09-17 02:38:26 +00:00
Tobias Urdin
3702dd1314 Add Puppet Litmus
Depends-On: https://review.opendev.org/#/c/751890/
Change-Id: Ia2973a8d37c3cef6255faf449d89121a60cecd10
2020-09-14 23:40:34 +02:00
Takashi Kajinami
10af7893ed Add support for setenv parameter in apache
Change-Id: I0ac20540603d9682354b90f06a852a53d74767e3
2020-09-06 23:37:13 +09:00
beagles
4622709879 Add support for yaml format policy files
This patch adds a 'file_format' field to the policies definition to
allow overriding the default file format which is currently json.

Change-Id: Iec610053a9250cb78c2a17bfc2c197bf55d9df86
Related-Bug: #1885602
2020-07-13 20:36:11 +00:00
Zuul
38e1f44ba0 Merge "Generate password hash from password" 2020-05-20 01:32:03 +00:00
Takashi Kajinami
556f742e5e Remove usage of apache::vhost::wsgi_daemon_process_options
because apache::vhost::wsgi_daemon_process_options has been deprecated
in puppetlabs-apache[1].

[1] 0d5e0bef88

Change-Id: I0a233190d2a1c9d96128d99e1ac1a612161a7446
2020-05-18 23:08:30 +00:00
Takashi Kajinami
e4b68e9ad6 Generate password hash from password
Currently openstacklib only accepts password_hash instead of password
for db credentials, thus we should implement hashing process in each
modules, with including puppet-mysql and puppet-postgresql.
This patch migrates that hash generation to puppet-openstacklib, so
that all logics related to db is gathered in one module.

In addition, because postgresql_password function was deprecated in
favor of postgresql::postgresql_password in puppet-postgresql
6.5.0[1], this patch also deals with that deprecation.

[1] 700d2c5bb5

Change-Id: I898d31e88188bfd3476412a37f48fc918122a98a
2020-05-18 14:29:39 +09:00
Takashi Kajinami
9d08100f29 Add CentOS8 to nodesets
Change-Id: I60bc3b7c77de81f28e08de05636ffff2633ba878
2020-05-10 21:49:04 +09:00
Zuul
c08a4daa08 Merge "mysql: allow specifying the authentication method" 2020-04-02 00:20:16 +00:00
Zuul
60b6455598 Merge "Refactor os distro/version check in client ut" 2020-04-01 17:55:03 +00:00
Takashi Kajinami
f74e9b84fb Refactor os distro/version check in client ut
... so that we can easilty understand which distro and which version
we expect python3 packages.

Change-Id: I385a4a5b123519961791ef6d842ab84990ea00c9
2020-04-01 15:57:52 +09:00
Damien Ciabrini
f4e9903458 mysql: allow specifying the authentication method
MySQL users can be configured to require a specific authentication
method when connecting to the MySQL server, e.g. GSSAPI, SHA-256
or ed25519.

Expose a new attribute $plugin, that is passed to puppetlabs-mysql
When creating/updating a user in the MySQL database.

Change-Id: I1c7b40d110190eba861ed466d2644c2f1abbf7b0
Related-Bug: #1866093
2020-03-25 19:47:11 +00:00
Alex Schultz
3b39a090b1 Use non-legacy fact name for os_workers
processorcount is a legacy fact name. Lets switch the newer fact name.

Change-Id: I310818ffd03ef92890bc5d9f10ad334b5281b126
Related-Bug: #1868454
2020-03-25 13:46:39 -06:00
Takashi Kajinami
74e0f5a317 Have doubled workers for keystone service
Since we have merged 2 keystone services(public and admin) into one,
we need to double keystone workers so that we have the same number of
workers, which is necessory to avoid performance degradation.

This patch introduced new facter, os_workers_keystone, which returns
2 x os_workers .

Change-Id: I737fb14739a69ac12c39c7faf6dd2be1f772daa6
2020-01-31 00:51:24 +09:00
Tobias Urdin
0b5a01dd87 Convert all class usage to relative names
Change-Id: Ic8aff2f30d3473d22c8eac49a431ed044e9ce584
2019-12-08 23:17:31 +01:00
Cédric Jeanneret
ad48860b75 Ensure no directory listing is active
By default, puppetlabs-apache module enables Indexes option, which can
lead in data/structure leak.

The following patch disable that option on a global base, since we
shouldn't need such a feature.

Closes-Bug: #1854442
Change-Id: Icba53f4e32237556608f4cb6dcd9da1a71705c19
2019-12-03 08:38:29 +01:00
Cédric Jeanneret
55668d9f33 Allow to activate WSGIImportScript option
This should prevent slow loading on the first request, especially
annoying when this first request is done by a healthcheck launched by an
inflight validation.

This patch is a reaction to the whole issue raised by the new nova
inflight validations[1], followed by some discussions in order to get a
faster application loading.

[1] https://review.opendev.org/#/q/status:merged+project:openstack/tripleo-heat-templates+branch:master+topic:bug/1842687
Related-Bug: #1843555

Change-Id: I27e37e30823c4312d9d7a93f18fe0f930ce70c49
2019-10-09 16:05:24 +02:00
Javier Pena
03170a9383 Replace url with endpoint in credentials provider
python-openstackclient has removed the --os-url option in [1]. This
commit is part of the upcoming 4.0.0 release.

The openstack provider relies on that for Keystone initial configuration,
so we need to fix this or puppet-keystone will be broken.

[1] - https://review.opendev.org/677795

Change-Id: I50830450efe7a13be801a31f4ef0333684c7837b
2019-09-17 13:15:23 +02:00
Tobias Urdin
7ffbfbf0f1 Remove CredentialsV2_0
This is not used anywhere but in Fuel which is dead.

Change-Id: I0c5957ec5690e8f52467fd38e780b067be7d8c5b
2019-06-10 22:33:16 +02:00
Harald Jensås
7d0f1b785d inet6_prefix() - add test with prefix in input
Extend the unit tests of the inet6_prefix() function to
test input including the prefix ('inet6:[2001::01]:80').

Change-Id: I68ead773868bd418f10a480f2b7b7fc08084374e
2019-05-24 16:43:06 +02:00
Harald Jensås
1f13d87fdf inet6 prefix utility
Utility to handle prefixing IPv6 address with `inet6:`.

This is useful for services relying on python-memcached
which require the inet6:[<ip_address]:<port> format.

Change-Id: Ibd280929f62bae61f34b2984af7710fbd422264b
2019-05-24 01:53:02 +02:00
Tobias Urdin
c0ffd3502d Update puppetlabs-mysql dependency to min 6.0.0
And change the upper constraint to be the latest
8.0.0 release. See all other patches on this topic.

Change-Id: I30f0367c9eeafe7e8b45c348c3df5630f29fa6a2
2019-02-13 12:17:21 +00:00
Tobias Urdin
63cd960253 Inherit pyvers from openstacklib::defaults
Depends-On: https://review.openstack.org/#/c/633162/
Change-Id: Ied4fcd08ce82543907c4cf8fd5f2780e63a47eed
2019-01-25 15:55:18 +00:00
Tobias Urdin
6eaab435b2 Add spec testing for multiple bind_port
Adds simple spec test for multiple ports in
the bind_port. The change in puppet-keystone
will need to support multiple ports so we
ensure nobody breaks this.

Change-Id: Ie2ab4641b0829b872fee0dea3aab236455eb3266
2018-12-05 22:09:06 +00:00
Tobias Urdin
4d08ab70cb Cleanup spec testing
Change-Id: I3c13fc2f8bc3582fd72206b3c711ff5863943b09
2018-11-01 17:54:51 +01:00
20ac1f925a Install python3-openstackclient in Fedora or RedHat > 7
Fedora repo [1] has python3 packages, start consuming those.

[1] http://trunk.rdoproject.org/fedora/current/

Change-Id: I8e2ec45e38275fb0910f5921662cb7cf865edd0a
2018-10-25 16:49:16 +05:30
Javier Pena
33b05434f3 Make sure the wsgi script dir mode is 0755
When the system-wide umask setting is more restrictive than the
default setting, the wsgi script directory permissions may not allow
the apache process to access them, resulting in errors.

A similar fix was applied to Keystone some time ago, see [1].

[1] - 4f15fb64b1

Change-Id: Ie9769657dc530bc895a3119b3e458864a8b5f293
2018-10-03 15:16:58 +02:00