21 Commits

Author SHA1 Message Date
David Vallee Delisle
f2ca66fdc9 Adding --check-defaults to validator
When troubleshooting, it's often interesting to see the deltas between
the default or sample configs and the running configuration. The
oslo_config.validator is a great tool to integrate this feature across
all projects. It can also be easily captured by data collection tools
like sosreports with the current deployment packages.

Change-Id: I172d82f19a81282093b0d5f7ae4c1817801cd887
Signed-off-by: David Vallee Delisle <dvd@redhat.com>
2020-11-24 13:55:33 -05:00
Hervé Beraud
6480356928 Add a new type HostDomain.
HostDomain is like HostAddress with the support of
_ character - RFC1033

openstack services are failing to start when a hostname with underscore
_ is provided.

Example:

```
overcloud-novacompute_edge1-0.internalapi.localdomain
overcloud-novacompute_edge1-0.internalapi
```

Nova use `HostAddressOpt` to define `live_migration_inbound_addr`,
and if a hostname with underscore is present in the config file
then the service fail to start.

Example:

```
/etc/nova/nova.conf
live_migration_inbound_addr =
overcloud-novacompute_edge1-0.internalapi.localdomain
```

FQDN is a domain name that specifies its exact
location in the tree hierarchy of the Domain Name System (DNS).

Underscore are allowed by RFC1033 [1][2][3]. Indeed, while a hostname may not
contain other characters, such as the underscore character (_), other
DNS names may contain the underscore.[1][2].
Systems such as DomainKeys and service records use the underscore.

These changes allow us to use underscore with the `HostDomain`.

[1] https://www.ietf.org/rfc/rfc1912.txt
[2] https://www.ietf.org/rfc/rfc1033.txt
[3] http://domainkeys.sourceforge.net/underscore.html

Co-authored-by: Daniel Bengtsson <dbengt@redhat.com>
Change-Id: I0a0670207f96a987996d329e5efa9a5eb2ce000c
Closes-Bug: #1892044
2020-10-22 12:30:39 +02:00
Hervé Beraud
ccb2fcd1e5 reword releasenote for py27 support dropping
Change-Id: I39fbf54eb74843cd268c8cbac0a372927fff00e7
2020-02-06 12:30:40 +01:00
Stephen Finucane
4e071eb034 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in Ussuri cycle.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: I13ce8e8264679a19c7b108381ccb969573ea9ac6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Sem-Ver: api-break
2020-02-03 15:12:02 +00:00
Dolph Mathews
18d1617caa Assume positional arguments are required
The 'positional' keyword specifically applies to oslo.config's argparse
support. Unlike oslo.config, argparse assumes that all positional
arguments are required by default, and you have to explicitly tell it
that a positional argument is optional if you'd like to opt into that
behavior.

This patch adopts that same behavior for oslo.config. When you define an
option to be non-positional (oslo.config's default, designed for config
files), then oslo.config makes that option optional:

However, when you define an option to be positional, oslo.config assumes
that the option is primarily going to be used on the CLI and thus sets
it as required, by default.

This change in behavior has the side effect of allowing argparse to
enforce required arguments on the CLI *while* parsing arguments, instead
of depending on oslo.config to detect the condition *after* argparse has
been allowed to parse "invalid" arguments. argparse correctly raises a
SystemExit in this case, and prints the actual command usage and a "hey,
you forgot this required argument", instead of allowing oslo.config to
dump a backtrace to the CLI with a context-less error message
("context-less" in that no useful CLI usage information is dumped along
with the crash to help you correct the condition).

Change-Id: Ifdc6918444fe72f7e1649483c237cce64b4c72d8
Partial-Bug: 1676989
2017-03-30 18:24:31 +00:00
Ben Nemec
c30d9c0a08 Add exclude-groups option to config validator
When dynamic groups are used, the sample config data may not know
about those group names. As a result, validation of such files will
always fail. This makes it hard to automate config checks since the
output would need to be inspected manually to verify that the missing
options are all from the dynamic group.

Ideally, we would provide some way to map sample config groups to
the dynamic group name used in the actual config, but that would be
more complicated and still might not work in every case if a project
doesn't include sample sections for a dynamic group (although they
_should_ be doing so).

Allowing certain group names to be excluded from validation lets
us to solve this problem in a simple way and maintain validation of
99%[0] of the config options and enables the validation to be scripted
since it won't need manual verification of the error output.

Change-Id: I352fd48f86ecb876fe26a5e50e9a2633af1fff3d
0: citation needed ;-)
2019-03-26 20:00:00 +00:00
Zuul
4e0e72d480 Merge "Add support for looking in environment for config" 2018-11-02 19:58:06 +00:00
Chris Dent
ea8a0f6a8b Add support for looking in environment for config
An _environment source is added that looks in os.environ for
values.

Using the environment is on by default, but can be shut down
by setting `use_env` to False when __call__ is called.

The enviroment is inspected before any other sources
of config data but the value is used after command line
arguments and before config file options.

This is done by checking both the command line and config
files and then inspecting the location of the result. If
it is command_line, we use it. If not, we use the environment
value (if any). If there's no environment value, the config
file value is used. If checking the command line and config
file results in a KeyError, the environment value is used,
if set.

The names of the environment variables follow the rules
described in oslo_config.sources._environment.

A new exception has been added: ConfigSourceValueError, this
is the superclass of the existing ConfigFileValueError. The
code in _do_get has been updated to only use
ConfigFileValueError when it is in fact a file from whence a
ValueError came.

Documentation has been updated and a rlease note created to
indicate the new functionality.

Change-Id: I3245c40ebdcc96f8e3b2dc0bab3b4aa71d07ad15
2018-10-17 20:49:05 +01:00
Zuul
8449a0444c Merge "Add release note and documentation for config validator" 2018-10-08 11:40:07 +00:00
Ben Nemec
1f17b823e0 Add release note and documentation for config validator
Change-Id: Id23d7bbb38c6780621f09b2e24c0cc88831bdb1a
2018-09-26 19:49:43 +00:00
Ben Nemec
5f8b0e0185 Optionally use oslo.log for deprecated opt logging
While we can't add a hard dependency on oslo.log because it uses
oslo.config, in most cases oslo.log will be installed anyway.  In
the interest of being able to make use of features like
fatal_deprecations in oslo.log, let's use it if it's available.

Change-Id: If9499aa6fc28a6b92447b3825d3ca1957cb2255a
2018-06-21 14:50:47 +00:00
Stephen Finucane
bc9b7f5d2f Provide descriptions for choices
Nova uses a common pattern for choices where a 'choices' parameter is
provided and the choices are later documented in the help text. This
frequently leads to code and docs getting out-of-sync and requires
authors to be consistent in how they write option descriptions.
Eliminate the need to do this by allowing users to describe the choices
in the same place as the choices are declared.

Change-Id: Ic084b04ebf232fb72c9c05bbea3a216391b15c83
2017-12-13 14:33:03 +00:00
Ben Nemec
a29c084cb1 Machine Readable Sample Config
Adds the ability for the sample config generator to output the
config data in the machine readable formats yaml and json.

bp machine-readable-sample-config

Change-Id: I236918f0c1da27358aace66914aae5c34afef301
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
2017-06-01 20:47:01 +00:00
Doug Hellmann
12fd34b963 add deprecation_reason to log message
When we log the warning that an option is deprecated for removal,
include the reason if we have one. Previously the deprecation reason was
only visible in the sample configuration file.

Add some tests for the log messages emitted when deprecated options are
used.

Change-Id: I5e309a3651041580fdf529ff31e18bbd90714f35
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-04-25 11:01:42 -04:00
Jenkins
bb8846a3ab Merge "Fixing HostName and adding support for HostAddress" 2016-12-07 06:35:28 +00:00
Dharini Chandrasekar
babff882c0 Fixing HostName and adding support for HostAddress
When config options in different projects use IPOpt as the opt's
type, it restricts operators to only IP addresses.
When the opt is set to HostnameOpt type, currently even an
incomplete or invalid IP passes as a valid hostname. Also, currently
HostnameOpt does not make sure that there is a presense of at least one
non-numeric character in the provided host name.

According to RFC 1123, (https://tools.ietf.org/html/rfc1123),
a valid host name can never have the dotted-decimal form #.#.#.#,
since at least the highest-level component label will be alphabetic.

This patch fixes the existing Hostname Opt to abide by the stated RFC
and also adds a new opt type that would enable operators to provide
either a hostname or an IP and at the same time perform checks on
both IPOpt type and HostnameOpt type, by setting opt type to
"HostAddressOpt" type. This would ensure that an invalid IP does
not pass as a valid hostname and at the same time retains the rules
required to be followed for the validation of an acceptable hostname.

Change-Id: I77bdb64b7e6e56ce761d76696bc4448a9bd325eb
Closes-Bug: #1619044
Closes-Bug: #1615028
2016-12-07 03:27:45 +00:00
Thomas Bechtold
f24b04ea87 Add defaults for config-dir
If no --config-dir switches are given on the command
line, use default directories to search for config
snippets.
This is similar to the default config-file support
oslo.config already includes. It is useful in environments
where command line arguments can not easily be added, like
mod_wsgi Apache envs.

Change-Id: I4df977911539777d1510e8b579375aca5b5f15f4
2016-11-30 06:58:12 +01:00
Adit Sarfaty
14ec2c67eb Add Port type to allow configuration of a list of tcp/ip ports
There was already a PortOpt, but this is not composable with ListOpt.
The new type inherits from Integer and therefore supports min/max
value and choices. For example, openstack/vmware-nsx wants to use this
to configure a default list of ports for a firewall rule.

Change-Id: I83bbec6add8ce2951e94e69ec14bb6d8137d4f0c
2016-08-11 12:05:59 +03:00
Adit Sarfaty
15d3ab88f2 Add min and max values to Float type and Opt
Just like Integers, Floats configuration should also have a minimum and
maximum possible values.
For example, the vmware-nsx plugin needs it for the QoS support.
See https://review.openstack.org/#/c/344763/

Change-Id: If1c47444e0c12b68d9d9cb645b8251e4462cfd49
2016-07-21 14:38:23 +03:00
Doug Hellmann
8247698827 add a release note mentioning our use of reno
Change-Id: I2748f4316627ebc6a270a66d1fcc3774557144e8
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2016-02-22 17:35:40 +00:00
Alexis Lee
68d5def976 Add reno for release notes management
The release management team have indicated this is their preferred way
to collect release notes in future.

Change-Id: Icab591ed1b12c1fd84c15f7b4a5d1d56d1d7588a
2016-02-22 11:28:17 +00:00