This is a slow import and the single user of it, the 'split_by_commas'
helper, does not appear to have any users outside of python-glareclient
(which is a dead project). We might want to remove the user at some
point, but for now simply defer loading of the library.
Change-Id: I91d0c6eec5333a660f995a9d1436e4b068693900
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Those regexes will fix Object style representation output.
See the payload used in tests for details. This kind
of output can be obtained by using the command:
```
$ openstack --debug
```
Co-Authored-By: Daniel Bengtsson <dbengt@redhat.com>
Change-Id: I9024be93b109d1b64ca736546c0f69db7a5e06d0
Some use cases are poorly handled by the regex used
to mask password. Indeed when the password contains
quotes or double quotes in the middle such as `pass"word`,
the mask_password method will return `***"word`.
For more details please see
https://bugs.launchpad.net/oslo.utils/+bug/1949623
Closes-Bug: #1949623
Change-Id: I941750b4d49d2d75f0831b24d6dd17f4040f70a2
The abstract base classes previously defined in 'collections' were moved
to 'collections.abc' in 3.3. The aliases will be removed in 3.10.
Preempt this change now with a simple find-replace:
$ ag -l 'collections.($TYPES)' | \
xargs sed -i 's/\(collections\)\.\($TYPES\)/\1.abc.\2/g'
Where $TYPES is the list of moved ABCs from [1].
[1] https://docs.python.org/3/library/collections.abc.html
Change-Id: I85f2757852c0313967f5d82166124feb10aa4c6a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
As the name suggests this is simply an id referring to a key and not the
actual key itself. As such we should stop masking this in an effort to
ease debugging and troubleshooting when it is logged.
This was previously incorrectly masked by
I9e684cd8bab85728ff0117f95a30eb7dbb5bf51c as part of bug #1814365.
Closes-Bug: #1905264
Change-Id: I856e3cf32c409debdfb15aa96415c3309fe2f516
The "r" for raw strings has to repeated when several strings are
concatenated, otherwise the backslash in the strings that miss
the "r" are interpreted directly, and result in invalid escapes.
Change-Id: I3736a40043c8aa6c36c21803bf68f842af5cd55e
In Icc19b7c8bdb6a3182939d5e9fdef21288b19f43d mask_password was made case
insensitive but mask_dict_password wasn't. This update makes the
behaviour of these functions the same.
Instead of lowering _SANITIZE_KEYS each time the source list is lowered.
New password patterns from realworld logs were added to the patterns.
Change-Id: Ic3ee301857630a15b9c26fd5d0fc907c43199517
Related-Bug: #1850843
It appears that Mistral service logs everything, and doesn't use yet
the mask_password (nor mask_dict_password) method. In order to ensure
all is properly masked, we have to add some new patterns, and make it
case insensitive in order to simplify and avoid duplicated entries.
Change-Id: Icc19b7c8bdb6a3182939d5e9fdef21288b19f43d
Related-Bug: #1850843
Signed-off-by: Cédric Jeanneret <cjeanner@redhat.com>
mask_dict_password doesn't actually have a dependency on the dict
type specifically. It can work on any subclass of collections.Mapping.
This changes the isinstance check to reflect that and adds a unit
test using a collections.Mapping subclass.
Change-Id: I28781acf027b9b34f8274196db5dd4d2a9adc9ba
Closes-Bug: 1804528
This method is called recursively (ie deeply) by default
so there doesn't seem to be a good reason to deepcopy over
and over and over at every recusion level especially since
a new output dictionary is getting created anyway.
Change-Id: I644ef881e487c06dc4db77d60cfe765b0e59b547
In python dict keys can be anything that is hashable; which
includes non-strings such as ints. Currently the code is blowing
up with these types of keys with exceptions like:
TypeError: argument of type 'int' is not iterable
So to fix that handle the case where non-string keys are found.
Change-Id: I4f576a089df6f68e43572bf0eee15e99f2b557fe
Nova's create keypair operation will currently log the generated
keypair if debug logs are enabled.
Closes-Bug: #1770683
Change-Id: I7f889f0bf254fad43b1e26d32fa145f88c668b39
This proposes a "mixed" parsing type that interprets the IEC-ish "i"
units as power-of-two and the SI-style "non-i" units as power-of-ten.
This is a round-about way of saying it is basically making it the same
as the way GNU parted interprets such strings [1].
While I'd be the first to admit it's a bit ugly, I think it could
serve a role for people who have to interpret such strings for
compatability reasons. In particuarly within diskimage-builder, we've
accepted these strings for partition creation (being modeled on
parted) and I'd certainly like to not have our own unit parser if we
can help it.
[1] https://www.gnu.org/software/parted/manual/html_node/unit.html
Change-Id: I772bcb6651484d68afabd1965e5db033439c6257
With pbr 2.0 and Sphinx 1.5, the setting for treat sphinx warnings as
errors is setting warning-is-error in build_sphinx section. Migrate the
setting from the old warnerrors one.
There are two issues:
- The `*` character is recognized as an emphasis character and must be
escaped
- The index references modules that have since been removed
These are resolved.
Change-Id: Id078022c0f75bf8f162b693d260f67116eb6428b
Method bool_from_string accept boolean, string, and integer value.
This commit adjusts the document to reflect acceptable argument type.
Change-Id: I836d2bed89c22dc123052fbb580c0601076c12ab
mask_dict_password method removes the keys from passed dict if those
are not
- indicative of password
- Dict
- String type
So keys with value None or other type than string/dict are being removed.
This method should just change the password with secret and return other
Key as it is without any removal or alter.
Change-Id: I21294ce6fd25a3e6438de41949d4427b17945bb2
Closes-Bug: #1600168
mask_password when given a value of:
{'ipmi_password': 'foo'} will return {'ipmi_password': '****'}
But passing the same dict to mask_dict_password() would return the dict
unchanged.
This fixes it so that it works consistently on which keys will trigger a
masking of the password.
Change-Id: I279f245fa750a57809c9d000137c4f9baf44cd40
Closes-Bug: #1594935
The Huawei cinder driver needs to mask out any instances of the
CHAPPASSWORD in the logs [1]. This patch adds to the sanitize
keys list so that cinder may make use of the common mask_password
function.
[1] https://review.openstack.org/#/c/271595
Change-Id: I679e576121e2a74ac6684ecc634690708cf39315
Partial-Bug: #1535706
The mask_password function was missing the versionchanged to
signify the addition of 'configdrive' to the list of sanitized
keys.
Change-Id: Ia2bf37ad6d757f3fcb1f4144c979182b05e672a3
The widely used mask_password() function will mask passwords in
strings using a list of keys and patterns. This change extends it to
work on dictionaries as well. This allows one to directly invoke
mask_dict_passwords() on a dictionary and this will have the
effect of masking not only the strings (values) but also mask the
values if the keys in the dictionary are part of the list of
sanitization keys.
If the dictionary contains nested dictionaries, those will be
recursively masked as well.
Change-Id: I7ebafdeb671da36e0fdc9d6983a17ac5481b6f28
Closes-Bug: 1526041
The node representation in Ironic can contain a key called "configdrive"
which the value can be either a base64 string or a Swift temporary URL
(which is a secret). We need to mask that key when logging the node
representation either in Ironic or Nova driver so reusing the oslo's
mask_password() makes sense.
Change-Id: I10f9f37ff635f9a2dddfd7f7aaaac7d4fe6244b9
We have the same conversation every 6 months or so. Let's document our
reasoning here so folks are less likely to waste time adding new
arguments to the function and having their patch rejected.
Related-Bug: #1407811
Change-Id: I0d0a30a1256ddc34edce6c312abe9a53fa8ac7ae
Some parts of the code have been annoted with "# nosec" as they made bandit
raise false positives.
Change-Id: I9d8a8ab79e8b5de98e7d291ac132f8971a44216d
Document in which version new types and functions were added using
".. versionadded:: x.y". Document changes using
".. versionchanged:: x.y."
For new modules, add the versionadded tag in the module top
docstring, not on each type/function.
Add fileutils to documentation. The doc part was forgotten during the
graduation.
Add docstrings to convert versions of versionutils.
I used "git blame" + "git tag --contains=SHA1" to find these version,
and then I checked manually each version.
Change-Id: Ia2f00aa29eb36410a49fc1d350896a569a7737a1
Typically the pattern of the following happens:
>>> s = dict()
>>> v = s.get("blah", True)
>>> v = strutils.bool_from_string(v)
In this case we can avoid converting the value of 'v' to a bool
if it was already a boolean (and if it wasn't then the rest of
the code can be ran to attempt to convert it to one). This avoids
needlessly converting things from bool -> string -> bool which
is not really needed in this case.
Change-Id: Id7397e91e754ff2c63b3f112e95aedf82cd31717
Only apply substitution patterns related to key values that appear in
the string.
Replace .*? in patterns with more explicit patterns that don't require
backtracing.
Add a performance test script for future testing work, with references
to large data files that can be downloaded but that we don't want to
check into the source repository because of their sizes.
Change-Id: Ic3ed252d181c93b8a0db465db6c8c4a7ca97da42
Related-bug: #1408362
Zuul