Update git submodules
* Update oslo.policy from branch 'master'
- Fix sample config value when set_defaults is used
By calling set_default[1] on a conf object it only applies to opts
registered to that object. This causes an incorrect value to appear in
the generated sample config because it deals with a list of raw opts,
not a conf object.
To fix this, we can call the global set_defaults[2] on the cfg module
which alters the opts directly. This is the method used in the cors
middleware[3] and works as expected there.
This does complicate the unit tests, however. Because we're altering
global state we need to save the original opts and restore them after
the test. Furthermore, the conf.reset() call in the config fixture
doesn't sufficiently reset the conf object to allow it to recognize
the replaced opts. For the purposes of this test we can just create
a standalone conf object though, which gets past that problem.
It's possible that we should fix reset() so it actually removes opts
in groups completely, but I'm unsure what implications that might
have for other users of the function.
1: b5df53543f/oslo_config/cfg.py (L2433)
2: b5df53543f/oslo_config/cfg.py (L391)
3: 8c7fa5bb10/oslo_middleware/cors.py (L88)
Change-Id: I3af9de1b39b6360ecfcb448d8c37b463e1a42ca7
Closes-Bug: 1807184
- Merge "Use oslo.config instead of argparse."
- Use template for lower-constraints
Small cleanups:
* Use openstack-lower-constraints-jobs template, remove individual
jobs.
* Sort list of templates
Change-Id: If1babd86d4695fe216ff703ed862c3f06d448e15
Needed-By: https://review.openstack.org/623229
- Use oslo.config instead of argparse.
Changing arg consumption from argparse to oslo.config in
order to also provide behavior control using config files.
Change-Id: Iec4dab763b973b70c98077cb29708acd9cbbcec4
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
- Prevent sensitive target data from being logged
A previous commit made some changes to allow for more robust logging
of RBAC enforcement data:
I4642c57990b145c0e691140970574412682e66a5
This also included logging of the target data, which is provided by
the service calling policy enforcement.
This commit makes it so that target data is protected from exposing
sensitive information. A good example is doing operations on users
in keystone since keystone would populate the target dictionary
with user information, and possibly passwords.
This issue was found in keystone unit testing while trying to consume
oslo.policy 1.43.0.
Change-Id: I2702df8f3d7c040312eb863f7772b129e0e2c45c
- Merge "Change openstack-dev to openstack-discuss"
- Merge "Add domain scope support for scope types"
- Merge "Make upgrades more robust with policy overrides"
- Change openstack-dev to openstack-discuss
Mailinglists have been updated. Openstack-discuss replaces openstack-dev.
Change-Id: I84f738109f098415496619df423db7778b2fdcf2
- Merge "Fully log RBAC enforcement data"
- Merge "oslopolicy-checker: iterate through rules in sorted order"
- Fully log RBAC enforcement data
Data passed to the RBAC enforce function consistes of 3 items:
* rule name or rule object
* credential data
* target data
Both the credential and target are dicts. When policy enforcement
does not work as expected it's essential to capture the input to
the enforcement engine as to ascertain why the rule did not work
as expected. It would also be highly advantageous if the logging
were in a format that could be digested by other tools
(e.g. oslopolicy-checker).
This patch does two things:
1) It logs the policy relevant input to Enforcer.enforce()
2) It eschews the use of Python's string formatting which may not
fully dump the contents of the dicts and is not easily parsed in favor
of using JSON format which does fully capture the object's content and
can be used in data exchange (and can be read by oslopolicy-checker).
Contents of the credentials dict are filtered to scrub security
sensitive data.
Closes-Bug: #1804073
Change-Id: I4642c57990b145c0e691140970574412682e66a5
Signed-off-by: John Dennis <jdennis@redhat.com>
- Add domain scope support for scope types
This commit makes it easier for services to protect APIs meant for
domain-only operations. It does this by making "domain-scope" an
official scope type to check for during policy enforcement.
A good example of where this would be useful is protecting the user
API in keystone, since user's are technically owned by domains.
This commit bumps the version of oslo.context to 2.22.0, which also
has domain support.
Depends-On: https://review.openstack.org/#/c/613635/
Change-Id: Ifc83a5f261bc823060eca5c4d0a4bf07966794c4
- Make upgrades more robust with policy overrides
Previously, we'd notify operators of changing or deprecated policies
by logging a warning while loading rules. However, this doesn't
prevent unintended access if an operator is overriding a policy
by its old policy name. In this case, let's make sure we check if the
old policy is being overridden and use that override for the new
policy's check value.
This commit introduces this change along with a few tests. It also
refactors the deprecated rule logic in load_rules() to separate
methods so that it's a little easier to understand where that logic
happens within the load_rules() method without cluttering it.
Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
Closes-Bug: 1800259
Change-Id: Ice27cdb44241da94693625776037ea6164bbb913
- Merge "Enhance test to prevent JSON parsing regression"
- oslopolicy-checker: iterate through rules in sorted order
This makes it easier for folks checking their policies to just
execute their rule checks and compare them with the original output.
Instead of having to manually pipe the result and sort it.
Change-Id: I8d45173578d3b309b97caaa7d4e87cb2aec0e8f2
- Enhance test to prevent JSON parsing regression
Change I43782d245d7652ba69613b26fe598ac79ec19929 added a policy
file parsing optimization that had the affect of allowing some
strictly speaking invalid JSON policy files to be accepted.
Enhance the test for bad JSON to look for this to prevent
formerly acceptable policy files from becoming invalid if
the code is refactored at some point.
Also updates two related comments that had gotten out of sync
with the code.
Change-Id: I6a269b91436cac29bd72e11dbdc51ee74feca028
- Correct typo in docs
In the "special checks" section, 'role' and 'rule' are keywords
and should not be enclosed in '< >'.
Change-Id: Ia3c1b47f1c8452bcca62961de4414d21d7ebf481
- Fix usage of token fixture in shell tests
The way we were using the token fixture in the shell tests was modifying
the structure. If the test would get run by the same process, it would
then use the modified structure and the test would fail.
This uses deepcopy instead, so this way, we never modify that fixture.
Change-Id: Ib88feee7d7fe72c66b4e8af510f9f28411ac47df
- Add ability to pass in target data for the oslopolicy-checker
This allows us to test the policy for other services which might have
different or unusual target data formats (such as Barbican). It would be
possible to pass it as a nested dictionary, e.g.:
{
"target": {
"secret": {
"project_id": "my project id"
}
}
}
or as a key pair (as oslo.policy would expect):
{
"target.secret.project_id": "my project id"
}
Both will work (note that this logic was taken from barbican).
This fixes around the limitation that the target is hardcoded to be
"project_id", and thus allows to test more scenarios (such as the
project ID not matching).
Change-Id: Ia9f7462072a8cb142251c8bb5ef19d9a25a98119
- Pass in policy name as part of the oslopolicy-check check call
We were not passing the policy name, which made it quite hard to test
out external checks given that this is information that is passed in
there. This passes that parameter.
Change-Id: I217a6545bdf753470e08b39de2c0df08ffa1f82f
- Unit test for CLI
This adds unit tests for the shell.py file, which is what we use for the
oslopolicy-checker command.
Change-Id: I52d8669b30e868a4fbdb33316f4db31947b08fa2
- Merge "Update sphinx extension logging"
- Update sphinx extension logging
Sphinx 1.6 deprecated using the application object to perform logging
and it will be removed in the upcoming 2.0 release. This updates our
extensions to use the recommended sphinx.util.logging instead.
Change-Id: Ia9edbfd551d260b798818940e4d156957f382324
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
- Add minor nits in testing documentation
This commit addresses nits from
https://review.openstack.org/#/c/604192/1
Change-Id: I2ca0cd67eca4d1b2d0320f7ecb40c9ff55229b12
- Merge "Add docs for developers testing APIs"
- Merge "Add guidelines for naming policies"
- Clean up .gitignore references to personal tools
Developers run all sorts of different tools within Git repositories,
any of which can leave their own special trashfiles all over the
place. We can't every hope to catalog them all, so better to
recommend developers simply configure a global core.excludesfile to
filter the irrelevant files which tend to get created by their
personal choice of tools.
To this end, remove the long-standing sections for "Mr Developer"
and "Editors" since their mere existence here sends the signal that
we welcome (and have time to review) additions for any old tool
someone ever might happen to try. Also add a comment block
explaining this, for clarity.
We can, and should of course, continue to list files created by the
tools recommended by our workflow (test frameworks called from tox,
documentation and packaging builds, et cetera).
This change is a port of I1b41efac219fca44e2548fc36633724d0ecfc0cb
from the openstack-dev/oslo-cookiecutter repository.
Change-Id: I3eeb6157ed79e2b75e14e8e94fcfe40c4bf7ff42
- Add guidelines for naming policies
Inconsistent policy names across OpenStack services has been a pain
point for operators and users for a long time. This is an attempt at
documenting a set of conventions for developers to work towards to
provide a more uniform experience. These conventions were discussed
publicly on the mailing list:
http://lists.openstack.org/pipermail/openstack-dev/2018-September/134597.html
Change-Id: I8831c44a3544d11c0bb1c0ce58d1a140f861e22b
- Merge "sphinxext: Start parsing 'DocumentedRuleDefault.description' as rST"
- Add docs for developers testing APIs
Change-Id: Ie752d7e9d40be33ba29f6c14d6a6f16e1fcc66f1
- Merge "Docs: Remove references to JSON format"
- Merge "add lib-forward-testing-python3 test job"
- sphinxext: Start parsing 'DocumentedRuleDefault.description' as rST
Users expect this to be parsed as rST and write their docstrings accordingly.
This has the potential to introduce warnings for users with improperly
formatted rST and these warnings can be promoted to errors if 'sphinx-build' is
used with the '-W' option. As a result, we disable the 'warning-is-error'
logger for these options. We may wish to change this behavior in the future.
It is not really possible to test this yet as the output wouldn't look much
different. In addition, the error messages generated are rather unhelpful. Both
of these can be changed in a future modification.
Change-Id: I4572eef31a8675eabb791c14279490348e949cd0
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
- Docs: Remove references to JSON format
The Syntax section of this is supposed to be describing the YAML
format, but references the JSON format.
This commit updates the documentation to remove this reference.
Change-Id: I7bfc11a3f1ecc57e3865d308b7a9bffd8453bce2
- Merge "Move _capture_stdout to a common place"
- Merge "add python 3.6 unit test job"
- add lib-forward-testing-python3 test job
This is a mechanically generated patch to add a functional test job
running under Python 3 as part of the python3-first goal.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I6cb066225c88f174b90eb21e8a0d26974641c41b
Story: #2002586
Task: #24322
- Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html
Change-Id: I3919b7000ca46212d17a4e9e7c3fea55ed74cd78
- add python 3.6 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I051bf6721001c99065c0790c6768417e61db015b
Story: #2002586
Task: #24322
- Move _capture_stdout to a common place
The _capture_stdout method was duplicated in a couple different test
classes that all inherited from PolicyBaseTestCase.
This commit just moves the method into PolicyBaseTestCase and removes
the duplicate implementations across test classes.
Change-Id: Id62db9bb0e6f0e6d0d2917714302947415cb9829
- Remove PyPI downloads
According to official site,
https://packaging.python.org/guides/analyzing-pypi-package-downloads/
PyPI package download statistics is no longer maintained and thus
should be removed.
Change-Id: I7214f39634d8c5711aa2217f69945a798bc87bec
- import zuul job settings from project-config
This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.
Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I96e96b197c418b0fe411b54d1d3fd7e98320be51
Story: #2003250
- Merge "generator: Reimplement wrapping of 'description'"
- Update reno for stable/rocky
Change-Id: I40d2c2007c9cf3b372d44f8915567da33a048472
- Merge "Avoid redundant policy syntax checks"
- Avoid redundant policy syntax checks
Introduce a private variable inside Enforcer class to remember
status of the last policy syntax checks in order to avoid
redundant calls to the check_rules() method.
Having this flag makes the whole rules mechanism faster, as under
certain conditions check_rules() method was being executed
multiple times even when not needed.
Change-Id: Id3992fc0cb567451049a12ebdc6851e737573bb8
Closes-bug: #1723030
Co-Authored-By: Ben Nemec <bnemec@redhat.com>
- Merge "Teach Enforcer.enforce to deal with context objects"
- Teach Enforcer.enforce to deal with context objects
The ``creds`` dictionary passed into oslo.policy's enforce() method
assumes a lot of the same values already specified by oslo.context
RequestContext objects.
This commit teaches enforce() to handle being passed an instance of
a RequestContext object, and populate credential values accordingly.
Change-Id: Ia74bf6c40b1e05a1c958f4325e00f68be28d91b9
Closes-Bug: 1779172
- Merge "Pass dictionary as creds in policy tests"
- Merge "Fix requirements and convert to stestr"
- Pass dictionary as creds in policy tests
According to the documentation strings for the ``enforce`` method, the
``creds`` parameter is expected to be a dictionary. The implementation
also treats it like a dictionary.
This commit updates three of the tests in test_policy.py to pass a
dictionary into enforce instead of just a string called 'creds'.
Change-Id: Id05a41b84123aeb57b193bba97cb9e8442587a51
- Fix requirements and convert to stestr
This commit fixes two issues that are currently blocking the gate.
The first is that it bumps the Sphinx requirement to be within
acceptable constraints. The second is that it converts oslo.policy to
use stestr instead of testr. This is all being done in one patch
because proposing them individually causes deadlock (the patch to
bump the sphinx requirement fails because we're still using testr and
the patch to convert to stestr fails the requirements-check job).
The following explains the reasoning behind the stestr change.
With the upgrade to oslotest 3.6.0 [0], testr no longer works [1].
This is because oslotest no longer requires testr and we don't depend
on it directly in oslo.policy.
[0] d5a3c58f71
[1] 897823fbd6
Change-Id: I6dac4c8e7b39c9b80cc8f3728763e8d783c9e940
- Add blueprints and releasenotes link to README
Change-Id: If5fa8c9f3ec24881196d36eace6b6f5f58efdb4f
- Merge "Add examples and clarification around scope_types"
- generator: Reimplement wrapping of 'description'
The current implementation of wrapping for the Policy.description field
leaves a lot to be desired. It takes each line in the original
description independently and wraps that, ignoring the fact that the
line may be part of a paragraph, or could be a literal that shouldn't be
wrapped. As an example, imagine that wrapping occurred at 40
characters instead of 70. In this case, the below:
A sample lines with more than forty characters
which continues down onto the next line.
would become:
A sample lines with more than forty
characters
which continues down onto the next line.
when clearly, what we want is something like this:
A sample lines with more than forty
characters which continues down onto the
next line.
This is resolved.
In addition, it should be possible to include basic literal blocks and
those should be preserved. For example:
Here's a sample literal block
This is a really long literal block but it should not wrap
should be output in the exact same way. This is also resolved.
Note that we're not accounting for things like bullet points. Doing so
would bring us close to rST parser implementation territory, and we
don't want to go there. We can revisit this if someone turns out to want
this feature.
Change-Id: I3ea2aac73e3c0a4f77f3f4097540de01264cd618
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
- Merge "Add CLI usage documentation"
- Merge "Clarify CLI documentation"
- fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: Ifeed5b9c1357cbb00502c8c7d2932a5e2ad2e2a6
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
- Merge "trivial: Fix file permissions"
- Merge "Remove erroneous newline in sample generation"
- Add CLI usage documentation
This commit adds usage and examples for generating sample policy
files and listing redundant policy rules.
Change-Id: I2ff00a0a038fde5596ec2fe35de1b7647efcbb9c
Closes-Bug: 1741073
- Clarify CLI documentation
This commit simply expands on the CLI usage document so that we can
easily expand it to include new sections.
Change-Id: Ib1c4d6999e2cdcb078609f0cf612f7073d2747c4
- Remove erroneous newline in sample generation
The sample generation code for policies has a couple different cases
that make sure deprecated rules have descriptions and reasoning
formatted in the comment section. One of the cases that handles
policies deprecated for removal was injecting an extra newline in the
comment text that threw off the formatting of the sample, leaving
the subsequent policy uncommented, and visually unpleasing.
This commit removes the extra newline in the formatting logic for
deprecated policies and adds a test case for the behavior.
Change-Id: I76338d2fbaccf3b43e0da04732fd9df3c05dfbda
Closes-Bug: 1771442
- Update sphinxext to include scope_types in docs
Since we've added ``scope_types`` as an attribute to policy rules, it
makes sense to include this information in documentation. End users
will need to know what type of scope is required to pass a specific
policy rule when services start incorporating system scope and scope
types.
Change-Id: I86d89e9f45740b39cef04773cec8846c1ab97c3a
Closes-Bug: 1773473
- Merge "Fix document formatting"
- Fix document formatting
After openstackdocstheme migration, extra whitespaces at the beginning
of lines lead to vertical lines and different fonts in rendered HTML.
This commit cleans up them for readability.
Change-Id: I27f25d068cff801dd4702278ecf8be14baebc890
- Add examples and clarification around scope_types
When we initially implemented the scope_types attribute, we included
some documentation but we never explicitly described how to use
scope_types, what was officially supported, or why the attribute is
important.
This commit attempts to add documentation that clarifies those areas.
Change-Id: I46d351b3c9888a703d520082f10ebbedc53973ff
Closes-Bug: 1771621
- Include deprecated_reason when deprecated_rule is set
Previously deprecated_reason was only included in the sample policy
file in the deprecated_for_removal case. There's no reason users
wouldn't want to be able to provide an explanation for deprecated_rule
deprecations too.
Change-Id: I4036bf8efcd42ca4b1a35ae6940ac69af7fe205b
Closes-Bug: 1771416
- Include both new and deprecated rules in generated sample
Previously, if a deprecated_rule was set on a rule, only the
deprecated rule was written to the sample policy file. This is
wrong since we want to be setting the new, undeprecated rule going
forward.
Change-Id: I181aeecda686d1d499616254c93002f51a5462c3
- trivial: Fix file permissions
No reason for this to be executable.
Change-Id: I125a5b73101fa6f1d55f4ded91d3b4192c91d87e
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
- Merge "Update documentation to include usage for new projects"
- Merge "Remove stale pip-missing-reqs tox test"
- Remove stale pip-missing-reqs tox test
pip_missing_reqs tool is no longer maintained and has broken with
release 10 of pip
Refer to:
http://lists.openstack.org/pipermail/openstack-dev/2018-April/130027.html
Change-Id: Ifdc94e6cd8f39007577f1825ad6a65e5bc505911
- make the sphinxpolicygen extension handle multiple input/output files
Some projects have multiple policy files for different parts of their
API. Make the sample generator extension support this by letting the
policy_generator_config_file option be set to a list of tuples mapping
the config file to the output file base name, as we do in the sample
generator in oslo.config.
Change-Id: I0c7fa409a1ed0f49d65c9b90b71317066f6d3505
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
- Merge "Trivial: Update pypi url to new url"
- Update documentation to include usage for new projects
The oslo.policy bits to generate documentation and use the oslo.policy
CLI scripts is usually copy/pasted from other services that have
already done that work. This commit attempts to pull that information
into the usage section of oslo.policy, complete with examples, instead
of making OpenStack developers hunt through projects to find examples
they can use. Or wonder why or how this tooling works.
Change-Id: Ie74888b09bb836c192a88c5beddd86297c8bceda
Closes-Bug: 1766953
- Trivial: Update pypi url to new url
Pypi url changed from [1] to [2]
[1] https://pypi.python.org/pypi/<package>
[2] https://pypi.org/project/<package>
Change-Id: I2abba3a89f3f89d45b97e5cc050f616136809b1d
- set default python to python3
Set the default python to python3 except for the py27 environment. We
have to set that explicitly to override the new default.
Change-Id: I55118c8c80884d6f71d2730dfc26b28e5b1fdf28
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This commit is contained in:
Ben Nemec
Gerrit Code Review
parent
4e3e871936
commit
f9d720c6ba
|
|
@ -1 +1 @@
|
|||
Subproject commit 509cf0839a0ce6732d8a76faaa85fd82735fe7e7
|
||||
Subproject commit 3d85afb24a014f43e961887c4e5b679e7eb7dec8
|
||||
Loading…
Reference in New Issue