From 256160b849e49d819f6abd492a99989dec285ae7 Mon Sep 17 00:00:00 2001 From: Ghanshyam Mann Date: Wed, 25 Nov 2020 14:04:22 -0600 Subject: [PATCH] [goal] Deprecate the JSON formatted policy file As per the community goal of migrating the policy file the format from JSON to YAML[1], we need to do two things: 1. Change the default value of '[oslo_policy] policy_file'' config option from 'policy.json' to 'policy.yaml' with upgrade checks. 2. Deprecate the JSON formatted policy file on the project side via warning in doc and releasenotes. Also replace policy.json to policy.yaml ref from doc and tests. [1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html Change-Id: Ic65d2fd6ce7215b4a47a6fb41b9cbf991f27773b --- .../cli-manage-projects-users-and-roles.rst | 4 ++-- doc/source/admin/identity-concepts.rst | 18 ++++++++--------- doc/source/admin/upgrading.rst | 2 +- doc/source/configuration/policy.rst | 9 +++++++++ doc/source/getting-started/policy_mapping.rst | 2 +- keystone/cmd/status.py | 3 +++ keystone/common/rbac_enforcer/enforcer.py | 8 ++++++++ keystone/conf/__init__.py | 7 +++++++ keystone/tests/unit/test_policy.py | 2 +- keystone/tests/unit/test_v3.py | 2 +- lower-constraints.txt | 6 +++--- ...ormatted-policy-file-95f6307f88358f58.yaml | 20 +++++++++++++++++++ requirements.txt | 6 +++--- 13 files changed, 68 insertions(+), 21 deletions(-) create mode 100644 releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml diff --git a/doc/source/admin/cli-manage-projects-users-and-roles.rst b/doc/source/admin/cli-manage-projects-users-and-roles.rst index f27979d2c2..8d2f83702c 100644 --- a/doc/source/admin/cli-manage-projects-users-and-roles.rst +++ b/doc/source/admin/cli-manage-projects-users-and-roles.rst @@ -10,8 +10,8 @@ define which actions users can perform. You assign roles to user-project pairs. You can define actions for OpenStack service roles in the -``/etc/PROJECT/policy.json`` files. For example, define actions for -Compute service roles in the ``/etc/nova/policy.json`` file. +``/etc/PROJECT/policy.yaml`` files. For example, define actions for +Compute service roles in the ``/etc/nova/policy.yaml`` file. You can manage projects, users, and roles independently from each other. diff --git a/doc/source/admin/identity-concepts.rst b/doc/source/admin/identity-concepts.rst index 3d615c0da1..0f8cfc51ff 100644 --- a/doc/source/admin/identity-concepts.rst +++ b/doc/source/admin/identity-concepts.rst @@ -122,9 +122,9 @@ Identity user management examples: Individual services assign meaning to roles, typically through limiting or granting access to users with the role to the operations that the service supports. Role access is typically - configured in the service's ``policy.json`` file. For example, + configured in the service's ``policy.yaml`` file. For example, to limit Compute access to the ``compute-user`` role, edit the - Compute service's ``policy.json`` file to require this role for + Compute service's ``policy.yaml`` file to require this role for Compute operations. The Identity service assigns a project and a role to a user. You might @@ -139,25 +139,25 @@ A user can have different roles in different projects. For example, Alice might also have the ``admin`` role in the ``Cyberdyne`` project. A user can also have multiple roles in the same project. -The ``/etc/[SERVICE_CODENAME]/policy.json`` file controls the +The ``/etc/[SERVICE_CODENAME]/policy.yaml`` file controls the tasks that users can perform for a given service. For example, the -``/etc/nova/policy.json`` file specifies the access policy for the -Compute service, the ``/etc/glance/policy.json`` file specifies +``/etc/nova/policy.yaml`` file specifies the access policy for the +Compute service, the ``/etc/glance/policy.yaml`` file specifies the access policy for the Image service, and the -``/etc/keystone/policy.json`` file specifies the access policy for +``/etc/keystone/policy.yaml`` file specifies the access policy for the Identity service. -The default ``policy.json`` files in the Compute, Identity, and +The default ``policy.yaml`` files in the Compute, Identity, and Image services recognize only the ``admin`` role. Any user with any role in a project can access all operations that do not require the ``admin`` role. To restrict users from performing operations in, for example, the Compute service, you must create a role in the Identity service and -then modify the ``/etc/nova/policy.json`` file so that this role +then modify the ``/etc/nova/policy.yaml`` file so that this role is required for Compute operations. -For example, the following line in the ``/etc/cinder/policy.json`` +For example, the following line in the ``/etc/cinder/policy.yaml`` file does not restrict which users can create volumes: .. code-block:: none diff --git a/doc/source/admin/upgrading.rst b/doc/source/admin/upgrading.rst index 687dba4e37..709d98dac7 100644 --- a/doc/source/admin/upgrading.rst +++ b/doc/source/admin/upgrading.rst @@ -42,7 +42,7 @@ Plan your upgrade: to re-read the release notes for the previous release (or two!). * Prepare your new configuration files, including ``keystone.conf``, - ``logging.conf``, ``policy.json``, ``keystone-paste.ini``, and anything else + ``logging.conf``, ``policy.yaml``, ``keystone-paste.ini``, and anything else in ``/etc/keystone/``, by customizing the corresponding files from the next release. diff --git a/doc/source/configuration/policy.rst b/doc/source/configuration/policy.rst index daafdea512..3f80c5ee22 100644 --- a/doc/source/configuration/policy.rst +++ b/doc/source/configuration/policy.rst @@ -2,6 +2,15 @@ Policy configuration ==================== +.. warning:: + + JSON formatted policy file is deprecated since Keystone 19.0.0 (Wallaby). + This `oslopolicy-convert-json-to-yaml`__ tool will migrate your existing + JSON-formatted policy file to YAML in a backward-compatible way. + +.. __: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html + + Configuration ~~~~~~~~~~~~~ diff --git a/doc/source/getting-started/policy_mapping.rst b/doc/source/getting-started/policy_mapping.rst index 2975b45b3f..a7cb27cfa7 100644 --- a/doc/source/getting-started/policy_mapping.rst +++ b/doc/source/getting-started/policy_mapping.rst @@ -2,7 +2,7 @@ Mapping of policy target to API =============================== -The following table shows the target in the policy.json file for each API. +The following table shows the target in the policy.yaml file for each API. ========================================================= === Target API diff --git a/keystone/cmd/status.py b/keystone/cmd/status.py index 3585c2e2b1..64b2e62f35 100644 --- a/keystone/cmd/status.py +++ b/keystone/cmd/status.py @@ -12,6 +12,7 @@ from oslo_policy import _checks from oslo_policy import policy +from oslo_upgradecheck import common_checks from oslo_upgradecheck import upgradecheck from keystone.common import driver_hints @@ -86,6 +87,8 @@ class Checks(upgradecheck.UpgradeCommands): check_trust_policies_are_not_empty), ("Check default roles are immutable", check_default_roles_are_immutable), + ("Policy File JSON to YAML Migration", + (common_checks.check_policy_json, {'conf': CONF})), ) diff --git a/keystone/common/rbac_enforcer/enforcer.py b/keystone/common/rbac_enforcer/enforcer.py index ca6a8e7b2b..7add048ce8 100644 --- a/keystone/common/rbac_enforcer/enforcer.py +++ b/keystone/common/rbac_enforcer/enforcer.py @@ -14,6 +14,7 @@ import functools import flask from oslo_log import log +from oslo_policy import opts from oslo_policy import policy as common_policy from oslo_utils import strutils @@ -39,6 +40,13 @@ _POSSIBLE_TARGET_ACTIONS = frozenset([ _ENFORCEMENT_CHECK_ATTR = 'keystone:RBAC:enforcement_called' +# TODO(gmann): Remove setting the default value of config policy_file +# once oslo_policy change the default value to 'policy.yaml'. +# https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49 +DEFAULT_POLICY_FILE = 'policy.yaml' +opts.set_defaults(CONF, DEFAULT_POLICY_FILE) + + class RBACEnforcer(object): """Enforce RBAC on API calls.""" diff --git a/keystone/conf/__init__.py b/keystone/conf/__init__.py index 77c26a11d6..5de0ec1833 100644 --- a/keystone/conf/__init__.py +++ b/keystone/conf/__init__.py @@ -18,6 +18,7 @@ from oslo_log import log from oslo_log import versionutils import oslo_messaging from oslo_middleware import cors +from oslo_policy import opts as policy_opts from osprofiler import opts as profiler from keystone.conf import application_credential @@ -185,6 +186,12 @@ def set_external_opts_defaults(): # configure OSprofiler options profiler.set_defaults(CONF, enabled=False, trace_sqlalchemy=False) + # TODO(gmann): Remove setting the default value of config policy_file + # once oslo_policy change the default value to 'policy.yaml'. + # https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49 + DEFAULT_POLICY_FILE = 'policy.yaml' + policy_opts.set_defaults(cfg.CONF, DEFAULT_POLICY_FILE) + # Oslo.cache is always enabled by default for request-local caching # TODO(morganfainberg): Fix this to not use internal interface when # oslo.cache has proper interface to set defaults added. This is diff --git a/keystone/tests/unit/test_policy.py b/keystone/tests/unit/test_policy.py index 8346549ec2..d58ea6399b 100644 --- a/keystone/tests/unit/test_policy.py +++ b/keystone/tests/unit/test_policy.py @@ -197,7 +197,7 @@ class PolicyJsonTestCase(unit.TestCase): def test_all_targets_documented(self): policy_keys = self._get_default_policy_rules() - # These keys are in the policy.json but aren't targets. + # These keys are in the policy.yaml but aren't targets. policy_rule_keys = [ 'admin_or_owner', 'admin_or_token_subject', 'admin_required', 'owner', 'service_admin_or_token_subject', 'service_or_admin', diff --git a/keystone/tests/unit/test_v3.py b/keystone/tests/unit/test_v3.py index 56ebf4a905..7d6c6b11f5 100644 --- a/keystone/tests/unit/test_v3.py +++ b/keystone/tests/unit/test_v3.py @@ -247,7 +247,7 @@ class RestfulTestCase(unit.SQLDriverOverrides, rest.RestfulTestCase, domain_id=DEFAULT_DOMAIN_ID) self.default_domain_user_id = self.default_domain_user['id'] - # create & grant policy.json's default role for admin_required + # create & grant policy.yaml's default role for admin_required self.role = unit.new_role_ref(name='admin') self.role_id = self.role['id'] PROVIDERS.role_api.create_role(self.role_id, self.role) diff --git a/lower-constraints.txt b/lower-constraints.txt index 9e92201990..431589d020 100644 --- a/lower-constraints.txt +++ b/lower-constraints.txt @@ -25,16 +25,16 @@ oauthlib==0.6.2 os-api-ref==1.4.0 oslo.cache==1.26.0 oslo.concurrency==3.26.0 -oslo.config==5.2.0 +oslo.config==6.8.0 oslo.context==2.22.0 oslo.db==6.0.0 oslo.i18n==3.15.3 oslo.log==3.44.0 oslo.messaging==5.29.0 oslo.middleware==3.31.0 -oslo.policy==3.0.2 +oslo.policy==3.6.0 oslo.serialization==2.18.0 -oslo.upgradecheck==0.1.0 +oslo.upgradecheck==1.3.0 oslo.utils==3.33.0 oslotest==3.2.0 osprofiler==1.4.0 diff --git a/releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml b/releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml new file mode 100644 index 0000000000..c9c5300045 --- /dev/null +++ b/releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml @@ -0,0 +1,20 @@ +--- +upgrade: + - | + The default value of ``[oslo_policy] policy_file`` config option has + been changed from ``policy.json`` to ``policy.yaml``. + Operators who are utilizing customized or previously generated + static policy JSON files (which are not needed by default), should + generate new policy files or convert them in YAML format. Use the + `oslopolicy-convert-json-to-yaml + `_ + tool to convert a JSON to YAML formatted policy file in + backward compatible way. +deprecations: + - | + Use of JSON policy files was deprecated by the ``oslo.policy`` library + during the Victoria development cycle. As a result, this deprecation is + being noted in the Wallaby cycle with an anticipated future removal of support + by ``oslo.policy``. As such operators will need to convert to YAML policy + files. Please see the upgrade notes for details on migration of any + custom policy files. diff --git a/requirements.txt b/requirements.txt index 33a2c423e1..3464fb3fc4 100644 --- a/requirements.txt +++ b/requirements.txt @@ -16,16 +16,16 @@ keystonemiddleware>=7.0.0 # Apache-2.0 bcrypt>=3.1.3 # Apache-2.0 scrypt>=0.8.0 # BSD oslo.cache>=1.26.0 # Apache-2.0 -oslo.config>=5.2.0 # Apache-2.0 +oslo.config>=6.8.0 # Apache-2.0 oslo.context>=2.22.0 # Apache-2.0 oslo.messaging>=5.29.0 # Apache-2.0 oslo.db>=6.0.0 # Apache-2.0 oslo.i18n>=3.15.3 # Apache-2.0 oslo.log>=3.44.0 # Apache-2.0 oslo.middleware>=3.31.0 # Apache-2.0 -oslo.policy>=3.0.2 # Apache-2.0 +oslo.policy>=3.6.0 # Apache-2.0 oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0 -oslo.upgradecheck>=0.1.0 # Apache-2.0 +oslo.upgradecheck>=1.3.0 # Apache-2.0 oslo.utils>=3.33.0 # Apache-2.0 oauthlib>=0.6.2 # BSD pysaml2>=5.0.0