diff --git a/doc/source/admin/archives/adv-features.rst b/doc/source/admin/archives/adv-features.rst index d5a98aa673a..f727f0626d3 100644 --- a/doc/source/admin/archives/adv-features.rst +++ b/doc/source/admin/archives/adv-features.rst @@ -418,7 +418,7 @@ VMware NSX QoS extension The VMware NSX QoS extension rate-limits network ports to guarantee a specific amount of bandwidth for each port. This extension, by default, is only accessible by a project with an admin role but is configurable -through the ``policy.json`` file. To use this extension, create a queue +through the ``policy.yaml`` file. To use this extension, create a queue and specify the min/max bandwidth rates (kbps) and optionally set the QoS Marking and DSCP value (if your network fabric uses these values to make forwarding decisions). Once created, you can associate a queue with diff --git a/doc/source/admin/archives/auth.rst b/doc/source/admin/archives/auth.rst index f4e3f2cf0aa..10a409dc5bd 100644 --- a/doc/source/admin/archives/auth.rst +++ b/doc/source/admin/archives/auth.rst @@ -31,7 +31,7 @@ Networking handles two kind of authorization policies: The actual authorization policies enforced in Networking might vary from deployment to deployment. -The policy engine reads entries from the ``policy.json`` file. The +The policy engine reads entries from the ``policy.yaml`` file. The actual location of this file might vary from distribution to distribution. Entries can be updated while the system is running, and no service restart is required. Every time the policy file is updated, the @@ -84,7 +84,7 @@ terminal rules: in the resource is equal to the project identifier of the user submitting the request. -This extract is from the default ``policy.json`` file: +This extract is from the default ``policy.yaml`` file: - A rule that evaluates successfully if the current user is an administrator or the owner of the resource specified in the request @@ -92,49 +92,42 @@ This extract is from the default ``policy.json`` file: .. code-block:: none - { - "admin_or_owner": "role:admin", - "tenant_id:%(tenant_id)s", - "admin_or_network_owner": "role:admin", - "tenant_id:%(network_tenant_id)s", - "admin_only": "role:admin", - "regular_user": "", - "shared":"field:networks:shared=True", - "default": + "admin_or_owner": "role:admin or tenant_id:%(tenant_id)s" + "admin_or_network_owner": "role:admin or tenant_id:%(network_tenant_id)s" + "admin_only": "role:admin" + "regular_user": "" + "shared": "field:networks:shared=True" - The default policy that is always evaluated if an API operation does - not match any of the policies in ``policy.json``. + not match any of the policies in ``policy.yaml``. .. code-block:: none - "rule:admin_or_owner", - "create_subnet": "rule:admin_or_network_owner", - "get_subnet": "rule:admin_or_owner", - "rule:shared", - "update_subnet": "rule:admin_or_network_owner", - "delete_subnet": "rule:admin_or_network_owner", - "create_network": "", - "get_network": "rule:admin_or_owner", + "default": "rule:admin_or_owner" + "create_subnet": "rule:admin_or_network_owner" + "get_subnet": "rule:admin_or_owner or rule:shared" + "update_subnet": "rule:admin_or_network_owner" + "delete_subnet": "rule:admin_or_network_owner" + "create_network": "" - This policy evaluates successfully if either *admin_or_owner*, or *shared* evaluates successfully. .. code-block:: none - "rule:shared", - "create_network:shared": "rule:admin_only" + "get_network": "rule:admin_or_owner or rule:shared" + "create_network:shared": "rule:admin_only" - This policy restricts the ability to manipulate the *shared* attribute for a network to administrators only. .. code-block:: none - , - "update_network": "rule:admin_or_owner", - "delete_network": "rule:admin_or_owner", - "create_port": "", - "create_port:mac_address": "rule:admin_or_network_owner", - "create_port:fixed_ips": + "update_network": "rule:admin_or_owner" + "delete_network": "rule:admin_or_owner" + "create_port": "" + "create_port:mac_address": "rule:admin_or_network_owner" + "create_port:fixed_ips": "rule:admin_or_network_owner" - This policy restricts the ability to manipulate the *mac_address* attribute for a port only to administrators and the owner of the @@ -142,11 +135,9 @@ This extract is from the default ``policy.json`` file: .. code-block:: none - "rule:admin_or_network_owner", - "get_port": "rule:admin_or_owner", - "update_port": "rule:admin_or_owner", - "delete_port": "rule:admin_or_owner" - } + "get_port": "rule:admin_or_owner" + "update_port": "rule:admin_or_owner" + "delete_port": "rule:admin_or_owner" In some cases, some operations are restricted to administrators only. This example shows you how to modify a policy file to permit project to @@ -155,21 +146,20 @@ perform all other operations: .. code-block:: none - { - "admin_or_owner": "role:admin", "tenant_id:%(tenant_id)s", - "admin_only": "role:admin", "regular_user": "", - "default": "rule:admin_only", - "create_subnet": "rule:admin_only", - "get_subnet": "rule:admin_or_owner", - "update_subnet": "rule:admin_only", - "delete_subnet": "rule:admin_only", - "create_network": "", - "get_network": "rule:admin_or_owner", - "create_network:shared": "rule:admin_only", - "update_network": "rule:admin_or_owner", - "delete_network": "rule:admin_or_owner", - "create_port": "rule:admin_only", - "get_port": "rule:admin_or_owner", - "update_port": "rule:admin_only", + "admin_or_owner": "role:admin or tenant_id:%(tenant_id)s" + "admin_only": "role:admin" + "regular_user": "" + "default": "rule:admin_only" + "create_subnet": "rule:admin_only" + "get_subnet": "rule:admin_or_owner" + "update_subnet": "rule:admin_only" + "delete_subnet": "rule:admin_only" + "create_network": "" + "get_network": "rule:admin_or_owner" + "create_network:shared": "rule:admin_only" + "update_network": "rule:admin_or_owner" + "delete_network": "rule:admin_or_owner" + "create_port": "rule:admin_only" + "get_port": "rule:admin_or_owner" + "update_port": "rule:admin_only" "delete_port": "rule:admin_only" - } diff --git a/doc/source/admin/archives/config-agents.rst b/doc/source/admin/archives/config-agents.rst index a813e25a6e8..634e84d035a 100644 --- a/doc/source/admin/archives/config-agents.rst +++ b/doc/source/admin/archives/config-agents.rst @@ -372,11 +372,11 @@ correctly using these > cd C:\OpenStack\neutron\ > python setup.py install -#. Copy the ``policy.json`` file: +#. Copy the ``policy.yaml`` file: .. code-block:: console - > xcopy C:\OpenStack\neutron\etc\policy.json C:\etc\ + > xcopy C:\OpenStack\neutron\etc\policy.yaml C:\etc\ #. Create the ``C:\etc\neutron-hyperv-agent.conf`` file and add the proper configuration options and the `Hyper-V related @@ -386,7 +386,7 @@ correctly using these [DEFAULT] control_exchange = neutron - policy_file = C:\etc\policy.json + policy_file = C:\etc\policy.yaml rpc_backend = neutron.openstack.common.rpc.impl_kombu rabbit_host = IP_ADDRESS rabbit_port = 5672 diff --git a/doc/source/admin/config-logging.rst b/doc/source/admin/config-logging.rst index 7d4aee002d8..6cd7ca2afec 100644 --- a/doc/source/admin/config-logging.rst +++ b/doc/source/admin/config-logging.rst @@ -80,16 +80,16 @@ To enable the logging service, follow the below steps. - If we don't specify ``local_output_log_base``, logged packets will be stored in system journal like ``/var/log/syslog`` by default. -Trusted projects policy.json configuration +Trusted projects policy.yaml configuration ---------------------------------------------- -With the default ``/etc/neutron/policy.json``, administrators must set up +With the default ``/etc/neutron/policy.yaml``, administrators must set up resource logging on behalf of the cloud projects. If projects are trusted to administer their own loggable resources in their -cloud, neutron's policy file ``policy.json`` can be modified to allow this. +cloud, neutron's policy file ``policy.yaml`` can be modified to allow this. -Modify ``/etc/neutron/policy.json`` entries as follows: +Modify ``/etc/neutron/policy.yaml`` entries as follows: .. code-block:: none diff --git a/doc/source/admin/config-qos.rst b/doc/source/admin/config-qos.rst index 69f5cf356ef..5025ad26960 100644 --- a/doc/source/admin/config-qos.rst +++ b/doc/source/admin/config-qos.rst @@ -248,13 +248,13 @@ not automatically copied to the outer header. If the ``dscp_inherit`` option is set to true, the previous ``dscp`` option is overwritten. -Trusted projects policy.json configuration +Trusted projects policy.yaml configuration ------------------------------------------ If projects are trusted to administrate their own QoS policies in -your cloud, neutron's file ``policy.json`` can be modified to allow this. +your cloud, neutron's file ``policy.yaml`` can be modified to allow this. -Modify ``/etc/neutron/policy.json`` policy entries as follows: +Modify ``/etc/neutron/policy.yaml`` policy entries as follows: .. code-block:: none @@ -294,12 +294,12 @@ To enable minimum bandwidth rule: User workflow ~~~~~~~~~~~~~ -QoS policies are only created by admins with the default ``policy.json``. +QoS policies are only created by admins with the default ``policy.yaml``. Therefore, you should have the cloud operator set them up on behalf of the cloud projects. If projects are trusted to create their own policies, check the trusted -projects ``policy.json`` configuration section. +projects ``policy.yaml`` configuration section. First, create a QoS policy and its bandwidth limit rule: diff --git a/doc/source/admin/config-rbac.rst b/doc/source/admin/config-rbac.rst index 7b0c1a1d610..962ff97f4a2 100644 --- a/doc/source/admin/config-rbac.rst +++ b/doc/source/admin/config-rbac.rst @@ -701,11 +701,11 @@ as any other RBAC ``access_as_external`` policy. Preventing regular users from sharing objects with each other ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The default ``policy.json`` file will not allow regular +The default ``policy.yaml`` file will not allow regular users to share objects with every other project using a wildcard; however, it will allow them to share objects with specific project IDs. If an operator wants to prevent normal users from doing this, the -``"create_rbac_policy":`` entry in ``policy.json`` can be adjusted +``"create_rbac_policy":`` entry in ``policy.yaml`` can be adjusted from ``""`` to ``"rule:admin_only"``. diff --git a/doc/source/admin/intro-os-networking.rst b/doc/source/admin/intro-os-networking.rst index 9b13d55d5ea..4ec4999a01e 100644 --- a/doc/source/admin/intro-os-networking.rst +++ b/doc/source/admin/intro-os-networking.rst @@ -71,7 +71,7 @@ at the cost of flexibility. By default only administrators can create or update provider networks because they require configuration of physical network infrastructure. It is possible to change the user who is allowed to create or update provider networks with the following parameters of -``policy.json``: +``policy.yaml``: * ``create_network:provider:physical_network`` * ``update_network:provider:physical_network`` diff --git a/doc/source/configuration/policy.rst b/doc/source/configuration/policy.rst index 8ee5872d627..616dd986aa6 100644 --- a/doc/source/configuration/policy.rst +++ b/doc/source/configuration/policy.rst @@ -2,6 +2,14 @@ Policy Reference ================ +.. warning:: + + JSON formatted policy file is deprecated since Neutron 18.0.0 (Wallaby). + This `oslopolicy-convert-json-to-yaml`__ tool will migrate your existing + JSON-formatted policy file to YAML in a backward-compatible way. + +.. __: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html + Neutron, like most OpenStack projects, uses a policy language to restrict permissions on REST API actions. diff --git a/doc/source/contributor/contribute.rst b/doc/source/contributor/contribute.rst index b45c32273a3..0c0ef9a15ae 100644 --- a/doc/source/contributor/contribute.rst +++ b/doc/source/contributor/contribute.rst @@ -615,7 +615,7 @@ Other repo-split items (These are still TBD.) -* Splitting policy.json? **ToDo** Armando will investigate. +* Splitting policy.yaml? **ToDo** Armando will investigate. * Generic instructions (or a template) for installing an out-of-tree plugin or driver for Neutron. Possibly something for the networking guide, and/or a diff --git a/doc/source/contributor/internals/policy.rst b/doc/source/contributor/internals/policy.rst index 037352fcbeb..58e36777737 100644 --- a/doc/source/contributor/internals/policy.rst +++ b/doc/source/contributor/internals/policy.rst @@ -86,7 +86,7 @@ The ``_build_match_rule`` routine returns a ``oslo_policy.RuleCheck`` instance built in the following way: * Always add a check for the action being performed. This will match - a policy like create_network in ``policy.json``; + a policy like create_network in ``policy.yaml``; * Return for ``GET`` operations; more detailed checks will be performed anyway when building the response; * For each attribute which has been explicitly specified in the request @@ -226,7 +226,7 @@ keyword, and provides a way to perform fine grained checks on resource attributes. For instance, using this class of rules it is possible to specify a rule for granting every project read access to shared resources. -In policy.json, a FieldCheck rules is specified in the following way:: +In policy.yaml, a FieldCheck rules is specified in the following way:: > field: := @@ -291,7 +291,7 @@ Notes * There is no way at the moment to specify an ``OR`` relationship between two attributes of a given resource (eg.: ``port.name == 'meh' or port.status == 'DOWN'``), unless the rule with the or condition is explicitly - added to the policy.json file. + added to the policy.yaml file. * ``OwnerCheck`` performs a plugin access; this will likely require a database access, but since the behaviour is implementation specific it might also imply a round-trip to the backend. This class of checks, when involving diff --git a/doc/source/contributor/internals/quota.rst b/doc/source/contributor/internals/quota.rst index eebbc78d199..7f8bfd40eb4 100644 --- a/doc/source/contributor/internals/quota.rst +++ b/doc/source/contributor/internals/quota.rst @@ -112,7 +112,7 @@ an attribute is added for every resource managed by the quota engine. Request authorisation is performed in this controller, and only 'admin' users are allowed to modify quotas for projects. As the neutron policy engine is not used, it is not possible to configure which users should be allowed to manage -quotas using policy.json. +quotas using policy.yaml. The driver operations dealing with quota management are: diff --git a/doc/source/contributor/policies/bugs.rst b/doc/source/contributor/policies/bugs.rst index e97db82d644..10b7003c6fd 100644 --- a/doc/source/contributor/policies/bugs.rst +++ b/doc/source/contributor/policies/bugs.rst @@ -378,7 +378,7 @@ more will be added over time if needed. +-------------------------------+-----------------------------------------+--------------------------+ | Tag | Description | Contact | +===============================+=========================================+==========================+ -| access-control_ | A bug affecting RBAC and policy.json | Miguel Lavalle | +| access-control_ | A bug affecting RBAC and policy.yaml | Miguel Lavalle | +-------------------------------+-----------------------------------------+--------------------------+ | api_ | A bug affecting the API layer | Akihiro Motoki | +-------------------------------+-----------------------------------------+--------------------------+ diff --git a/lower-constraints.txt b/lower-constraints.txt index a0dada5f670..4aea3cc6a7d 100644 --- a/lower-constraints.txt +++ b/lower-constraints.txt @@ -61,7 +61,7 @@ os-vif==1.15.1 osc-lib==1.8.0 oslo.cache==1.26.0 oslo.concurrency==3.26.0 -oslo.config==6.0.0 +oslo.config==6.8.0 oslo.context==2.22.0 oslo.db==4.44.0 oslo.i18n==3.20.0 @@ -74,8 +74,8 @@ oslo.reports==1.18.0 oslo.rootwrap==5.8.0 oslo.serialization==2.25.0 oslo.service==1.31.0 -oslo.upgradecheck==0.1.0 -oslo.utils==4.4.0 +oslo.upgradecheck==1.3.0 +oslo.utils==4.5.0 oslo.versionedobjects==1.35.1 oslotest==3.2.0 osprofiler==2.3.0 @@ -112,9 +112,8 @@ pytz==2013.6 PyYAML==5.3.1 reno==3.1.0 repoze.lru==0.7 -requests==2.14.2 +requests==2.18.0 requestsexceptions==1.2.0 -rfc3986==0.3.1 Routes==2.3.1 simplejson==3.5.1 smmap==0.9.0 diff --git a/neutron/cmd/upgrade_checks/checks.py b/neutron/cmd/upgrade_checks/checks.py index 5d455afb79f..a478bc6d0f4 100644 --- a/neutron/cmd/upgrade_checks/checks.py +++ b/neutron/cmd/upgrade_checks/checks.py @@ -17,6 +17,7 @@ from neutron_lib import context from neutron_lib.db import model_query from oslo_config import cfg from oslo_serialization import jsonutils +from oslo_upgradecheck import common_checks from oslo_upgradecheck import upgradecheck from sqlalchemy import or_ @@ -97,6 +98,8 @@ class CoreChecks(base.BaseChecks): self.nic_switch_agent_min_kernel_check), (_("VLAN allocations valid segmentation ID check"), self.vlan_allocations_segid_check), + (_('Policy File JSON to YAML Migration'), + (common_checks.check_policy_json, {'conf': cfg.CONF})), ] @staticmethod diff --git a/neutron/common/config.py b/neutron/common/config.py index 40d289f81ff..5eaae86ed64 100644 --- a/neutron/common/config.py +++ b/neutron/common/config.py @@ -26,6 +26,7 @@ from oslo_config import cfg from oslo_log import log as logging import oslo_messaging from oslo_middleware import cors +from oslo_policy import opts from oslo_service import wsgi from neutron._i18n import _ @@ -138,6 +139,11 @@ def set_config_defaults(): """This method updates all configuration default values.""" set_cors_middleware_defaults() + # TODO(gmann): Remove setting the default value of config policy_file + # once oslo_policy change the default value to 'policy.yaml'. + # https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49 + opts.set_defaults(cfg.CONF, policy.DEFAULT_POLICY_FILE) + def set_cors_middleware_defaults(): """Update default configuration options for oslo.middleware.""" diff --git a/neutron/policy.py b/neutron/policy.py index ce6c62949fa..18e965ffa5c 100644 --- a/neutron/policy.py +++ b/neutron/policy.py @@ -28,6 +28,7 @@ from neutron_lib.services import constants as service_const from oslo_config import cfg from oslo_db import exception as db_exc from oslo_log import log as logging +from oslo_policy import opts from oslo_policy import policy from oslo_utils import excutils import stevedore @@ -51,6 +52,13 @@ _RESOURCE_FOREIGN_KEYS = { } +# TODO(gmann): Remove setting the default value of config policy_file +# once oslo_policy change the default value to 'policy.yaml'. +# https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49 +DEFAULT_POLICY_FILE = 'policy.yaml' +opts.set_defaults(cfg.CONF, DEFAULT_POLICY_FILE) + + def reset(): global _ENFORCER if _ENFORCER: @@ -226,7 +234,7 @@ def _build_match_rule(action, target, pluralized): # This check is registered as 'tenant_id' so that it can override # GenericCheck which was used for validating parent resource ownership. # This will prevent us from having to handling backward compatibility -# for policy.json +# for policy.yaml # TODO(salv-orlando): Reinstate GenericCheck for simple tenant_id checks @policy.register('tenant_id') class OwnerCheck(policy.Check): diff --git a/neutron/tests/fullstack/resources/config.py b/neutron/tests/fullstack/resources/config.py index 9d469cefe41..a3a3db85f15 100644 --- a/neutron/tests/fullstack/resources/config.py +++ b/neutron/tests/fullstack/resources/config.py @@ -100,7 +100,7 @@ class NeutronConfigFixture(ConfigFixture): self.config['DEFAULT']['api_paste_config'] = ( self._generate_api_paste()) - policy_file = self._generate_policy_json() + policy_file = self._generate_policy_yaml() if policy_file: self.config['oslo_policy'] = {'policy_file': policy_file} @@ -154,8 +154,8 @@ class NeutronConfigFixture(ConfigFixture): def _generate_api_paste(self): return c_helpers.find_sample_file('api-paste.ini') - def _generate_policy_json(self): - return c_helpers.find_sample_file('policy.json') + def _generate_policy_yaml(self): + return c_helpers.find_sample_file('policy.yaml') def get_host(self): return self.config['DEFAULT']['host'] diff --git a/neutron/tests/unit/test_policy.py b/neutron/tests/unit/test_policy.py index d6f0c55bede..f5010fc7492 100644 --- a/neutron/tests/unit/test_policy.py +++ b/neutron/tests/unit/test_policy.py @@ -157,7 +157,7 @@ class DefaultPolicyTestCase(base.BaseTestCase): def setUp(self): super(DefaultPolicyTestCase, self).setUp() - tmpfilename = self.get_temp_file_path('policy.json') + tmpfilename = self.get_temp_file_path('policy.yaml') self.rules = { "default": '', "example:exist": '!', diff --git a/releasenotes/notes/deprecate-json-formatted-policy-file-4be7b9394a2ae40f.yaml b/releasenotes/notes/deprecate-json-formatted-policy-file-4be7b9394a2ae40f.yaml new file mode 100644 index 00000000000..c9c53000458 --- /dev/null +++ b/releasenotes/notes/deprecate-json-formatted-policy-file-4be7b9394a2ae40f.yaml @@ -0,0 +1,20 @@ +--- +upgrade: + - | + The default value of ``[oslo_policy] policy_file`` config option has + been changed from ``policy.json`` to ``policy.yaml``. + Operators who are utilizing customized or previously generated + static policy JSON files (which are not needed by default), should + generate new policy files or convert them in YAML format. Use the + `oslopolicy-convert-json-to-yaml + `_ + tool to convert a JSON to YAML formatted policy file in + backward compatible way. +deprecations: + - | + Use of JSON policy files was deprecated by the ``oslo.policy`` library + during the Victoria development cycle. As a result, this deprecation is + being noted in the Wallaby cycle with an anticipated future removal of support + by ``oslo.policy``. As such operators will need to convert to YAML policy + files. Please see the upgrade notes for details on migration of any + custom policy files. diff --git a/requirements.txt b/requirements.txt index 5a1cb5b1e86..c346719c499 100644 --- a/requirements.txt +++ b/requirements.txt @@ -11,7 +11,7 @@ decorator>=3.4.0 # BSD eventlet>=0.22.1 # MIT pecan>=1.3.2 # BSD httplib2>=0.9.1 # MIT -requests>=2.14.2 # Apache-2.0 +requests>=2.18.0 # Apache-2.0 Jinja2>=2.10 # BSD License (3 clause) keystonemiddleware>=5.1.0 # Apache-2.0 netaddr>=0.7.18 # BSD @@ -26,7 +26,7 @@ alembic>=0.9.6 # MIT stevedore>=1.20.0 # Apache-2.0 oslo.cache>=1.26.0 # Apache-2.0 oslo.concurrency>=3.26.0 # Apache-2.0 -oslo.config>=6.0.0 # Apache-2.0 +oslo.config>=6.8.0 # Apache-2.0 oslo.context>=2.22.0 # Apache-2.0 oslo.db>=4.44.0 # Apache-2.0 oslo.i18n>=3.20.0 # Apache-2.0 @@ -39,8 +39,8 @@ oslo.reports>=1.18.0 # Apache-2.0 oslo.rootwrap>=5.8.0 # Apache-2.0 oslo.serialization>=2.25.0 # Apache-2.0 oslo.service>=1.31.0 # Apache-2.0 -oslo.upgradecheck>=0.1.0 # Apache-2.0 -oslo.utils>=4.4.0 # Apache-2.0 +oslo.upgradecheck>=1.3.0 # Apache-2.0 +oslo.utils>=4.5.0 # Apache-2.0 oslo.versionedobjects>=1.35.1 # Apache-2.0 osprofiler>=2.3.0 # Apache-2.0 os-ken >= 0.3.0 # Apache-2.0 diff --git a/setup.cfg b/setup.cfg index 38c4a48846e..133ec719fc3 100644 --- a/setup.cfg +++ b/setup.cfg @@ -162,7 +162,7 @@ oslo.config.opts = neutron.ovn.metadata.agent = neutron.conf.agent.ovn.metadata.config:list_metadata_agent_opts nova.auth = neutron.opts:list_auth_opts oslo.config.opts.defaults = - neutron = neutron.common.config:set_cors_middleware_defaults + neutron = neutron.common.config:set_config_defaults oslo.policy.enforcer = neutron = neutron.policy:get_enforcer oslo.policy.policies =