openstack/neutron
Code Issues Proposed changes

[goal] Deprecate the JSON formatted policy file

Browse Source 
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also replace policy.json to policy.yaml ref from doc and tests.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: I0dbb8484e749e645627756e88ec79c1b26a6414a
This commit is contained in:
Ghanshyam Mann 2020-11-26 21:16:46 -06:00 committed by Slawek Kaplonski
parent 0a852dab99
commit fe413fe01d
21 changed files with 121 additions and 87 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
doc/source/admin/archives/adv-features.rst
View File

@ -418,7 +418,7 @@ VMware NSX QoS extension
The VMware NSX QoS extension rate-limits network ports to guarantee a
specific amount of bandwidth for each port. This extension, by default,
is only accessible by a project with an admin role but is configurable
through the ``policy.json`` file. To use this extension, create a queue
through the ``policy.yaml`` file. To use this extension, create a queue
and specify the min/max bandwidth rates (kbps) and optionally set the
QoS Marking and DSCP value (if your network fabric uses these values to
make forwarding decisions). Once created, you can associate a queue with

90
doc/source/admin/archives/auth.rst
View File

@ -31,7 +31,7 @@ Networking handles two kind of authorization policies:
The actual authorization policies enforced in Networking might vary
from deployment to deployment.
The policy engine reads entries from the ``policy.json`` file. The
The policy engine reads entries from the ``policy.yaml`` file. The
actual location of this file might vary from distribution to
distribution. Entries can be updated while the system is running, and no
service restart is required. Every time the policy file is updated, the
@ -84,7 +84,7 @@ terminal rules:
in the resource is equal to the project identifier of the user
submitting the request.
This extract is from the default ``policy.json`` file:
This extract is from the default ``policy.yaml`` file:
- A rule that evaluates successfully if the current user is an
administrator or the owner of the resource specified in the request
@ -92,49 +92,42 @@ This extract is from the default ``policy.json`` file:
.. code-block:: none
{
"admin_or_owner": "role:admin",
"tenant_id:%(tenant_id)s",
"admin_or_network_owner": "role:admin",
"tenant_id:%(network_tenant_id)s",
"admin_only": "role:admin",
"regular_user": "",
"shared":"field:networks:shared=True",
"default":
"admin_or_owner": "role:admin or tenant_id:%(tenant_id)s"
"admin_or_network_owner": "role:admin or tenant_id:%(network_tenant_id)s"
"admin_only": "role:admin"
"regular_user": ""
"shared": "field:networks:shared=True"
- The default policy that is always evaluated if an API operation does
not match any of the policies in ``policy.json``.
not match any of the policies in ``policy.yaml``.
.. code-block:: none
"rule:admin_or_owner",
"create_subnet": "rule:admin_or_network_owner",
"get_subnet": "rule:admin_or_owner",
"rule:shared",
"update_subnet": "rule:admin_or_network_owner",
"delete_subnet": "rule:admin_or_network_owner",
"create_network": "",
"get_network": "rule:admin_or_owner",
"default": "rule:admin_or_owner"
"create_subnet": "rule:admin_or_network_owner"
"get_subnet": "rule:admin_or_owner or rule:shared"
"update_subnet": "rule:admin_or_network_owner"
"delete_subnet": "rule:admin_or_network_owner"
"create_network": ""
- This policy evaluates successfully if either *admin_or_owner*, or
*shared* evaluates successfully.
.. code-block:: none
"rule:shared",
"create_network:shared": "rule:admin_only"
"get_network": "rule:admin_or_owner or rule:shared"
"create_network:shared": "rule:admin_only"
- This policy restricts the ability to manipulate the *shared*
attribute for a network to administrators only.
.. code-block:: none
,
"update_network": "rule:admin_or_owner",
"delete_network": "rule:admin_or_owner",
"create_port": "",
"create_port:mac_address": "rule:admin_or_network_owner",
"create_port:fixed_ips":
"update_network": "rule:admin_or_owner"
"delete_network": "rule:admin_or_owner"
"create_port": ""
"create_port:mac_address": "rule:admin_or_network_owner"
"create_port:fixed_ips": "rule:admin_or_network_owner"
- This policy restricts the ability to manipulate the *mac_address*
attribute for a port only to administrators and the owner of the
@ -142,11 +135,9 @@ This extract is from the default ``policy.json`` file:
.. code-block:: none
"rule:admin_or_network_owner",
"get_port": "rule:admin_or_owner",
"update_port": "rule:admin_or_owner",
"delete_port": "rule:admin_or_owner"
}
"get_port": "rule:admin_or_owner"
"update_port": "rule:admin_or_owner"
"delete_port": "rule:admin_or_owner"
In some cases, some operations are restricted to administrators only.
This example shows you how to modify a policy file to permit project to
@ -155,21 +146,20 @@ perform all other operations:
.. code-block:: none
{
"admin_or_owner": "role:admin", "tenant_id:%(tenant_id)s",
"admin_only": "role:admin", "regular_user": "",
"default": "rule:admin_only",
"create_subnet": "rule:admin_only",
"get_subnet": "rule:admin_or_owner",
"update_subnet": "rule:admin_only",
"delete_subnet": "rule:admin_only",
"create_network": "",
"get_network": "rule:admin_or_owner",
"create_network:shared": "rule:admin_only",
"update_network": "rule:admin_or_owner",
"delete_network": "rule:admin_or_owner",
"create_port": "rule:admin_only",
"get_port": "rule:admin_or_owner",
"update_port": "rule:admin_only",
"admin_or_owner": "role:admin or tenant_id:%(tenant_id)s"
"admin_only": "role:admin"
"regular_user": ""
"default": "rule:admin_only"
"create_subnet": "rule:admin_only"
"get_subnet": "rule:admin_or_owner"
"update_subnet": "rule:admin_only"
"delete_subnet": "rule:admin_only"
"create_network": ""
"get_network": "rule:admin_or_owner"
"create_network:shared": "rule:admin_only"
"update_network": "rule:admin_or_owner"
"delete_network": "rule:admin_or_owner"
"create_port": "rule:admin_only"
"get_port": "rule:admin_or_owner"
"update_port": "rule:admin_only"
"delete_port": "rule:admin_only"
}

6
doc/source/admin/archives/config-agents.rst
View File

@ -372,11 +372,11 @@ correctly using these
> cd C:\OpenStack\neutron\
> python setup.py install
#. Copy the ``policy.json`` file:
#. Copy the ``policy.yaml`` file:
.. code-block:: console
> xcopy C:\OpenStack\neutron\etc\policy.json C:\etc\
> xcopy C:\OpenStack\neutron\etc\policy.yaml C:\etc\
#. Create the ``C:\etc\neutron-hyperv-agent.conf`` file and add the proper
configuration options and the `Hyper-V related
@ -386,7 +386,7 @@ correctly using these
[DEFAULT]
control_exchange = neutron
policy_file = C:\etc\policy.json
policy_file = C:\etc\policy.yaml
rpc_backend = neutron.openstack.common.rpc.impl_kombu
rabbit_host = IP_ADDRESS
rabbit_port = 5672

8
doc/source/admin/config-logging.rst
View File

@ -80,16 +80,16 @@ To enable the logging service, follow the below steps.
- If we don't specify ``local_output_log_base``, logged packets will be
stored in system journal like ``/var/log/syslog`` by default.
Trusted projects policy.json configuration
Trusted projects policy.yaml configuration
----------------------------------------------
With the default ``/etc/neutron/policy.json``, administrators must set up
With the default ``/etc/neutron/policy.yaml``, administrators must set up
resource logging on behalf of the cloud projects.
If projects are trusted to administer their own loggable resources in their
cloud, neutron's policy file ``policy.json`` can be modified to allow this.
cloud, neutron's policy file ``policy.yaml`` can be modified to allow this.
Modify ``/etc/neutron/policy.json`` entries as follows:
Modify ``/etc/neutron/policy.yaml`` entries as follows:
.. code-block:: none

10
doc/source/admin/config-qos.rst
View File

@ -248,13 +248,13 @@ not automatically copied to the outer header.
If the ``dscp_inherit`` option is set to true, the previous ``dscp`` option
is overwritten.
Trusted projects policy.json configuration
Trusted projects policy.yaml configuration
------------------------------------------
If projects are trusted to administrate their own QoS policies in
your cloud, neutron's file ``policy.json`` can be modified to allow this.
your cloud, neutron's file ``policy.yaml`` can be modified to allow this.
Modify ``/etc/neutron/policy.json`` policy entries as follows:
Modify ``/etc/neutron/policy.yaml`` policy entries as follows:
.. code-block:: none
@ -294,12 +294,12 @@ To enable minimum bandwidth rule:
User workflow
~~~~~~~~~~~~~
QoS policies are only created by admins with the default ``policy.json``.
QoS policies are only created by admins with the default ``policy.yaml``.
Therefore, you should have the cloud operator set them up on
behalf of the cloud projects.
If projects are trusted to create their own policies, check the trusted
projects ``policy.json`` configuration section.
projects ``policy.yaml`` configuration section.
First, create a QoS policy and its bandwidth limit rule:

4
doc/source/admin/config-rbac.rst
View File

@ -701,11 +701,11 @@ as any other RBAC ``access_as_external`` policy.
Preventing regular users from sharing objects with each other
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The default ``policy.json`` file will not allow regular
The default ``policy.yaml`` file will not allow regular
users to share objects with every other project using a wildcard;
however, it will allow them to share objects with specific project
IDs.
If an operator wants to prevent normal users from doing this, the
``"create_rbac_policy":`` entry in ``policy.json`` can be adjusted
``"create_rbac_policy":`` entry in ``policy.yaml`` can be adjusted
from ``""`` to ``"rule:admin_only"``.

2
doc/source/admin/intro-os-networking.rst
View File

@ -71,7 +71,7 @@ at the cost of flexibility. By default only administrators can create or
update provider networks because they require configuration of physical
network infrastructure. It is possible to change the user who is allowed to
create or update provider networks with the following parameters of
``policy.json``:
``policy.yaml``:
* ``create_network:provider:physical_network``
* ``update_network:provider:physical_network``

8
doc/source/configuration/policy.rst
View File

@ -2,6 +2,14 @@
Policy Reference
================
.. warning::
JSON formatted policy file is deprecated since Neutron 18.0.0 (Wallaby).
This `oslopolicy-convert-json-to-yaml`__ tool will migrate your existing
JSON-formatted policy file to YAML in a backward-compatible way.
.. __: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html
Neutron, like most OpenStack projects, uses a policy language to restrict
permissions on REST API actions.

2
doc/source/contributor/contribute.rst
View File

@ -615,7 +615,7 @@ Other repo-split items
(These are still TBD.)
* Splitting policy.json? **ToDo** Armando will investigate.
* Splitting policy.yaml? **ToDo** Armando will investigate.
* Generic instructions (or a template) for installing an out-of-tree plugin or
driver for Neutron. Possibly something for the networking guide, and/or a

6
doc/source/contributor/internals/policy.rst
View File

@ -86,7 +86,7 @@ The ``_build_match_rule`` routine returns a ``oslo_policy.RuleCheck`` instance
built in the following way:
* Always add a check for the action being performed. This will match
a policy like create_network in ``policy.json``;
a policy like create_network in ``policy.yaml``;
* Return for ``GET`` operations; more detailed checks will be performed anyway
when building the response;
* For each attribute which has been explicitly specified in the request
@ -226,7 +226,7 @@ keyword, and provides a way to perform fine grained checks on resource
attributes. For instance, using this class of rules it is possible to specify
a rule for granting every project read access to shared resources.
In policy.json, a FieldCheck rules is specified in the following way::
In policy.yaml, a FieldCheck rules is specified in the following way::
> field: <resource>:<field>=<value>
@ -291,7 +291,7 @@ Notes
* There is no way at the moment to specify an ``OR`` relationship between two
attributes of a given resource (eg.: ``port.name == 'meh' or
port.status == 'DOWN'``), unless the rule with the or condition is explicitly
added to the policy.json file.
added to the policy.yaml file.
* ``OwnerCheck`` performs a plugin access; this will likely require a database
access, but since the behaviour is implementation specific it might also
imply a round-trip to the backend. This class of checks, when involving

2
doc/source/contributor/internals/quota.rst
View File

@ -112,7 +112,7 @@ an attribute is added for every resource managed by the quota engine.
Request authorisation is performed in this controller, and only 'admin' users
are allowed to modify quotas for projects. As the neutron policy engine is not
used, it is not possible to configure which users should be allowed to manage
quotas using policy.json.
quotas using policy.yaml.
The driver operations dealing with quota management are:

2
doc/source/contributor/policies/bugs.rst
View File

@ -378,7 +378,7 @@ more will be added over time if needed.
+-------------------------------+-----------------------------------------+--------------------------+
| Tag | Description | Contact |
+===============================+=========================================+==========================+
| access-control_ | A bug affecting RBAC and policy.json | Miguel Lavalle |
| access-control_ | A bug affecting RBAC and policy.yaml | Miguel Lavalle |
+-------------------------------+-----------------------------------------+--------------------------+
| api_ | A bug affecting the API layer | Akihiro Motoki |
+-------------------------------+-----------------------------------------+--------------------------+

9
lower-constraints.txt
View File

@ -61,7 +61,7 @@ os-vif==1.15.1
osc-lib==1.8.0
oslo.cache==1.26.0
oslo.concurrency==3.26.0
oslo.config==6.0.0
oslo.config==6.8.0
oslo.context==2.22.0
oslo.db==4.44.0
oslo.i18n==3.20.0
@ -74,8 +74,8 @@ oslo.reports==1.18.0
oslo.rootwrap==5.8.0
oslo.serialization==2.25.0
oslo.service==1.31.0
oslo.upgradecheck==0.1.0
oslo.utils==4.4.0
oslo.upgradecheck==1.3.0
oslo.utils==4.5.0
oslo.versionedobjects==1.35.1
oslotest==3.2.0
osprofiler==2.3.0
@ -112,9 +112,8 @@ pytz==2013.6
PyYAML==5.3.1
reno==3.1.0
repoze.lru==0.7
requests==2.14.2
requests==2.18.0
requestsexceptions==1.2.0
rfc3986==0.3.1
Routes==2.3.1
simplejson==3.5.1
smmap==0.9.0

3
neutron/cmd/upgrade_checks/checks.py
View File

@ -17,6 +17,7 @@ from neutron_lib import context
from neutron_lib.db import model_query
from oslo_config import cfg
from oslo_serialization import jsonutils
from oslo_upgradecheck import common_checks
from oslo_upgradecheck import upgradecheck
from sqlalchemy import or_
@ -97,6 +98,8 @@ class CoreChecks(base.BaseChecks):
self.nic_switch_agent_min_kernel_check),
(_("VLAN allocations valid segmentation ID check"),
self.vlan_allocations_segid_check),
(_('Policy File JSON to YAML Migration'),
(common_checks.check_policy_json, {'conf': cfg.CONF})),
]
@staticmethod

6
neutron/common/config.py
View File

@ -26,6 +26,7 @@ from oslo_config import cfg
from oslo_log import log as logging
import oslo_messaging
from oslo_middleware import cors
from oslo_policy import opts
from oslo_service import wsgi
from neutron._i18n import _
@ -138,6 +139,11 @@ def set_config_defaults():
"""This method updates all configuration default values."""
set_cors_middleware_defaults()
# TODO(gmann): Remove setting the default value of config policy_file
# once oslo_policy change the default value to 'policy.yaml'.
# https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49
opts.set_defaults(cfg.CONF, policy.DEFAULT_POLICY_FILE)
def set_cors_middleware_defaults():
"""Update default configuration options for oslo.middleware."""

10
neutron/policy.py
View File

@ -28,6 +28,7 @@ from neutron_lib.services import constants as service_const
from oslo_config import cfg
from oslo_db import exception as db_exc
from oslo_log import log as logging
from oslo_policy import opts
from oslo_policy import policy
from oslo_utils import excutils
import stevedore
@ -51,6 +52,13 @@ _RESOURCE_FOREIGN_KEYS = {
}
# TODO(gmann): Remove setting the default value of config policy_file
# once oslo_policy change the default value to 'policy.yaml'.
# https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49
DEFAULT_POLICY_FILE = 'policy.yaml'
opts.set_defaults(cfg.CONF, DEFAULT_POLICY_FILE)
def reset():
global _ENFORCER
if _ENFORCER:
@ -226,7 +234,7 @@ def _build_match_rule(action, target, pluralized):
# This check is registered as 'tenant_id' so that it can override
# GenericCheck which was used for validating parent resource ownership.
# This will prevent us from having to handling backward compatibility
# for policy.json
# for policy.yaml
# TODO(salv-orlando): Reinstate GenericCheck for simple tenant_id checks
@policy.register('tenant_id')
class OwnerCheck(policy.Check):

6
neutron/tests/fullstack/resources/config.py
View File

@ -100,7 +100,7 @@ class NeutronConfigFixture(ConfigFixture):
self.config['DEFAULT']['api_paste_config'] = (
self._generate_api_paste())
policy_file = self._generate_policy_json()
policy_file = self._generate_policy_yaml()
if policy_file:
self.config['oslo_policy'] = {'policy_file': policy_file}
@ -154,8 +154,8 @@ class NeutronConfigFixture(ConfigFixture):
def _generate_api_paste(self):
return c_helpers.find_sample_file('api-paste.ini')
def _generate_policy_json(self):
return c_helpers.find_sample_file('policy.json')
def _generate_policy_yaml(self):
return c_helpers.find_sample_file('policy.yaml')
def get_host(self):
return self.config['DEFAULT']['host']

2
neutron/tests/unit/test_policy.py
View File

@ -157,7 +157,7 @@ class DefaultPolicyTestCase(base.BaseTestCase):
def setUp(self):
super(DefaultPolicyTestCase, self).setUp()
tmpfilename = self.get_temp_file_path('policy.json')
tmpfilename = self.get_temp_file_path('policy.yaml')
self.rules = {
"default": '',
"example:exist": '!',

20
releasenotes/notes/deprecate-json-formatted-policy-file-4be7b9394a2ae40f.yaml Normal file
View File

@ -0,0 +1,20 @@
---
upgrade:
- |
The default value of ``[oslo_policy] policy_file`` config option has
been changed from ``policy.json`` to ``policy.yaml``.
Operators who are utilizing customized or previously generated
static policy JSON files (which are not needed by default), should
generate new policy files or convert them in YAML format. Use the
`oslopolicy-convert-json-to-yaml
<https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html>`_
tool to convert a JSON to YAML formatted policy file in
backward compatible way.
deprecations:
- |
Use of JSON policy files was deprecated by the ``oslo.policy`` library
during the Victoria development cycle. As a result, this deprecation is
being noted in the Wallaby cycle with an anticipated future removal of support
by ``oslo.policy``. As such operators will need to convert to YAML policy
files. Please see the upgrade notes for details on migration of any
custom policy files.

8
requirements.txt
View File

@ -11,7 +11,7 @@ decorator>=3.4.0 # BSD
eventlet>=0.22.1 # MIT
pecan>=1.3.2 # BSD
httplib2>=0.9.1 # MIT
requests>=2.14.2 # Apache-2.0
requests>=2.18.0 # Apache-2.0
Jinja2>=2.10 # BSD License (3 clause)
keystonemiddleware>=5.1.0 # Apache-2.0
netaddr>=0.7.18 # BSD
@ -26,7 +26,7 @@ alembic>=0.9.6 # MIT
stevedore>=1.20.0 # Apache-2.0
oslo.cache>=1.26.0 # Apache-2.0
oslo.concurrency>=3.26.0 # Apache-2.0
oslo.config>=6.0.0 # Apache-2.0
oslo.config>=6.8.0 # Apache-2.0
oslo.context>=2.22.0 # Apache-2.0
oslo.db>=4.44.0 # Apache-2.0
oslo.i18n>=3.20.0 # Apache-2.0
@ -39,8 +39,8 @@ oslo.reports>=1.18.0 # Apache-2.0
oslo.rootwrap>=5.8.0 # Apache-2.0
oslo.serialization>=2.25.0 # Apache-2.0
oslo.service>=1.31.0 # Apache-2.0
oslo.upgradecheck>=0.1.0 # Apache-2.0
oslo.utils>=4.4.0 # Apache-2.0
oslo.upgradecheck>=1.3.0 # Apache-2.0
oslo.utils>=4.5.0 # Apache-2.0
oslo.versionedobjects>=1.35.1 # Apache-2.0
osprofiler>=2.3.0 # Apache-2.0
os-ken >= 0.3.0 # Apache-2.0

2
setup.cfg
View File

@ -162,7 +162,7 @@ oslo.config.opts =
neutron.ovn.metadata.agent = neutron.conf.agent.ovn.metadata.config:list_metadata_agent_opts
nova.auth = neutron.opts:list_auth_opts
oslo.config.opts.defaults =
neutron = neutron.common.config:set_cors_middleware_defaults
neutron = neutron.common.config:set_config_defaults
oslo.policy.enforcer =
neutron = neutron.policy:get_enforcer
oslo.policy.policies =