This commit introduces a framework for policy-in-code support
in the neutron stadium and converts the existing policy.json
in the neutron repository into the policy-in-code style.
NOTES:
1) This commit tries not to change the existing policy behavior
provided by the neutron repository even if there are some stale policies
or policies to be defined in a neutron-related project.
They should be clean up later in Stein release.
2) 'default' policy should be dropped from the default policies
as all default policies should be defined in the code (as many projects
which already completed policy-in-code do). However, dropping 'default'
policy potentially affects policy behavior in neutron-related projects,
so it needs to be visit carefully. Considering this, this commit decides
to keep the 'default' policy.
Partially Implements: blueprint neutron-policy-in-code
Change-Id: I6a61079da4d4f5080ee32d640144e6bdb14735fa
"neutron availability-zone-list" checks for "get_availability_zone" rule
and policy.json doesn't contain it so adding it now.
Change-Id: I7e5e706c44136e0b565f3ee18e15f1166d82040d
Add support for listing floating ip pools (subnets).
A new API resource ``floatingip-pools`` is introduced.
This API endpoint can return a list floating ip pools
which are essentially mappings between network UUIDs and
subnet CIDRs. Users can use this API to find out the pool
to create the floating IPs.
Related patches:
* neutron-lib: https://review.openstack.org/#/c/556674/
* tempest-plugin: https://review.openstack.org/#/c/562038/
APIImpact add floatingip pools api
Change-Id: Iaa995630645042520df67d95271e14f11ffcff8c
Partial-Bug: #1653932
The deprecated rpc_backend parameter was removed from oslo.messaging in
I193cc0e613459a6dbbfd54ed0901a54ded78d712
Use transport_url parameter instead
Change-Id: Ia73fe052986617dd6ce5e68cced0c4c88516a33e
Closes-Bug: #1795878
Add common parent owner check for the resources which introduced by
service plugin.
Then port forwarding resource will share the same tenant_id with
floatingip. That means only the fip owner can create/update/get/delete
the associated port forwarding resource.
Partially-Implements: blueprint port-forwarding
Partial-Bug: #1491317
Change-Id: I450c674e55ca15e1d9a6a6224138f3305427da68
This patch implements the plugin.
This patch introduces an new service plugin for port forwarding resources,
named 'pf_plugin', and supports create/update/delete port forwarding
operation towards a free Floating IP.
This patch including some works below:
* Introduces portforwarding extension and the base class of plugin
* Introduces portforwarding plugin, support CRUD port forwarding
resources
* Add the policy of portforwarding
The race issue fix in:
https://review.openstack.org/#/c/574673/
Fip extend port forwarding field addition in:
https://review.openstack.org/#/c/575326/
Partially-Implements: blueprint port-forwarding
Change-Id: Ibc446f8234bff80d5b16c988f900d3940245ba89
Partial-Bug: #1491317
This patchset adds missing policy actions to the policy.json
file for several reasons:
1) It signals to operators all the policy actions that are
enforced in the system. With the governance spec [0]
urging projects toward policy in code documentation,
it makes sense to document all policy actions in the
policy.json as Neutron doesn't have policy in code.
2) It is consistent with Neutron's policy enforcement
documentation [1]:
"For each attribute which has been explicitly specified in the
request create a rule matching policy names in the form
<operation>_<resource>:<attribute> rule"
So it makes sense to capture each policy that is enforced,
including all those with these special attributes.
3) Why include "update_router:external_gateway_info" but not
"create_router:external_gateway_info"? This is inconsistent.
4) It makes it difficult to validate Neutron's policy via Patrole
if the policies aren't contained in the policy.json -- how else
is it possible to determine which policies to expect if they
aren't documented anywhere?
[0] https://governance.openstack.org/tc/goals/queens/policy-in-code.html
[1] https://docs.openstack.org/neutron/pike/contributor/internals/policy.html#authorization-workflow
Change-Id: I40f84134f0b56cfd574dfd69e5ebbf6a3fc2b3df
Grouped similar items into create/get/update/delete
order so they are easier to find, as some were even
in separate blocks.
Trivialfix
Change-Id: I283cd31fadf26f00fb18ccffea19a6b59d4277a1
Currently if a new port is created by a tenant with whom
the network is shared (tenant is not the owner but has
network shared via RBAC) , the port is allocated on the default
subnet. This patch allows the tenant to create/update a port on
any subnet which is actually a part of a shared network, owned by
another tenant.
Tempest test in [1]
[1]: https://review.openstack.org/521413
Change-Id: I1046f6b13e68b1e274cc8f62f5b30aa5f8d71cdc
Closes-Bug: #1543756
The versions endpoint is now handled by Pecan
and there is no way to reach this old endpoint
now that we have dropped the web_framework option.
Change-Id: I5d493623b107f125cb9426310f2e43612c94bb7f
Drops unit tests related to fwaas policies.
Tests related to get_firewall_policy/rule are also unnecessary.
They have been migrated to neutron-fwaas already.
Closes-Bug: #1703347
Depends-On: I6dc6b2295a605444c918e44949f4b1485177e82e
Change-Id: Ie866f140fd4e5537ff0d757304ab5279f0cf0a79
This patch introduces the logging api definition and initial
implementation of LoggingApiPlugin. The api definition code will
be removed after [1] has been merged on neutron lib.
[1]https://review.openstack.org/#/c/415817/
Co-Authored-By: Yushiro FURUKAWA <y.furukawa_2@jp.fujitsu.com>
Partially-implements: blueprint security-group-logging
Related-Bug: #1468366
Change-Id: Iace31506502de25da9dce5fcfdbfe2c726bea27f
Gets rid of these log lines dumped to output when executing a test with
ostestr.
...
Did not find expected name "Test_portsecurity" in
neutron/tests/unit/extensions/test_portsecurity.py
Did not find expected name "Test_providernet" in
neutron/tests/unit/extensions/test_providernet.py
Did not find expected name "Test_quotasv2" in
neutron/tests/unit/extensions/test_quotasv2.py
...
TrivialFix
Change-Id: I6a7326e1ef4097455303b963526fe1d638a2c59c
Currently, Neutron's policy.json does not exhaustively
list all the policy actions within Neutron.
This has some downsides:
1) It makes it harder to override these policy actions
2) It is inconsistent
3) The policy.json should be a "golden copy" of all the
policy actions enforced by the system.
4) It makes it harder to RBAC test Neutron
(because it is very difficult to determine which
policy actions are valid and which are not).
The current policy actions that are enforced by the system
but not contained in the policy.json are as follows:
- create_security_group
- delete_security_group
- delete_security_group_rule
- get_security_group_rules
- get_security_groups
- get_security_group_rule
- get_security_group
- update_security_group
- update_router
- update_router:external_gateway_info
- update_router:external_gateway_info:network_id
Closes-Bug: #1676674
Change-Id: I4625c8f55bfa46b1a2209642e425677a47455219
Implements the port data plane status extension. Third parties
can report via Neutron API issues in the underlying data plane
affecting connectivity from/to Neutron ports.
Supported statuses:
- None: no status being reported; default value
- ACTIVE: all is up and running
- DOWN: no traffic can flow from/to the Neutron port
Setting attribute available to admin or any user with specific role
(default role: data_plane_integrator).
ML2 extension driver loaded on request via configuration:
[ml2]
extension_drivers = data_plane_status
Related-Bug: #1598081
Related-Bug: #1575146
DocImpact: users can get status of the underlying port data plane;
attribute writable by admin users and users granted the
'data-plane-integrator' role.
APIImpact: port now has data_plane_status attr, set on port update
Implements: blueprint port-data-plane-status
Depends-On: I04eef902b3310f799b1ce7ea44ed7cf77c74da04
Change-Id: Ic9e1e3ed9e3d4b88a4292114f4cb4192ac4b3502
This sets up the factory methods needed to have paste deploy create the
pecan app and return it. It also changes the legacy factory methods to
conditionally use the pecan factory methods if the web_framework config
option is set to 'pecan'. This way, all deployments of neutron will not
need to change their api-paste.ini files to get pecan toggled on. It
should just happen without notice once pecan becomes the default.
Also, by moving this to be loaded by paste deploy, there is a good chunk of
code that has been removed because it is no longer necessary.
Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Change-Id: I8b1bbea8d90fdc62715cd8b6738ad955df53d7cd
The FWaaS policy.json entries are being relocated into the FWaaS
repository.
Co-Authored-By: Nate Johnston <Nate_Johnston@cable.comcast.com>
Depends-On: Idd8993da78dbd779e2b3d38a4039dabf60d416d5
Change-Id: Iec36638a7a57cc14753e93c5d207853d4a3e3b6e
This patch introduces the front end implementation for QoS
minimum bandwidth rule.
APIImpact: New type of parameter for QoS rule in neutron API
DocImpact
Change-Id: I6b619a96a2bfde164646c71409b671352bc6ce7d
Partial-Bug: #1560963
This reverts commit b35d7fa3cb.
Patchset set is reverted after deciding Nova can get the segment_id
from the subnet associated to the port
Change-Id: Idf4ba107bf6e1427c82ab5b6a5cea0dba9264699
In this patchset, attribute 'ipam_segment_id' is added to ports to
contain the id of the segment where the port is bound in the case of a
routed network. This new attribute will be used by the Nova scheduler
to place an instance using the port in a host bound to the segment
Change-Id: Ide13a7c53c5f4a33894f459c38924830ba583f98
Partially-Implements: blueprint routed-networks
This patch enables basic CRUD operations to support
Subnet service-types.
Partially-implements: blueprint service-subnets
Co-Authored-By: John Davidge <john.davidge@rackspace.com>
Change-Id: I0a1724ad00f0a3e675bb700cdd291f55f898c6f3
This patch enables basic CRUD operations on trunk ports and defines
related API extensions. Trunk ports and sub-ports can be persisted
in the Neutron model and are made visible through the API, but the
L2 agent is not notified and no trunk ports or subports are actually
instantiated on compute hosts.
This one of the main patches in the series that implement the end
to end functionality.
Partially-implements: blueprint vlan-aware-vms
Co-Authored-By: Armando Migliaccio <armamig@gmail.com>
Change-Id: I26453eb9a1b25e116193417271400994ac57e4c1
Since oslo.config 3.8.0 (that included
Ibd0566f11df62da031afb128c9687c5e8c7b27ae), config_dir option is a list,
not a string. While our custom provider configuration parser for
multistring options assumes the latter.
It makes all installations that 1) pass at least one --config-dir option
in CLI and 2) enable any service plugin that relies on provider
definitions, to fail to start neutron-server. For example, this affects
any RDO Mitaka installation with *aas service plugins enabled.
Since Newton requires >=3.9.0, we are fine to switch to the list type
without any code to support backwards compatibility with older option
type. For Mitaka backport, we will need to handle both cases.
Change-Id: I10e399a852d9fba0fd1aea79a10e2e7c906e4b3c
Closes-Bug: #1585102
There still are BGP codes in neutron repo, this patch remove it.
It was missed in commit I20b6ddd37d10eae70e8294d578e53137c0f866fe
Change-Id: I6bf71b14a03ccc005dcd4d67078e7e9f4a7ed197
Closes-Bug: #1584333
This patch enables CRUD on Segments by defining a new entity called
'Segment' with an extension definition and some DB code to read the
existing segments DB. A basic framework for create, update, and
delete are provided.
For now, this is just the basic boiler-plate but I've got to start
somewhere. It is implemented as a service plugin that is disabled by
default because it has not been fully tested with any plugin.
Follow-on patches will implement support for this new extension in
ML2 and OVN at least.
Change-Id: Ifc370fdd38f9a5b296334635fa85bd93d270b910
Partially-Implements: blueprint routed-networks
Turns out the patch with Git commit hash prefix 13993764
disabled functional tests logging completely. This patch
fixes that by moving the neutron-db-manage logging setup
from import to the main function. Fixing that, it looks like
patch with Git commit hash prefix 4980f031fe turned off
DEBUG level logging for functional tests. I changed the
tests default logging from INFO to DEBUG to fix that.
Closes-Bug: #1567613
Change-Id: If02c18af20a236514409f37fa993224903877a8b
Once the spinout is undergoing we should perform the eviction.
Partially-implements: blueprint bgp-spinout
Depends-on: I8be510153edbc496575cde34943ca4c56645e0fb
Change-Id: I20b6ddd37d10eae70e8294d578e53137c0f866fe
oslo logging complains that:
"Option verbose from group DEFAULT is deprecated for removal. Its
value may be silently ignored in the future."
Switch to using default_log_levels instead.
Change-Id: Ibddb8b379753193624d1a3df1cf2c53018c0a62b
This patch adds the front end and back end implementation of QoS DSCP.
Associated patches that are dependent on this one:
* python-neutronclient: https://review.openstack.org/#/c/254280
* openstack-manuals: https://review.openstack.org/#/c/273638
* API Guide: https://review.openstack.org/#/c/275253
* Heat:
* Spec: https://review.openstack.org/#/c/272173
* QoSDscpMarkingRule resource: https://review.openstack.org/#/c/277567
* Fullstack tests: https://review.openstack.org/#/c/288392/
APIImpact - The API now supports marking traffic egressing from a VM's
dscp field with a valid dscp value.
Co-Authored-By: Nate Johnston <nate_johnston@cable.comcast.com>
Co-Authored-By: Victor Howard <victor.r.howard@gmail.com>
Co-Authored-By: Margaret Frances <margaret_frances@cable.comcast.com>
Co-Authored-By: James Reeves <james.reeves5546@gmail.com>
Co-Authored-By: John Schwarz <jschwarz@redhat.com>
Needed-By: I25ad60c1b9a66e568276a772b8c496987d9f8299
Needed-By: I881b8f5bc9024c20275bc56062de72a1c70c8321
Needed-By: I48ead4b459183db795337ab729830a1b3c0022da
Needed-By: Ib92b172dce48276b90ec75ee5880ddd69040d7c8
Needed-By: I4eb21495e84feea46880caf3360759263e1e8f95
Needed-By: I0ab6a1a0d1430c5791fea1d5b54106c6cc93b937
Partial-Bug: #1468353
Change-Id: Ic3baefe176df05f049a2e06529c58fd65fe6b419
Implements an API extension for reporting availibility of IP
addresses on Neutron networks/subnets based on the blueprint
proposed at https://review.openstack.org/#/c/180803/
This provides an easy way for operators to count the number of
used and total IP addresses on any or all networks and/or
subnets.
Co-Authored-By: David Bingham <dbingham@godaddy.com>
Co-Authored-By: Craig Jellick <craig.jellick@gmail.com>
APIImpact
DocImpact: As a new API, will need all new docs. See devref for details.
Implements: blueprint network-ip-usage-api
Closes-Bug: 1457986
Change-Id: I81406054d46b2c0e0ffcd56e898e329f943ba46f
This patch implements a new extension called "bgp_dragentscheduler" which
does instant & auto scheuling of BgpSpeakers to an active BgpDrAgent. In
addition to this the patch also implements the basic CRUD requirement for
binding BgpSpeakers and BgpDrAgent.
BgpSpeaker to BgpDrAgent association can be 1-to-n. An admin user can only
associate/disassociate BgpSpeaker to/from a BgpDRAgent. Default scheduler
class will only assign non-scheduled BgpSpeaker to an active BgpDrAgent.
Partially-Implements: blueprint bgp-dynamic-routing
Co-Authored-By: Ryan Tidwell <ryan.tidwell@hpe.com>
Co-Authored-By: Jaume Devesa <devvesa@gmail.com>
Co-Authored-By: vikram.choudhary <vikram.choudhary@huawei.com>
Co-Authored-By: Numan Siddique <nusiddiq@redhat.com>
Change-Id: Id305d9a583116e155441ac5979bf3f6aa6a8258b
This patch enables basic CRUD on BGP dynamic routing
entities bgp_speaker and bgp_peer, as well as
bgp_speaker-bgp_peer and bgp_speaker-network
bindings.
An admin user can create BgpSpeakers and configure
peering entities (BgpPeers) for BgpSpeakers. BgpSpeaker
to BgpPeer association is n-to-n. An admin user can
also associate networks with BgpSpeakers. Relationship
between BgpSpeaker and Network is 1-to-n.
This patch provides BGP-related functionality only to
the admin users.
Partially-Implements: blueprint bgp-dynamic-routing
Co-Authored-By: Ryan Tidwell <ryan.tidwell@hpe.com>
Co-Authored-By: Jaume Devesa <devvesa@gmail.com>
Co-Authored-By: vikram.choudhary <vikram.choudhary@huawei.com>
Change-Id: I2412c1689683da9d7ec884a4cea506d4eed99453
This patch introduces an API to allocate an externally connected
private tenant network on demand. The API is idempotent in that,
once the topology is provisioned, further API calls keep returning
the same topology to the caller.
The API, as introduced by the patch, is not currently on, and its
design carefully ensures minimal impact on the existing codebase.
In fact the feature depends on and enhances the external-net extension,
but it does so via callbacks.
A subsequent patch in this series will make it available by default,
and API tests will be added to validate the functionality.
Partially-implements: blueprint get-me-a-network
Co-Authored-By: Armando Migliaccio <armamig@gmail.com>
Co-Authored-By: Henry Gessau <HenryG@gessau.net>
Change-Id: I4abd45252026431452f0d2cb2805043489c2f6ad
When a port is validated, we check for the user to be the owner of
corresponding network, among other things. Sadly, this check requires a
plugin call to fetch the network, which goes straight into the database.
Now, if there are multiple ports to validate with current policy, and
the user is not admin, we fetch the network for each port, f.e. making
list operation on ports to scale badly.
To avoid that, we should postpone OwnerCheck (tenant_id) based
validations that rely on foreign keys, tenant_id:%(network:...)s, to as
late as possible. It will make policy checks avoid hitting database in
some cases, like when a port is owned by current user.
Also, added some unit tests to avoid later regressions:
DbOperationBoundMixin now passes user context into API calls. It allows
us to trigger policy engine checks when executing listing operations.
Change-Id: I99e0c4280b06d8ebab0aa8adc497662c995133ad
Closes-Bug: #1513782
Make flavor service profile store actual driver instead of
hardcoded dummy driver. Ensure service type on flavor persisted.
Raise ServiceProfileDriverNotFound if non-empty driver is not part
of ServiceTypeManager providers.
Raise ServiceProfileEmpty if profile has neither a driver nor
any metainfo.
Raise InvalidFlavorServiceType if invalid service type passed.
Show flavors associated with a profile, not just profiles associated
with a flavor, to ease diagnosis when ServiceProfileInUse raised.
Create method to extract provider given a flavor for use with
neutron-lbaas plus tests.
Ensure various boolean forms accepted for enabled flag.
To enable in DevStack, add to local.conf:
enable_plugin neutron https://git.openstack.org/openstack/neutron
enable_service q-flavors
Add associated unit tests. Fix tempest api test that used invalid
LOADBALANCERS service type.
Change-Id: I5c22ab655a8e2a2e586c10eae9de9b72db49755f
Implements: blueprint neutron-flavor-framework
This patch adds a new boolean 'is_default' property to subnetpools. This
allows the admin to set the default v4/v6 subnetpools via the API rather
than the existing neutron.conf options - which are deprecated by this patch.
Only one subnetpool per IP family can be set to default.
DocImpact
ApiImpact
Co-Authored-By: Carl Baldwin <carl@ecbaldwin.net>
Change-Id: I5daba2347cfb91fac0b155b2c1b459ee7d9e4505
Closes-Bug: 1501328
We may be unable to get rid of this code, but at least we can
make it slimmer and more oslo_config friendly.
This patch also fixes an issue where plain ConfigParser cannot
handle config options with the same name under same section.
Related-bug: #1492069
Change-Id: I596613eb7b2fc3e894a3c1cc7b7cb24d7137cbc5
This patch adjusts the FieldCheck class in the policy engine to
allow a regex rule. It then leverages that to prevent users from
setting the device_owner field to anything that starts with
'network:' on networks which they do not own.
This policy adjustment is necessary because any ports with a
device_owner that starts with 'network:' will not have any security
group rules applied because it is assumed they are trusted network
devices (e.g. router ports, DHCP ports, etc). These security rules
include the anti-spoofing protection for DHCP, IPv6 ICMP messages,
and IP headers.
Without this policy adjustment, tenants can abuse this trust when
connected to a shared network with other tenants by setting their
VM port's device_owner field to 'network:<anything>' and hijack other
tenants' traffic via DHCP spoofing or MAC/IP spoofing.
Closes-Bug: #1489111
Change-Id: Ia64cf16142e0e4be44b5b0ed72c8e00792d770f9
Without this empty policy rule, get_rule_type will use default, which
will demand admin role or tenant_id in object. but rule_type has no
tenant_id in its body.
Change-Id: I92b1222fbcdc2efd13ca6f586cfefefc55b59189
Closes-bug: #1487324
This adds the new API endpoint to create, update, and delete
role-based access control entries. These entries enable tenants
to grant access to other tenants to perform an action on an object
they do not own.
This was previously done using a single 'shared' flag; however, this
was too coarse because an object would either be private to a tenant
or it would be shared with every tenant.
In addition to introducing the API, this patch also adds support to
for the new entries in Neutron networks. This means tenants can now
share their networks with specific tenants as long as they know the
tenant ID.
This feature is backwards-compatible with the previous 'shared'
attribute in the API. So if a deployer doesn't want this new feature
enabled, all of the RBAC operations can be blocked in policy.json and
networks can still be globally shared in the legacy manner.
Even though this feature is referred to as role-based access control,
this first version only supports sharing networks with specific
tenant IDs because Neutron currently doesn't have integration with
Keystone to handle changes in a tenant's roles/groups/etc.
DocImpact
APIImpact
Change-Id: Ib90e2a931df068f417faf26e9c3780dc3c468867
Partially-Implements: blueprint rbac-networks
fix policy.json to not allow tenants to create policies or rules
by default and allow tenants attach ports and networks to policies,
please note that policy access is checked in the QoSPolicy neutron
object in such case.
Closes-Bug: #1485858
Change-Id: Ide1cd30979f99612fe89dddf3dc0e029d3f4d34a
Also applied the following fixes:
===
1. cleaned up some pylint failures that were not spotted before:
Module neutron.objects.qos.policy: Metaclass class method __new__ should
have 'mcs' as first argument
Module neutron.objects.qos.rule: Lambda may not be necessary
===
2. Revert "Introduce the AFTER_READ callback for ports and networks"
This reverts commit e3dba14241.
We don't use callbacks to extend resources anymore, instead relying on
ml2 extension drivers. No need for the patch to achieve QoS, and it also
breaks test_delete_subnet_with_callback that was added in master
recently.
===
3. updated requirements.txt and test-requirements.txt based on:
https://review.openstack.org/#/c/204398/
to avoid requirements gate checks failing due to incompatible
requirements comparing to global-requirements.txt
Change-Id: I744ab2d8327a428a5467f2d07d073a5f8c333520
This patch introduces API and DB plugin for flavor framework.
API adds Flavors and Service Profiles which are resources
available only for admins to operate.
This framework then should be leveraged by advanced services.
Included tempest API tests in neutron tree
Implements: blueprint neutron-flavor-framework
Change-Id: I99ba0ce520ae3d8696eca5c994777c7d5ba3d4b1
Co-Authored-By: Doug Wiegley <dougw@a10networks.com>
Co-Authored-By: Madhusudhan Kandadai <madhusudhan.kandadai@hp.com>
Akihiro Motoki