security_groups_db._check_security_group is supposed to check the
security_group_id of the _create_security_group_rule payload.
When using an integer e.g. 0, as security_group_id, the check
succededs because mysql accepts following query:
SELECT * FROM securitygroups WHERE id in (0)
Forcing validation of security_group_id as uuid fixes the problem
Closes-Bug: #1968343
Change-Id: I7c36b09309c1ef66608afacfb281b6f4b06ea5b8
- Add api extension and db model changes to support remote_address_group_id
in SG rules.
- RPC and firewall agent changes will be in the follow-up patches.
Change-Id: I99681736d05eefd82bdba72b3866eab9468ef5dd
Implements: blueprint address-groups-in-sg-rules
With python 3.x, six.text_type and six.string_type
are just str.
Also removed a six.integer_type since it was the only
one left in a file.
Another step in removing all of six usage from neutron.
Change-Id: I5208dc41bff1983ecd323286f427296b722da62a
With python 3.x, classes can use the metaclass= logic
to not require usage of the six library.
One step in removing all of six usage from neutron.
Change-Id: I2f815e412d9a96eb5faf2b3bb3a1e393a9db9309
When I'm trying to introduce a central sort-keys validation within
patch [1] to `get_sorts` method in `neutron.api.api_common` module,
I get blocked by some resource schemas and test cases. After reading
neutron API docs and some inspection, I believe there exists uses of
improper sort keys in test cases and some resource schemes need to
keep aligned with offical documents.
* Schemas of resource SecurityGroups/SG Rules/Segments don't provide
`is_sort_key` flag for their sort key properties claimed in offical
docs as neutron-lib does. See [2] for more details.
* Test cases of resource NetworkSegmentRange use unsupported sort keys,
e.g: physical_network. Replace it with `name` property. See [2] for more
details.
[1] https://review.opendev.org/#/c/653903/
[2] https://developer.openstack.org/api-ref/network/v2/index.html
Change-Id: I45a51736e4075e3dbc16827486869d70b659622d
After taking a closer look at bug 1818385, I found a couple
of follow-on things to fix in the security group code.
First, there are very few protocols that accept ports,
especially via iptables. For this reason I think it's
acceptable that the API rejects them as invalid.
Second, UDPlite has some interesting support in iptables. It
does not support using --dport directly, but does using
'-m multiport --dports 123', and also supports port ranges using
'-m multiport --dports 123:124'. Added code for this special
case.
Change-Id: Ifb2e6bb6c7a2e2987ba95040ef5a98ed50aa36d4
Closes-Bug: #1818385
Improve error message that is shown when
port ranges are specified for protocols that don't
support port ranges.
Change-Id: I70ad9bd01f986efaed6682928cc5579c75bd4f6e
Closes-Bug: #1815478
Signed-off-by: Andreas Karis <akaris@redhat.com>
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.
NeutronLibImpact
Change-Id: I9704f20eb21da85d2cf024d83338b3d94593671e
Enforce validation on filter parameters on list requests.
If an API request contains an unknown or unsupported parameter,
the server will return a 400 response instead of silently ignoring
the invalid input.
In resource attributes map, all filter parameters are annotated by
the ``is_filter`` keyword. Attributes with is_filter set to True
are candidates for validation.
Enabling filter validation requires support from core plugin and
all service plugins so each plugin need to indicate if it supports
the validation by setting ``__filter_validation_support`` to True.
If this field is not set, the default is False and validation is
turned off. Right now, the ML2 plugin and all the in-tree service
plugin support filter validation. Out-of-tree plugins will have
filter validation disabled by default.
An API extension is introduced to allow API users to discover this
new API behavior. This feature can be disabled by cloud operators
if they choose to do that. If it is disabled, the extension won't
be presented.
Depends-On: Ic3ab5b3ffdc378d570678b9c967cb42b0c7a8a9b
Depends-On: I4397df1c35463a8b532afdc9c5d28b37224a37b4
Depends-On: I3f2e6e861adaeef81a1a5819a57b28f5c6281d80
Depends-On: I1189bc9a50308df5c7e18c329f3a1262c90b9e12
Depends-On: I057cd917628c77dd20c0ff7747936c3fec7b4844
Depends-On: I0b24a304cc3466a2c05426cdbb6f9d99f1797edd
Change-Id: I21bf8a752813802822fd9966dda6ab3b6c4abfdc
Partial-Bug: #1749820
This incorporates flake8 2.6.x and pycodestyle will be used
instead of older pep8. This ensures future python3 compatibility
and a bit better code styling.
Change-Id: Ia7c7c5a44727f615a151e1e68dd94c7ed42f974f
This patch removes the explicit API definition of the 'description'
attribute from the security group API extension. This
attributes is redundant as its added by default by the
standardattrdescription extension and moreover are clobbered by the
latter.
In addition this patch updates the UTs to account for the extension
manager not adding the standardattrdescription to the security group
definition during test runtime.
Also note that the segment API extension has the same issue, but will be
handled with the consumption of https://review.openstack.org/#/c/562331/
Change-Id: I1fcc2adf13792705fc4bb6faf757ee59faee1349
Partial-Bug: 1757513
Iptables only supports port-ranges for certain protocols,
others will generate failures, possibly leaving the agent
looping trying to apply rules. Change to not allow port
ranges outside of the list of known good protocols.
Change-Id: I5867f77fc5aedc169b42f50def0424ff209c164c
Closes-bug: #1749667
This patch disallows cases that following name is specified.
1. name whose type is not String.
2. name whose characters is more than 255.
Change-Id: Ib72a4e480b62a22da2171ed24449321f2b27258b
Closes-Bug: #1610764
Remove The following _MAX_LEN constants from
neutron/api/v2/attributes.py and use the corresponding DB field size
constants from neutron_lib.db.constants instead.
NAME_MAX_LEN --> NAME_FIELD_SIZE
TENANT_ID_MAX_LEN --> PROJECT_ID_FIELD_SIZE
DESCRIPTION_MAX_LEN --> DESCRIPTION_FIELD_SIZE
LONG_DESCRIPTION_MAX_LEN --> LONG_DESCRIPTION_FIELD_SIZE
DEVICE_ID_MAX_LEN --> DEVICE_ID_FIELD_SIZE
DEVICE_OWNER_MAX_LEN --> DEVICE_NAME_FIELD_SIZE
In alembic migration scripts, the raw numerical value is used.
For more information, see:
http://lists.openstack.org/pipermail/openstack-dev/2016-October/105789.html
NeutronLibImpact
Change-Id: I734890372584fe27e5d6ec38c0cad2de882ff11c
Currently, we check whether specified value in list
matches with UUID format or not.
However, we don't check whether data type of specified value of
security_groups key is list in the first place.
This patch adds check above.
Change-Id: If027df3ff34b219e4a043934b9461ac0fcb40156
Closes-Bug: #1477921
Adopt the class rehomed in https://review.openstack.org/389825
This change impacts subprojects that use ExtensionDescriptor from
neutron core. All subprojects should switch to using the class
directly from neutron-lib.
NeutronLibImpact
Change-Id: I0da90fb9d27a0a2008b60e56ed69bc9b9ae63c3e
It's written to but never read from!
Related-Bug: #1642426
This patch is required by neutron-lib adoption because it is part of
rehoming neutron/api/v2/attributes.py. All subprojects that reference
PLURALS must be patched to remove the reference.
NeutronLibImpact
Change-Id: Ifdf29b8fc4c824c0ee840c4f51593a5aa8a22127
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
As per [1], neutron-lib 0.3.0 provides a public add/get API for
local validator registration/access. This API is preferred over
directly accessing the validators dict module-level attribute
that's done today when adding a new validator and in fact
direct access to the validators dict is deprecated.
This patch changes all neutron's usage of the validators dict
to use the public API.
Closes-Bug: #1584237
[1] https://review.openstack.org/#/c/324090/
Change-Id: I97680b0a8d88cb753199fce7949cfb24b506a0ff
Since neutron_lib 0.2.0 contains PROTO_NAME_IPV6_ICMP_LEGACY, remove
it from neutron.common.constants
Change-Id: Idd150ce1cbe660fc9529e5d46678f37142490a28
Closes-bug: #1585047
The patch https://review.openstack.org/#/c/252155/ renamed
'icmpv6' protocol to 'ipv6-icmp'.
This leads to backward compatiblity of the security group API.
This commit allows to specify 'icmpv6' as well.
TODO(amotoki): The constant for 'icmpv6' will be moved to
neutron-lib soon after Mitaka is shipped.
Change-Id: I0d7e1cd9fc075902449c5eb5ef27069083ab95d4
Closes-Bug: #1558774
In order to give users and operators more flexibility in
annotating the purpose of various Neutron resources, this patch
adds a description field limited to 255 chars to all of the
Neutron resources that reference the standard attribute table.
The resource that reference the table are the following:
security_group_rules, security_groups, ports, subnets,
networks, routers, floatingips, subnetpools
This patch adds a description table related to standard attributes
and migrates over the existing security group description to the new
table as well.
Co-Authored-By: James Dempsey <jamesd@catalyst.net.nz>
APIImpact
DocImpact: Adds a description field to all resources outline in
commit message.
Closes-Bug: #1483480
Change-Id: I6e1ef53d7aae7d04a5485810cc1db0a8eb125953
The Neutron Securitygroup extension defines two resources:
security-group
security-group-rule
So that other extensions could extend one or both of this resources, the
security-group extension descriptor must override the base class method,
"neutron.extensions.ExtensionDescriptor.update_attributes_map".
Change-Id: I8c462a4ee6f60ef716bf9e4d7f83a35c7e1dead0
Closes-Bug: #1529109
Add these additional protocols listed below to
security groups brings convenience to operators
on configuring these protocols. In addition, make
the security group rules more readable.
The added protocols are: ah, dccp, egp, esp, gre,
ipv6-encap, ipv6-frag, ipv6-nonxt, ipv6-opts,
ipv6-route, ospf, pgm, rsvp, sctp, udplite, vrrp.
A related patch is submitted to neutron-lib project:
https://review.openstack.org/259037
DocImpact: You can specify protocol names rather than
protocol number in API and CLI commands. I'll update
the documentation when it is merged.
APIImpact
Change-Id: Iaef9b650449b4d9d362a59305c45e0aa3831507c
Closes-Bug: #1475717
This patch resolves the issue where wrong message was being
shown when ethertype input parameter was not amongst one of
the types supported. New message made akin to other input
parameters like 'protocol'.
Change-Id: I5636f3582c9d9877dad4d091a374284b656923f4
Closes-Bug: #1508106
It doesn't make much sense to allow updating only the direction.
I suppose it was a bug in the first place.
Partial-Bug: #1521099
Change-Id: Idfd48c801be3cd34286595f5ca3c9d629a296200
This patch will add the validation function to make sure security_group
rule is not created when ethertype is IPv4 and protocol is icmpv6.
Closes-Bug: #1505832
Depends-On: Ia1a5342a1d568cb1a015e1b7acecf38b8d1f46e1
Co-Authored By: Matt Dorn <madorn@gmail.com>
Change-Id: I4a15935c564aaa48555ed08f6da51113787ecb73
This patch introduces application logic support for tracking
Neutron resource usage data, thus introducing a different
way of enforcing quota limits, which now relies on records
tracking resource usage for each tenant.
When these records go "out-of-sync" with actual resource usage,
the quota usage table entry is marked "dirty".
And it will be resynchronized the next time a resource count is
requested. Changes in resource utilization are detected using
SQLAlchemy events.
This patch however does not automatically enable resource usage
tracking for any plugin. Plugins must explicitly declare for which
resources they wish to enable tracking. To this aim, this patch
provides the @tracked_resources decorator.
As some operators might wish to not use resource usage tracking, this
patch adds the track_quota_usage configuration option. If set to
False, DB-level usage tracking will not be performed, and
resources will be counted as before, ie: invoking a plugin method.
Usage tracking is performed using a new resource type,
TrackedResource, which can work side by side with CountableResource,
the one Neutron used so far. To this aim a ResourceRegistry class
has been introduced for registering and managing resources. Most
of the resource management code previously embedded in the
QuotaEngine class is being moved to ResourceRegistry as a part of
this patch.
Partially implements blueprint better-quota
Change-Id: If461399900973a38d7b82e0b3ac54fc052f09529
In Python 3, dict.items() returns an iterator. Iterators cannot be added.
Blueprint: neutron-python3
Change-Id: I487178ebceae9946cb53dea1e847d7715f4577f3
In Python 3, there is no "basestring". In Python 3, "six.string_types" is
"basestring", and "str" in Python 3.
Change-Id: Ic22e932cbf3c4b75cd424f4b41428da869f197cf
Blueprint: neutron-python3
support using icmpv6 (protocol num 58) in the protocol option, when creating
a security group rule.At this time, port_range_min/port_range_max represent
icmpv6 type/code, and you can use only port_range_min to specify just one type.
eg:neutron security-group-rule-create --direction ingress \
--ethertype ipv6 --protocol icmpv6 --port-range-min 134 SECURITY_GROUP
ApiImpact
DocImpact
Partial-Bug:#1427973
Change-Id: Ide4f7476cdb8a4f04f72983917ce7dbfc7be90a5
Andrew Karpow