431 Commits

Author SHA1 Message Date
Brian Haley
4350ed3c35 Better handle ports in security groups
After taking a closer look at bug 1818385, I found a couple
of follow-on things to fix in the security group code.

First, there are very few protocols that accept ports,
especially via iptables.  For this reason I think it's
acceptable that the API rejects them as invalid.

Second, UDPlite has some interesting support in iptables.  It
does not support using --dport directly, but does using
'-m multiport --dports 123', and also supports port ranges using
'-m multiport --dports 123:124'.  Added code for this special
case.

Change-Id: Ifb2e6bb6c7a2e2987ba95040ef5a98ed50aa36d4
Closes-Bug: #1818385
2019-03-15 13:54:33 -04:00
Zuul
7198fb6a0a Merge "Remove deprecated 'external_network_bridge' option" 2019-03-13 15:42:44 +00:00
Doug Wiegley
5e0fc3d2da Allow sharing of security groups via RBAC mechanism
Neutron-lib api ref: https://review.openstack.org/#/c/635313/
Tempest tests: https://review.openstack.org/#/c/635312/
Client: https://review.openstack.org/#/c/635428/

Partial-Bug: #1817119
Depends-On: https://review.openstack.org/635313
Change-Id: I974b0a603b6ca75cf080fb7b0751c7fb87df8443
2019-03-09 17:30:14 -06:00
Sławek Kapłoński
b09b44608b Remove deprecated 'external_network_bridge' option
This option is deprecated and marked to be deleted in Ocata. So
as we are now in Stein development cycle I think that it's good time
to remove it.

Change-Id: I07474713206c218710544ad98c08caaa37dbf53a
2019-03-09 22:07:38 +00:00
Kailun Qin
563a536d02 Support Network Segment Range CRUD as extensions
This patch adds the support for network segment range CRUD. Subsequent
patches will be added to use this network segment range on segment
allocation if this extension is loaded.

Changes include:
- an API extension which exposes the segment range to be administered;
- standard attributes with tagging support for the new resource;
- a new service plugin "network_segment_range" for the feature
  enabling/disabling;
- a new network segment range DB table model along with operation
  logic;
- Oslo Versioned Objects for network segment range data model;
- policy-in-code support for network segment range.

Co-authored-by: Allain Legacy <Allain.legacy@windriver.com>

Partially-implements: blueprint network-segment-range-management
Change-Id: I75814e50b2c9402fe6776229d469745d7a72290b
2019-03-07 08:20:30 +00:00
Zuul
018595d4b0 Merge "Enable adoption of subnets into a subnet pool" 2019-03-02 20:37:11 +00:00
Doug Wiegley
61b231a999 Change process name of neutron-server to match worker role
We spawn a lot of neutron-servers, on all but the smallest systems.
It's often hard to tell which are busy/overloaded or spinning.
Add an option to set the process names to their role.

This has a small chance of breaking existing scripting, depending
how they're parsing ps output.

Sample output:
$ ps xw | grep neutron-server
 1126 pts/2    S+     0:00 grep --color=auto neutron-server
25355 ?        Ss     0:26 /usr/bin/python /usr/local/bin/neutron-server \
  --config-file /etc/neutron/neutron.conf \
  --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
25368 ?        S      0:00 neutron-server: api worker
25369 ?        S      0:00 neutron-server: api worker
25370 ?        S      0:00 neutron-server: api worker
25371 ?        S      0:00 neutron-server: api worker
25372 ?        S      0:02 neutron-server: rpc worker
25373 ?        S      0:02 neutron-server: rpc worker
25374 ?        S      0:02 neutron-server: services worker

The "normal" looking ps output is the main parent.

Partial-Bug: #1816485
Depends-On: https://review.openstack.org/637119
Change-Id: I0e664a5f8e792d85b8f5483fb8c6f1cd59a677cd
2019-03-01 14:18:09 -05:00
Ryan Tidwell
d5896025b7
Enable adoption of subnets into a subnet pool
This patch enables the adoption of existing subnets into a
subnetpool. Adoption of a subnet is done by passing the ID
of the hosting network and the address family (ip_version)
which causes all subnets in the specified address family
on the given network to be adopted by the subnet pool. This
continues to work within the constraints on subnet pool
membership of subnets on the same network. This also ensures
prefix uniqueness across an address scope before comitting the
adoption of subnets.

Change-Id: I5d3c07beb7f109142d2e3633e69f86ca39edc450
Partially-Implements: blueprint subnet-onboard
Co-Authored-By: Ryan Tidwell <rtidwell@suse.com>
Co-Authored-By: Reedip <reedip.banerjee@nectechnologies.in>
Co-Authored-By: Trevor McCasland <TM2086@att.com>
Co-Authored-By: Bernard Caffarelli <bcafarel@redhat.com>
2019-02-26 11:13:09 -06:00
Zuul
5323e9549d Merge "Reject QoS minimum bandwidth rule updates for bound ports as NotImplemented" 2019-02-23 05:16:37 +00:00
Zuul
b6d2eb4308 Merge "Modify api and rpc default number of workers" 2019-02-21 18:00:43 +00:00
Bence Romsics
facc4e94ca Reject QoS minimum bandwidth rule updates for bound ports as NotImplemented
Updating QoS policies and rules backed by resources tracked in Placement
is a complex task, which was left out of scope for the Stein release.
Of course this is only relevant for policies/rules already in use on
bound ports. Rules of unbound ports can still be updated freely.

Please note this patch cannot catch all possible problems. There will
always be a time window between the allocation committed to Placement
and Nova sending the port binding request to Neutron. If the policy or
rule is changed in this window we cannot do anything against it.

APIImpact: Reject QoS minimum bandwidth policy/rule updates for bound
           ports as NotImplemented

Change-Id: I477edb0ae35b385ac776a58195f22382e2fce4ed
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
2019-02-21 16:59:31 +01:00
Doug Wiegley
7e09b25b96
Modify api and rpc default number of workers
- Limit number of api workers to roughly using half of system
  RAM. Spawning a bunch, just to have the OOM killer nuke them
  regularly is not useful.
- Bump the rpc_workers default to half of the api_workers.
  A default of 1 falls behind on any reasonably sized node.

Change-Id: I8b84a359f83133014b3d4414aafc10e6b7c6a876
Closes-bug: #1815629
2019-02-19 13:24:01 -07:00
LIU Yulong
e8b7e768a2 Add dependency for service plugin
Adds a required list 'required_service_plugins' to each service plugin,
then we can initialize the service plugin with required dependency.
And also adds the 'router' plugin to port forwarding service plugin
required list.

Closes-Bug: #1809238
Change-Id: I53fdaee0cd96a5315a7abc39799657d613eb3a2e
2019-02-16 00:17:53 +08:00
Zuul
a159a7294d Merge "Add port forwarding floating IP QoS" 2019-02-01 18:31:34 +00:00
LIU Yulong
e108ac6bdf Add port forwarding floating IP QoS
Port forwarding floating IPs QoS should be limited under
the binding QoS policy. So this patch extends the l3-agent
fip-qos agent extension floating IP list with the port
forwarding related IPs.

Change-Id: Iddabfabafc0803edd1e4ac0893dc188f1907234a
Closes-Bug: #1796925
2019-01-30 14:04:00 +00:00
Harald Jensås
b0d758e1b4 Fix port update deferred IP allocation with host_id + new MAC
IP allocation was initially deffered due to lack of binding
information. On port update the with both `mac_address` and
`binding_host_id`` in the request 'fixed_ips: []' was
appended to the new_port data. This caused the check for
fixed_ips_requested to return True, which in turn cause
deferred_ip_allocation to evaluates False.

Only set the new_port default fixed_ips to original_ips if
the original port had fixed_ips.

Closes-Bug: #1811905
Change-Id: If98a82f8432b09a29f9d0cc6627e9649b43bc4a1
2019-01-29 22:31:17 +00:00
nicky
f1592c42d9 Add a new configuration parameter rpc_response_max_timeout
A new parameter rpc_response_max_timeout is added and registered into
neutron.conf.

The rpc_response_max_timeout plays a role of the ceiling of timeout
seconds when waiting for the response of a remote rpc server.

During an RPC call, the waiting time starts from the existing parameter
rpc_response_timeout(default 60s) and doubled each time until it reaches
the ceiling which is currently set as 10 times rpc_response_timeout.
It seems to be less flexible since user cannot directly change the
ceiling value unless he/she changes the rpc_response_timeout.
By adding rpc_response_max_timeout, user can now modify it without
changing any other parameters.

Co-Authored-By: Allain Legacy<Allain.legacy@windriver.com>

Change-Id: I170113c2946cc95308edcb1a703a99c71e50b6f9
Related-Bug: #1805769
Story: 2004456
Task: 28171
2019-01-18 09:55:31 +00:00
LIU Yulong
433228dd78 Prevent bind fip to port has port forwarding
If one port has port forwarding and the port is under
a dvr router, then binding floating IP to this port
will not be allowed.

Change-Id: Ia014e18264b43cf751a5bc0e82bc55d106582620
Closes-Bug: #1799138
2019-01-03 12:41:55 +08:00
Zuul
d8c06b002f Merge "Support fetching specific db column in OVO" 2018-12-20 03:56:14 +00:00
Zuul
ff887f9981 Merge "Use publish for AGENT AFTER_CREATE/AFTER_UPDATE" 2018-12-20 02:07:41 +00:00
Zuul
3ac1bea967 Merge "Upgrade check command - add support for 3rd party checks" 2018-12-19 12:50:19 +00:00
Lajos Katona
6d99fb19ab Use publish for AGENT AFTER_CREATE/AFTER_UPDATE
This patch switches callbacks over to the payload object style events
for AGENT AFTER_CREATE and AFTER_UPDATE based notifications. To do
so a DBEventPayload object is used with the publish() method to
pass along the API related data.

Change-Id: Ibefa495be41c91957c2e8d797130e569bccc3765
2018-12-19 10:26:05 +01:00
Slawek Kaplonski
f5b01e0e12 Upgrade check command - add support for 3rd party checks
CLI command "neutron-status upgrade check" now can load
checks from 3rd party and stadium projects using entry_points.

Such additional checks should be available under
"neutron.status.upgrade.checks" namespace in entry_points.

Change-Id: I416abbcdd7b397ec6acd2827129d7a56095ea63d
Story: 2003657
2018-12-18 11:23:03 +01:00
Lajos Katona
97c9c4656d get_standard_device_mappings for mechdriver
get_mappings for machanism drivers return back different dicts for
different mechanism drivers. By adding get_standard_device_mappings
method SRIOV and OVS mechanism drivers can return a mapping dict in a
form like this one:
{'physnet_name': ['device_or_bridge_1', 'device_or_bridge_2']}

Change-Id: Ieddd9b3a4f3d7269aafc373f040c55b025f5c201
Related-Bug: #1578989
2018-12-13 09:52:01 +00:00
Hongbin Lu
f24f422373 Support fetching specific db column in OVO
There is a analysis [1] suggested to run queries against specific
columns rather than full ORM entities to optimize the performance.
Right now, it is impossible to execute such optimization because
OVO doesn't support fetching specific column yet.

This commit introduces a new method 'get_values' in the base
neutron object class. Subclass of neutron object can leverage
this method to fetch specific field of a OVO. It supports fetching
non-synthetic fields only as syntheic fields are not directly backed
by corresponding DB table columns.

neutron-lib patch: https://review.openstack.org/#/c/619047/

[1] https://review.openstack.org/#/c/592361/

Needed-By: https://review.openstack.org/#/c/610184/

Change-Id: Ib90eae7738a5d2e4548fe9fed001d6cdaffddf3b
Partial-Implements: blueprint adopt-oslo-versioned-objects-for-db
2018-12-11 19:29:28 +00:00
LIU Yulong
00bf365025 [L3][QoS] Neutron server side router gateway IP QoS
This patch enables to bind a QoS policy to the router gateway,
then in L3 agent side SNAT traffic for the VMs without floating
IPs can be limited under the policy bandwidth rules. This is
suit for all kinds of L3 routers: DVR, DVR with SNAT HA, L3 HA
and Legacy.

API update router gateway json:
{
router": {
  "external_gateway_info": {
    ...
    "qos_policy_id": "policy-uuid"
    }
  }
}

Depends-On: https://review.openstack.org/#/c/567497/

Partially-Implements blueprint: router-gateway-ip-qos
Closes-Bug: #1757044
Related-Bug: #1596611
Change-Id: I26e22bce7edd1f93b2ac0048b61b14f858938537
2018-12-01 17:45:46 +08:00
Zuul
a16ea46224 Merge "Add propagate_uplink_status to port" 2018-12-01 08:09:59 +00:00
Zuul
934a54a2ed Merge "Secure dnsmasq process against external abuse" 2018-11-30 21:57:33 +00:00
Hongbin Lu
f0678b9b09 Add propagate_uplink_status to port
Introduce an attribute 'propagate_uplink_status' to port.
This attribute can be implemented for VF port to indicate if the VF
link state should follow the state of the PF.

Note: ML2 extension driver loaded on request via configuration:

  [ml2]
  extension_drivers = uplink_status_propagation

Other related patches:
* neutron-lib: https://review.openstack.org/#/c/571821/
* tempest test: https://review.openstack.org/#/c/586719/
* OSC: https://review.openstack.org/#/c/586684/
* neutronclient: https://review.openstack.org/#/c/586712/

APIImpact Add 'propagate_uplink_status' attribute to 'port' resource

Change-Id: Ie8260c332e24c1880f9f82e6b6dacca8415be842
Close-Bug: #1722720
2018-11-29 19:33:16 +00:00
Zuul
4b7a070b3f Merge "Add protocol to port_forwarding uniq constraint" 2018-11-29 19:00:33 +00:00
Jens Harbott
0fce3ca2c1 Secure dnsmasq process against external abuse
Currently any dhcp agent instance will work as an open resolver. For
deployments using publicly routed addresses for tenant networks, this
allows the agent being abused in dDoS attacks, see [1].

By setting the `--local-service` option dnsmasq will filter DNS queries
and reply only to queries from directly attached networks.

[1] https://bugs.launchpad.net/neutron/+bug/1501206

Closes-Bug: 1501206
Change-Id: I76d810aad2ce0f15a88bd798963012fa0efca74e
2018-11-28 15:49:25 -05:00
lizheng
e17dac3ae9 Add protocol to port_forwarding uniq constraint
Floating IP port forwardings with different protocols can not have
the same internal or external port number to the same VM port. But
we can have different application servers, for instance TCP server
and UDP server, listen to the same port at same time.

This patch adds the protocol attribute to the DB uniq constraint
to allow creating different protocol port forwardings with same
internal or external port number.

Co-Authored-By: LIU Yulong <i@liuyulong.me>
Closes-Bug: #1799155
Change-Id: Ifbb5f3ee2473aac98982bff0d2e6bb9b3e5ab5d6
2018-11-06 23:03:06 +08:00
Zuul
c4f081d024 Merge "supported_vnic_type configurable for sriov" 2018-11-06 04:14:20 +00:00
Zuul
2a7079ddd1 Merge "notification: Add 'status' to agent after_create/update" 2018-11-05 21:25:10 +00:00
Lajos Katona
d860109bf6 supported_vnic_type configurable for sriov
Now supported_vnic_types is hardcoded to the mechanism drivers, but that
can depend on several factors, like type of the NIC, admin decision,
etc.
With this patch we put the right to decide which vnic types are
supported for ovs agent into the hands of the admin, by allowing
blacklisting items from the mechanism driver specific list.
Background: http://eavesdrop.openstack.org/meetings/neutron_qos/2018/
neutron_qos.2018-07-31-15.00.log.html#l-58

Change-Id: Iad9e2e966df53b4164d2a56a93215c69825b5241
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
2018-11-05 11:40:13 +01:00
Zuul
eb8759aa98 Merge "Introduce floating IP pool resource" 2018-10-30 06:54:59 +00:00
Zuul
03310bd9b0 Merge "Add neutron-status upgrade check command framework" 2018-10-30 04:57:24 +00:00
Kailun Qin
1d98f0a7d4 Event driven periodic resync task for DHCP agents
The DHCP agent will resync its state with Neutron to recover from any
transient notification or RPC errors. Currently, the periodic resync
task waits on a timer to determine whether a re-sync is necessary. The
interval between attempts by default is 5 seconds and can be longer
thru config. This may cause a potentially long delay before an agent
gets new work via an agent_updated RPC call.

The idea of this RFE is to change the timer based periodic resync task
into an event driven one. It also proposes a new DHCP agent config
option "resync_throttle" to ensure the minimum interval taken between
resync state events to avoid too frequent resyncing. In this way, we
could force the agent to act on the resync request immediately therefore
decreasing how much time is needed before DHCP services are available.

Co-authored-by: Allain Legacy <Allain.legacy@windriver.com>

Closes-Bug: #1780370
Change-Id: Ie9d758ba5f750a38dc19ea5ce8b2c6b414f9ef80
2018-10-26 01:24:24 +08:00
Bence Romsics
8bd5ecd4fc notification: Add 'status' to agent after_create/update
Make the same status information available to notification consumers
(resource==agent, event==after_create/after_update) as it was already
available where the notification is sent in class AgentDbMixin.

Change-Id: Ie74091da934c7e49fd29ae4c6f930a7eb47e14b2
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
2018-10-25 15:34:06 +02:00
Slawek Kaplonski
152364dbc8 Add neutron-status upgrade check command framework
This adds basic framework for neutron-status upgrade check commands.

For now it has only "check_nothing" check implemented.
Real checks can be added to this tool in the future.

Depends-On: https://review.openstack.org/#/c/610061/

Change-Id: Ib08119e1bb8de80856edb6a39769d4bc9d98c587
Story: 2003657
Task: 26144
2018-10-25 12:11:27 +02:00
Zuul
e5726c7bda Merge "sriov-agent: Report resource info in heartbeat" 2018-10-24 16:27:31 +00:00
Zuul
0fdec013de Merge "supported_vnic_type configurable for ovs" 2018-10-24 04:00:36 +00:00
Zuul
5273f85eec Merge "ovs-agent: Report resource info in heartbeat" 2018-10-22 20:03:06 +00:00
Lajos Katona
a85c5ed3fb supported_vnic_type configurable for ovs
Now supported_vnic_types is hardcoded to the mechanism drivers, but that
can depend on several factors, like type of the NIC, admin decision,
etc.
With this patch we put the right to decide which vnic types are
supported for ovs agent into the hands of the admin, by allowing
blacklisting items from the mechanism driver specific list.
Background: http://eavesdrop.openstack.org/meetings/neutron_qos/2018/
neutron_qos.2018-07-31-15.00.log.html#l-58

Change-Id: I63e562e2eccc5b02c1c767d6a2c28cb803131e99
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
2018-10-15 20:35:49 +02:00
Brian Haley
cf37563c83 Remove deprecated vsctl ovsdb_interface api
This was deprecated in https://review.openstack.org/#/c/503070/
so remove all the vsctl-related code, leaving just the native
ovsdb api.

Also removed renamed ovs_vsctl_timeout value, which was changed
to ovsdb_timeout in https://review.openstack.org/#/c/518391/

Change-Id: I50dfcea3deb41df1bd01fd06b76522453a6ba50b
2018-10-11 14:59:34 -04:00
Bence Romsics
d7d433f729 sriov-agent: Report resource info in heartbeat
Example config for sriov-agent:

sriov_agent.ini:
[sriov_nic]
physical_device_mappings = physnet0:ens5,physnet0:ens6
resource_provider_bandwidths = ens5:100000:100000,ens6:100000:100000

Agent configurations now includes 'resource_provider_bandwidths' and
'resource_provider_inventory_defaults'.

Change-Id: I1af798d404ed94cd30da3f92734bea911bbc3ac7
Co-Authored-By: Lajos Katona <lajos.katona@ericsson.com>
Depends-On: https://review.openstack.org/577220
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
2018-10-11 11:15:06 +02:00
Bence Romsics
f352f9faaa ovs-agent: Report resource info in heartbeat
Example config for ovs-agent:

ml2_conf.ini:
[ovs]
bridge_mappings = physnet0:br-test
resource_provider_bandwidths = br-test:100000:100000

Agent configurations now includes 'resource_provider_bandwidths' and
'resource_provider_inventory_defaults'.

Change-Id: Ib197573e5cdb60ef0db4e7a771c3179bf9d5bb95
Co-Authored-By: Lajos Katona <lajos.katona@ericsson.com>
Depends-On: https://review.openstack.org/577220
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
2018-10-11 11:08:07 +02:00
Hongbin Lu
4e3fb31919 Introduce floating IP pool resource
Add support for listing floating ip pools (subnets).
A new API resource ``floatingip-pools`` is introduced.
This API endpoint can return a list floating ip pools
which are essentially mappings between network UUIDs and
subnet CIDRs. Users can use this API to find out the pool
to create the floating IPs.

Related patches:
* neutron-lib: https://review.openstack.org/#/c/556674/
* tempest-plugin: https://review.openstack.org/#/c/562038/

APIImpact add floatingip pools api
Change-Id: Iaa995630645042520df67d95271e14f11ffcff8c
Partial-Bug: #1653932
2018-10-04 15:53:53 +00:00
Shashank Kumar Shankar
cfec395b8f Integration of Port Binding Level OVO
This patch integrates Port Binding Level OVO in /plugin/ml2/db.py
and introduces context instead of session for usage in object
operations.

Change-Id: Ifa779f5f70a7502bd96b34d64a84d272af2a6886
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Co-Authored-By: Anindita Das <anindita.das@intel.com>
Co-Authored-By: Slawek Kaplonski <slawek@kaplonski.pl>
2018-09-17 07:00:45 +00:00
Kim Bao Long
310bfa326f Fix incorrect log resources querying
This patch aims to fix a co-existence problem between security_group
and firewall_group log resources due to incorrect log querying from
database.

Change-Id: Ic60ad436e0fbb23cdae0e63eaeb73130ebf02089
Closes-Bug: #1787119
2018-08-29 08:12:56 +07:00