Neutron has implemented policy-in-code [1], which broke some of our
unit tests that were trying to load defaults from a neutron
policy.json file that no longer exists. Since policy defaults are now
in code, we do not need to load defaults from a file anymore, so this
removes that.
[1] f8984c66996c876d5bbd0cc0c4c47f567e515511
Change-Id: I862e27d9f2284415dcdc57074e2136603d79233d
The Stein supported runtimes proposal [1] has chosen py36 over py35
due to the communities move to testing on Ubuntu 18.04, where py36
is the default runtime. We update tox.ini and setup.cfg accordingly.
Python 3.6 handles dict order differently than in previous python
versions, so to get py36 to work this uses a sorted() call to make
the behavior consistent across python versions for our network
bridge cleanup and adjusts the corresponding test accordingly.
We also add a py37 option to the tox.ini to facilitate future-looking
tests per the resolution on keeping up with Python 3 releases [2].
The py35 option will eventually need to be removed from tox.ini, but
this keeps it for now in order to allow time for folks to transition.
As previously commented, automatic tox envs (pyXX) will use the python
version appropriate to that env and ignore basepython inherited from
[testenv], so we no longer specify a basepython explicitly for these.
[1] https://review.openstack.org/611080
[2] https://review.openstack.org/613145
Change-Id: Iebdc9357a79cae1923109fb5e37fefb8021af18d
Neutron added an optional host argument to the get_device_details RPC
method a long time ago [1] but a recent change [2] has made that no
longer really optional, at least for our usage. This adds host to our
calls so that VIF plug will not timeout with an error in the neutron
logs stating "Device %s has no active binding in host None".
[1] f7064f2b6c6ba1d0ab5f9872b2d5ad7969a64e7b
[2] 01bdb47199468805b714ce4c00c7492951267585
Closes-Bug: #1784950
Change-Id: Ida4102427405e17e42ddc463df4a63c2ba7dcb82
In anticipation of repointing our tox to use neutron master (rather than
staying one release behind via requirements, as we have been), we needed
to fix some pep8 errors that got un-ignored.
Change-Id: I1c25ebe62108e300c16c0a503f1dcc919259bf7d
Update translations for the Queens release.
Note: this is being cherry-picked from stable/queens to master
instead of the other way around. The translations were actually done
based on queens, so that is where they were originally merged. In
hindsight, they should also be merged in master so that it has the
latest translations as well, so we should have followed the normal
process of merging in master first, but too late for that now.
Change-Id: I03ab7bef6e9826631e45e3cb62c7f58600e3cd81
(cherry picked from commit eb5d4bf7f4)
Neutron was added to global-requirements [1] which breaks the tox
install of neutron. Neutron is now listed in requirements.txt and has
been removed as a dependency in tox.ini. One side effect is that we
based the policy.json path on the location of the neutron git directory
which is no longer the case. The test code has been updated to reflect
the new location of the policy file.
[1] https://review.openstack.org/#/c/550475/
Change-Id: Ibc42b8a835d1a5e6ce09640f5bdf833c2171a9a9
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by using neutron-lib's version of the module in
prep for Ia4a4604c259ce862597de80c6deeb3d408bf0e95
Change-Id: I7de0ceeeeee4dc19cfce499dab1697c7e5cc2568
This patchset handles maxcapacity in binding profile of Neutron port
for SR-IOV. maxcapacity is retrieved from binding profile and pushed into
vif object. Additional changes will be in nova_powervm vif driver to
gather and validate and pass onto pypowervm layer.
In both nova_powervm and pypowervm layers of code, maximum capacity is
validated. Maximum capacity cannot be less than capacity (minimum capacity)
and maximum capacity cannot be greater than 1 (100 %).
Change-Id: I0eb627b94e66158c68fddd7a2a5fc90467efc20d
Found some copyrights that were not updated the last time the
file was changed. Updating copyright dates to reflect the most
recent change.
Change-Id: Id9a2e718969fe5df57b6f8bcc8cbba71daab6f62
The ml2 plugin driver API was rehomed into neutron-lib with commit
Ifc829953ef4d5859c3475903965dc08aba42fd9c and the API was shimmed in
neutron with I86a10091b55d1123e8d16f16155e0312bb10e54c. In prep for
neutron consumption Ice49572e217eeaf820e48d40f2251d08766490b5 this
patch switches over to lib's version of the respective code.
Change-Id: Ic0f48e5c4659ceeb05fb878eddc2b3670290009a
Earlier changes [1] removed log translation markers from the code,
but the log translation po files were left behind. This removes
those files.
[1] 94f64c37b4
Change-Id: I7e5ef5b0510210eb76b27aa3248f9c130397d206
1.As mentioned in [1], we should avoid usingg
six.iteritems to achieve iterators. We can
use dict.items instead, as it will return
iterators in PY3 as well. And dict.items/keys
will more readable. 2.In py2, the performance
about list should be negligible, see the link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
Change-Id: I4b541390f6101619ed43cfdf24d9c693413cc853
Change Icdeabcc504ebcf3a0823587b042595ede420ce84 re-homed constants from
neutron.plugins.common.constants to neutron_lib.constants, but forgot to
fix up a couple of docstrings.
Change-Id: I4569e7c01b52cb2325ed8032e18dce2b9314e1d4
During startup of neutron agents - SEA and SR-IOV, a check should be
performed in sync with nova_powervm to verify if VIOS is available
in the environment. If not, agent startup should be aborted.
This patchset adds a check for VIOS's active status in agent_base
initialization.
Change-Id: I93530bbbad61c9da233c7bd958ede97de7835a08
Closes-Bug: #1694564
heal_and_optimize flow retrieves a list of vifs
using a method in util module. For SR-IOV agent,
_find_vifs methid is invoked, it is common to CNA
and VNIC objects. The vswitch_id is retrieved and
validated, since vswitch_id is not present in VNIC
object, getattr call fails. Since this method is
common to both CNA and VNIC, a default return value
of None should be included in the list of parameters.
networking_powervm/plugins/ibm/agent/powervm/util.py
_find_vifs method :
Instead of getattr(vif, 'vswitch_id')
getattr(vif, 'vswitch_id', None) should be used.
Change-Id: I38f1662801ad773408ab581c77161677b1cb6b36
Closes-Bug: #1691584
Some constants have been re-homed to neutron-lib, and will soon be
removed from neutron itself. This change set alters imports accordingly
in anticipation.
Change-Id: Icdeabcc504ebcf3a0823587b042595ede420ce84
We clearly hadn't tocuhed this project in a while. It failed
verification in a number of ways. In this change set, we:
- Remove log translation (now forbidden by hacking rule);
- Fix UT broken by new log messages introduced in neutron
(I16c4a9da6ae0e32f82d1e454500e2d7236169238);
- Move tox python3 version from 3.4 to 3.5;
- Remove pep8 ignores for docstring formats (H404,H405) and fix same;
- Fix incorrect usage of LOG.exception;
- Minor miscellaneous formatting cleanup.
Change-Id: I58411834efbd196da9dbea14b1acbe6d63d5c9b6
The current implementation of list_vifs will return a VIF even if the
VIF is not intended for an SEA. This change filters VIFs that aren't
in use by SEAs on either the RMC management switch or an Open vSwitch
and removes them from the list_vifs return.
Change-Id: I0611a2a647034f239b532cebe709566c005363f7
Closes-Bug: #1683381
Updates the import from neutron.agent.common.config to new location
neutron.conf.agent.common in repsonse to a change in neutron [1]
[1] https://review.openstack.org/#/c/343045/
Change-Id: I74746a92dbcac63e15fb0e0bebf7c68cb722a176
Invoke a heal_and_optimize if SR-IOV physical port config change events
are received.
Closes-Bug: #1658978
Change-Id: I1eb4bb1d802eee0a455acc9531f7547b48b331b0
The context.py file has been moved from neutron to neutron-lib.
This updates the import statement accordingly.
Change-Id: I061c0b64e19f1daedb506f3a4cc34edfc6306b4c
This patch refactors networking-powervm to use portbindings
from neutron-lib. For more details see [1].
NeutronLibImpact
[1] https://review.openstack.org/422210/
Change-Id: I05588f7a9dd9f495d879c7ebcf70b2c5616c3a46
SR-IOV mechanism driver in networking_powervm/plugins/ml2/drivers/
mech_pvm_sriov.PvmSRIOVMechanismDriver does not include
'physical network' in the custom vif details. Method
customize_vif_details prepares a dictionary like this :
{'redundancy': 1, 'physical_ports': [u'U78C9.001.WZS05TG-P1-C5-T1'],
'vlan': '2200', 'port_filter': False, 'capacity': 0.02}
It does not include physical_network.
This is not a bug, but SR-IOV vif plug mechanism in
nova_powervm/virt/powervm/vif.PvmVnicSriovVifDriver.plug can use
physical_network value to calculate physical ports in migration
scenario (future work in nova_powervm).
Above mentioned plug mechanism currently uses get_physical_network method
in nova/network/model.VIF class. This method expects physical_network in
custom vif details prepared by mechanism driver.
This patchset fixes this by adding physical_network to vif details by
getting it from network segment.
Unit tests added.
Change-Id: I379c9deceb72e4d5ea17891120d065230578d59e
Closes-Bug: #1651129
VIFEventHandler creates ProvisionRequest and passed down to agents for
processing. Currently, there is nothing in ProvisionRequest to
distinguish it between CNA or VNIC related. Due to this, SEA agent
gets VNIC related ProvisionRequests and processes to provision vlan
on SEA. This needs to be avoided. This patchset introduces vif_type
attribute into ProvisionRequest.
Refer to https://review.openstack.org/#/c/396739/ for a change in
nova_powervm. vif_type is now part of the event generated when a vif
is plugged or unplugged. This patchset uses the newly introduced
vif_type in the event while generating ProvisionRequest.
A new abstract property has been added to base_agent to carry
corresponding vif_type. Both sea_agent and sriov_agent have been
updated to return appropriate vif_type property. If vif_type in
incoming event does not match with vif_type supported by agent,
ProvisionRequest is not generated. And so, agent is not notified to
provision device.
For example, in the case of SR-IOV based plug, event carries
vif_type as pvm_sriov, and SEA agent supports vif_type as pvm_sea.
No ProvisionRequest is generated and sea_agent provison device
function is not invoked, so no vlan is provisioned.
Change-Id: Ifccaf4ee9ce492a295958393baf48bb383319805
Closes-Bug: #1640564
exceptions was previously being imported from neutron.common. This
has now been deprecated in favor of importing from neutron_lib.
This change updates to the correct import.
Change-Id: Ic8175983ea3634139a12eebe94138ded5b146276
Add debug logs for when the agent reports updated bridge mappings back
to neutron. If a vNIC gets deployed on backing devices with the wrong
label (physical network name), these logs help figure out why.
Change-Id: I955a58b4da264dc5d7b38bc7d821a311363cdfcf
Removal of the setup_adapter helper via Change-Id
Ie8a13da0b0cdc29d350ed91d96ff2384430c2676 introduced compatibility
issues; this change set restores it. The setup_adapter method simply
assigns self.adapter a valid pypowervm.adapter.Adapter.
Change-Id: Ib3520c130119335f68d58b77fa3442322f6bfb2d
Closes-Bug: #1634526
This change set represents a major refactor of the neutron SEA and
SR-IOV agents.
- Proper abstract properties/methods allow much code to be consolidated
into the BasePVMNeutronAgent from the individual subclasses.
- Conversely, CNA-specific code, such as helpers for the
sea_agent-specific heal_and_optimize loop, is moved to the sea_agent
module.
- PVMRpcCallbacks methods are folded into the agent base class itself.
The agent is now the thing that's registered for those callbacks.
- The network_delete callback is gone (it wasn't being used).
- The port_update callback is a no-op by default. In the SR-IOV agent,
it's refreshing the physical port label:physloc mappings back to
neutron.
- The vnic_required_vfs and vnic_vf_capacity conf options are moved to
sriov_agent.
- There is no longer a short timer in the rpc_loop; everything is
event-driven. Thus, the polling_interval conf option is removed.
- The rpc_loop is common to both agents. It only runs
heal_and_optimize, on the configurable interval (default 30min).
- ProvisionRequest is moved into its own module, prov_req.py. It is now
a factory class capable of producing ProvisionRequest instances for:
- Custom VIF Events from the nova-powervm VIF driver;
- pypowervm Wrappers of VIF types (CNA, VNIC);
- The VIFEventHandler (formerly CNAEventHandler) is now driven by VIF
events produced by the nova-powervm VIF driver. This entails a
significant processing/performance/load improvement over the previous
code, which acted on all LPAR events.
- Utility methods (utils.py) are renamed and genericized to handle all
VIF types (CNA, VNIC) rather than just CNAs.
- Obsolete utility methods and their unit tests are removed.
- Unit tests overhauled and extended.
Closes-Bug: #1632400
Change-Id: Ie8a13da0b0cdc29d350ed91d96ff2384430c2676
When the SR-IOV agent receives a port update notice from neutron, it
polls the server waiting for the corresponding vNIC to be created before
marking the port as active. If the vNIC create errors or doesn't
happen, the agent would keep polling for that vNIC indefinitely.
With this change set, if the vNIC doesn't spring into existence within
20 minutes, the agent gives up and stops polling. (If whatever caused
the vNIC not to be created hasn't already triggered rollback/cleanup of
the corresponding compute process, that process also times out after 20
minutes.)
Closes-Bug: #1629559
Change-Id: I5fd640dfb799c51738aa3613852b4ac69decb31b
Commit 0fcfc379de removed the PVID looper,
but left references to the corresponding config option. This change set
removes the config option from sea_agent and references thereto from the
usage.rst and devstack README.rst.
Change-Id: I8e50e9455b53489af2a05b5223c2dd8d51e4758d
Closes-Bug: 1626257
Per armax recommendation in https://review.openstack.org/#/c/374425/,
moving away from the neutron hacking checks, using neutron_lib hacking
checks instead.
We need to bypass N530 (direct neutron imports not allowed) for the
forseeable future.
Change-Id: I05804844445575a55ab33273b88fd3a5ff9df179
Closes-Bug: #1626298
The PowerVM SR-IOV neutron agent was sending update_device_up (almost)
immediately in response to the update_port push from neutron. If the
resulting network-vif-plugged event arrives at the compute process
*before* the PlugVifs task starts waiting for it, we'll miss it and thus
time out "waiting" for the vif to get "plugged" (even though the vNIC
gets created successfully).
With this change set, when the sriov_agent's rpc_loop pulls a port off
the queue, it will search the REST server for a vNIC with the matching
MAC address. If not found, it'll push the port back on the queue and go
back to sleep. It only sends the update_device_up once the vNIC is
found.
This is not an ideal solution for a number of reasons, including:
1) We have no way to know if the VNIC creation failed, in which case we
may wind up with stale port entries in the queue forever.
2) The current mechanism for finding the vNIC is very heavy: a feed GET
of all LPARs followed by a VNIC feed GET on each. We should find a more
lightweight way of doing this.
Change-Id: I99b3d36e3aabd145f598e5c5dbcaa3e64df00fc5
With the ability to pass the VLAN over to nova, via the mechanism driver
and vif binding details, we no longer need the PVID Updater.
Previously the function would wait for nova to provision a VIF on VLAN
1. Then the sea networking agent would detect this new port, update the
SEA to support the VLAN (if needed) and then update the VLAN on the VIF.
Now that we can pass the VLAN to Nova at the VIF create time, that last
step is not needed.
Change-Id: Ifc5bfe8b5e118f02faf0556cc3e67d11a66e2b25
Previously, the VF capacity for SR-IOV-backed vNICs had to come from the
vif's binding:profile; else it would default to None (allowing the
platform to default it).
With this change, the capacity can also be specified in the ml2
configuration file via the new 'vnic_vf_capacity' option in the AGENT
section. The binding:profile value takes precedence, followed by the
configuration value. The configuration default is None if unspecified.
Since binding:profile values will not be supported in the current
release (see Related-Bug), references thereto are removed from the
devref.
Related-Bug: 1615128
Change-Id: I91dc7cf5820bbdd66af80e8f492a80d03c21c02e
A new mechanism driver is included in this patchset. This has a
corresponding SR-IOV agent implementation as well.
The mechanism driver supports vnic_type direct. Both flat and vlan
network types are suported. The mappings reference maintained by this
driver is derived from agent configuration. Agent periodically updates
mechanism driver with a list of physical networks derived from port
labels of physical ports of SR-IOV adapters. Mechanism driver validates
incoming port for binding if it has valid physical network attached to
it. To do this, the check_segment_for_agent method is used by
try_to_bind_segement_for_agent. Under the covers, mappings from the
agent are matched with physical network attribute of the segment.
This mechanism driver also provides vlan information during binding to
vif plug mechanism on nova side. To do this, this mechanism driver
provides binding:vif_details with vlan id. On nova side, during vif plug
operation, vlan can be retrieved from ['details']['vlan'] attribute of
vif.
This patchset also includes changes to setup.cfg. A console script
driver to setup networking-powervm-sriov-agent along with its sea
counterpart.
setup.cfg changes also include Mechanism driver setup in ml2.conf
and entry_points.txt file in runtime environment.
Implements: bp/powervm-sriov
Change-Id: I87f6ed095d62afe4c683c2688fa27ddf367a19f6
If the system has an excessive amount of VLANs to clean up, the heal and
optimization path can block the requests coming in for an extensive
period of time. This may eventually lead to VIF plugging timeouts on
the nova component.
This change set will stop doing deletes of VLANs after three passes. It
logs the remaining VLANs that need to be cleaned up, but does not
actually execute the delete. It assumes the next pass through will do
the appropriate cleanup.
Change-Id: I97d74531b542177b105ca2625949075b1c8fbf24
Closes-Bug: #1612047
The related_href was used to parse out the bridge_mappings table.
Unfortunately, *how* you queried for the data identified what the URI
would be. An earlier change to remove some host_uuid's ended up
malforming the URI and breaking the bridge_mapping logic.
This change fixes how we look at the URIs to see if a SEA is on a given
VIOS. This should be more reliable moving forward.
Change-Id: I98f76a44db0f61d3e8ed4d3204c1e086bfe00fab
Closes-Bug: #1612047
This change set reduces the number of host_uuids being passed around in
networking-powervm. It also switches many of the calls from the very
verbose adapter.read() format to the wrapper get method.
Change-Id: If6f56d14f4161ea238d490ea44ecd74e4f468e8f
Closes-Bug: #1607338
The VIOS type element can be running true Client Network Adapters (VEAs
without additional trunk adapters) on it. The heal and optimize code
should take these adapters into account.
This change set updates some of the utility methods to account for VIOS
types. It also updates the heal and optimize code to take these VEAs
into account for the whole system (LPARs and VIOSes).
Change-Id: If21246b1d9c140a9b97201ced95f578a86b5c62d
This change allows for the utils list_lpar_uuids to take in a new
parameter that can exclude the mgmt UUID. This is an important change
because we may wish to explicitly exclude it to avoid erronerous logging
messages. This differs from pypowervm's base implementation where it
explicitly includes it (if that explicit include is not set to True, it
does not explicitly exclude it).
Reworks to use the pypowervm get_partitions method where able.
Closes-Bug #1602598
Change-Id: Ic47b4f20c04a7f98517a77cdb31d044a1f8f63e8
Zuul