Commit Graph

42 Commits (0634dcc6d0f08c18c69a2c360a2c5c0581ec7bb6)

Author SHA1 Message Date
Slawek Kaplonski 87a7a5e32e [Fullstack] Add segmentation_id update test
This patch adds new fullstack test which spawns 2 "hosts" and 2 "VMs" on
those hosts. Both VMs are plugged to the vlan network with some
segmentation id. Next segmentation id of the network is updated and test
ensures that new vlan id is configured in the physical bridge on both
"hosts" and connectivity between VMs still works fine.

Test runs only with Openvswitch agents as Linuxbridge doesn't supports
live update of the segmentation_id in the network.

Change-Id: I459aac7f4e9afe679d8ece1c27d0be49cec8e4ff
2 years ago
Brian Haley 8126f88894 Complete removal of dependency on the "mock" package
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.

This completes removal of all references to "import mock",
changing to "from unittest import mock" in fullstack and
functional tests.

Added a hacking check to enforce it in future patches.

Change-Id: Ifcaf1c21bea0ec3c35278e49cecc90a101a82113
3 years ago
Slawek Kaplonski 458fd224db [Fullstack] Fix waiting for packet with DSCP mark in test
In test_dscp_marking_packets, sender vm is sending ICMP packets to the
Before this patch even if first ICMP packet wasn't send properly, test
failed due to exception raised in execute method.

It can happend sometimes that first packet will fail to be send as there
can be e.g. some configuration on agent's side not done yet. So we
should try to ping receiver_vm for 60 seconds instead of failing on
first attempt.
This patch changes that behaviour in test.

Change-Id: Ic84e1ef507f017516d47a6b66e646e7e7b5ded8d
Closes-Bug: #1852724
4 years ago
Brian Haley b79842f289 Start enforcing E125 flake8 directive
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues.  Didn't think it was going to be
close to 100 files when I started.

Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
4 years ago
Boden R 9bbe9911c4 remove neutron.common.constants
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.


Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
4 years ago
Slawek Kaplonski 8c2a16796b Fix fullstack test_dscp_marking_packets test
Fullstack test test_dscp_marking_packets spawns tcpdump
process inside one fake vm's namespace and then tries to
ping this instance from second one.
After that it checks if tcdump captured any packet marked with
specific DSCP mark.

Ping was done usually only once (1 packet) because
it was done by using vm.wait_until_ping() method, so it could
happen sometimes that ping was send before tcpdump actually
started captuting traffic.
In such case test failed because there was no any packet captured.

This patch changes this by:
1. Start tcpdump async process with block=True, so it should be
   already really started before test will go to the next steps,
2. Send always 10 packets instead of (usually) only one.

In addition this patch adds logging of captured tcpdump's stdout
and stderr streams. It may help debugging issues with this test in
the future.

Change-Id: I23bbde59af0250267843623dde2c5407059d9db2
Closes-Bug: #1818335
4 years ago
Lucian Petrut 89915a752e Trivial: Move platform independent modules to common dir and are now platform
independent, for which reason we can move them to

Note that a few subprojects are using async_process. We'll use
debtcollector so that we don't break those projects, while logging
a deprecation warning.

Change-Id: I6a7418cb8680cd71fe16c5d98b9b09ef2d260d37
5 years ago
IWAMOTO Toshihiro 0f180d8e2e Adopt hacking 1.1.0
This incorporates flake8 2.6.x and pycodestyle will be used
instead of older pep8.  This ensures future python3 compatibility
and a bit better code styling.

Change-Id: Ia7c7c5a44727f615a151e1e68dd94c7ed42f974f
5 years ago
Sławek Kapłoński 6407a71d06 [Fullstack] Unmark test_dscp_marking_packets as unstable
Bug 1733649 is now closed with patch

This commit removes decorator which marked fullstack test
as unstable.

This commit removes also some additional logs of tcpdump's
output added only as temporary "debug" solution.

Change-Id: I0343b1b4fd8d6eed1c26838bf70bce9868e6ee0c
Related-Bug: #1733649
5 years ago
Sławek Kapłoński 386ec26b3d [Fullstack] Change how DSCP marking packets are tested
In fullstack test test_dscp_marking_packets two fake hosts are
created, each of them has got one fake instance (called sender and
Instance called "sender" sends ICMP packets to instance called
"receiver". In receiver's namespace tcpdump process is spawned with
filter to match only packets marked with specified DSCP value.
After sender instance successfully pings receiver, tcpdump process
is killed and stdout from it was examined to search logged ICMP
packets which were send from sender's IP to receiver's IP address.
That check was failing sometimes as is described in bug report.

It was failing when tcpdump doesn't capture any packets so there was
nothing on stdout from it. But even in such case tcpdump reports on
stderr summary of packets and it was like below:

0 packets captured
6 packets received by filter
0 packets dropped by kernel

which means that packets matching our filter was received by tcpdump but
were probably not processed yet. See [1] for more details.

So this patch changes filter used in tcpdump and way how its output is
Now in filter expression there are added src and dst IP addresses also so
it will match only packets send from sender instance to receiver instance.
After that tcpdump's stderr output is examined with regex to check if
line like:

X packets received by filter

is there and if X value is different than 0. If so, it means that
tcpdump received packets with wanted DSCP mark and test should pass.


Change-Id: Ia3522237dc787edb90d162ac4a5535ff5d2a03d5
Closes-Bug: #1733649
5 years ago
Sławek Kapłoński b841a7f5d9 [Fullstack] Shutdown tcpdump process gracefully
In fullstack test for dscp marking packets tcpdump is used to capture
packets send from one namespace (Fake VM) to another one.
When tcpdump is closed quickly with SIGKILL signal it may not displays
all packets and that might cause an issue with failed dscp marking test.

Now tcpdump will be kill with SIGINT signal so it should display
all captured packets to the stdout.
Also it should now return to stderr summary of all captured/dropped
packets which can be helpful in future debugging of issue with those
tests if that will not solve the problem.

Change-Id: Ib2d588d081a48f4f2b6e98a943bca95b9955a149
Related-Bug: #1733649
6 years ago
Sławek Kapłoński 7d5247bc4c [Fullstack] Additional log of tcpdump stderr output
This patch adds additional logging of tcpdump stderr output in
DSCP marking packets tests.
It is required to figure out why those tests are failing sometimes.

Change-Id: I02fe64bf817b47970e4e073f48ea8dea53bdd4f5
Related-Bug: #1733649
6 years ago
Sławek Kapłoński 034934aaf9 [Fullstack] Log tcpdump output lines in DSCP mark tests
This patch adds additional logging of captured tcpdump output during
dscp mark packets tests.

Change-Id: I1ab5727ea70afbe19d2557b9b5c4d77500658cbc
Related-Bug: #1733649
6 years ago
Sławek Kapłoński 69b8588a19 Remove translation of help messages from tests
There is no need to translate help message for config option
which is added and used only in tests.

Change-Id: I8b8bdd4131e6183245bcd2556b2107d769fd6427
6 years ago
Sławek Kapłoński 2d0d1a2d76 Add support for ingress bandwidth limit rules in ovs agent
Add support for QoS ingress bandwidth limiting in
openvswitch agent.
It uses default ovs QoS policies on bandwidth limiting

DocImpact: Ingress bandwidth limit in QoS supported by
           Openvswitch agent

Change-Id: I9d94e27db5d574b61061689dc99f12f095625ca0
Partial-Bug: #1560961
6 years ago
Jakub Libosvar bc979efdb8 fullstack: Don't let dhcp agents failover
It turned out dhcp tests work only because agents are considered dead
after 10 seconds while they report to server every 60 seconds. This led
to calling network resync after agent revival and hiding the fact dhcp
agent is not capable of receiving any amqp messages.

This patch sets the report interval of agents to the half of
agent_down_time on server side and uses eventlet dhcp agent in order to
trigger eventlet monkey patching code.

Eventlet was behind the failure with messages not getting processed. As
[1] notes: "Note: If the “eventlet” executor is used, the threading and
time library need to be monkeypatched."

Because each port calls dhclient to obtain IP address and each dhclient
instance overwrites /etc/resolv.conf there was added a script that
generates fullstack-dhclient-script from an existing dhclient-script
before starting fulltstack tests. This generated script is passed to
each dhclient process running in fake fullstack machine using -sf


Related-bug: 1453350

Change-Id: I0336176b9c364fe3a95be5cef9e7a3af1ef9d7e9
6 years ago
John Schwarz 977d254cc6 Throttle SIGHUPs to keepalived
Multiple SIGHUPs in quick succession might cause the master keepalived
to forfeit its mastership (which will cause keepalived to remove IPs of
its external devices, severing connectivity). This can happen when, for
example, associating or disassociating multiple floatingips.

The patch makes the agent throttle SIGHUP sent to keepalived: the very first
SIGHUP is always sent; as for subsequent signals, they are delayed till
agent threshold is reached. (It's 3 seconds by default.)

As an example, when three consequent router updates trigger keepalived
respawn then:
* the very first signal is sent as usual;
* the second signal is deferred and sent in up to 3 seconds since the
  first signal;
* the third signal is ignored, though the change that triggered it will
  be correctly applied by the second signal handler when it is triggered
  after threshold delay.

If the last time a spawn request occurred is older than current-time
minus threshold then there is no delay.

Co-Authored-By: Jakub Libosvar <>
Co-Authored-By: Cedric Brandily <>
Co-Authored-By: Ihar Hrachyshka <>

Closes-Bug: 1647432
Change-Id: I2955e0de835458a2eea4dd088addf33b656f8670
6 years ago
Sławek Kapłoński a12de8975e Fullstack tests for DHCP agent HA
This patch adds fullstack tests for DHCP agent HA functionallity.

There are two tests added:
* One is checking if network will be properly
  rescheduled to new agent if existing DHCP agent will be down,
* Second is checking if network handled by 2 DHCP agents will have properly
  working DHCP service even if one DHCP agent will be down

Change-Id: Iaad373cafd6f83f2c1f8e7ac58dc70070e7aabaf
7 years ago
Sławek Kapłoński fd3bf3327c DSCP packet marking support in Linuxbridge agent
Linuxbridge agent uses iptable rules in POSTROUTING chain
in the mangle table to mark outgoing packets with the
DSCP mark value configured by the user in QoS policy.

DocImpact: DSCP Marking rule support is extended to the
           Linuxbridge L2 agent

Closes-Bug: #1644369

Change-Id: I47e44cb2e67ab73bd5ee0aa4cca47cb3d07e43f3
7 years ago
Ihar Hrachyshka 2db889b768 Revert "Fix file permissions"
This reverts commit eb14f66d39.

Turned out that fullstack uses those files to start services, instead of
relying on console scripts generated by pbr. So without those files
having +x permissions, fullstack tests using those agents completely

Change-Id: I14f98fdd244ecfedfa97fb31bff9763a49e5cfb0
7 years ago
Gary Kotton eb14f66d39 Fix file permissions

Change-Id: Ic687c35bc12820af93d1eb6c9e5b6b3a657be42e
7 years ago
Sławek Kapłoński ea7b51655f Add fullstack test for check DSCP marks outbounds
New fullstack test is added to check if packets which
should be sent from port with DSCP mark set are really
sent to another port with this DSCP mark.

This test uses IPv4 only.

Change-Id: I4b26c3c644eb6f2f7813658c99d16fbc3cc61e06
Closes-Bug: #1625570
7 years ago
Nam Nguyen Hoai 7e5d12d49d Clean import in code
This patch set modified some lines of code,
which import object, it should import module as
Openstack's recommendation [1].


Change-Id: I31ecf6a295ffdd089b3333dcec762483e7c081ad
7 years ago
Jakub Libosvar a626172706 Move wait_until_true to neutron.common.utils
We need to be able to re-use wait_until_true in tempest scenario tests.
There is tempest bug
that prevents us to do so.

Also wait_until_true is not linux specific so it makes more sense to
have it in common package.

Change-Id: Ib8b0e51dbd9edaa58391774d428a737836dfdf77
7 years ago
Nate Johnston 73546f8503 QoS DSCP fullstack tests
This patch introduces fullstack testing for the QoS/DSCP
Open vSwitch implementation. It depends on the python-neutronclient
patches, therefore it could not be merged in the main patch.

Co-Authored-By: Miguel Angel Ajo <>
Change-Id: I0ab6a1a0d1430c5791fea1d5b54106c6cc93b937
Closes-Bug: #1468353
Depends-On: I25ad60c1b9a66e568276a772b8c496987d9f8299
7 years ago
Frode Nordahl 773394a188 OVS: Add support for IPv6 addresses as tunnel endpoints
Remove IPv4 restriction for local_ip configuration statement.

Check for IP version mismatch of local_ip and remote_ip before creating

Create hash of remote IPv6 address for OVS interface/port name with least
posibility for collissions.

Fix existing tests that fail because of the added check for IP version
and subsequently valid IP addresses in _setup_tunnel_port.


Change-Id: I9ec137ef8c688b678a0c61f07e9a01382acbeb13
Closes-Bug: #1525895
7 years ago
Henry Gessau 4148a347b3 Use constants from neutron-lib
With this we enable the deprecation warnings by default.

Related-Blueprint: neutron-lib

Change-Id: I5b9e53751dd164010e5bbeb15f534ac0fe2a5105
7 years ago
Victor Stinner 6355173e4c Reuse to_utf8() and jsonutils.dump_as_bytes()
Reactor code:

* Reuse oslo_utils.encodeutils.to_utf8() instead of existing
  isinstance(text, six.text_type) test
* Replace jsonutils.dumps(obj).encode('utf-8') with
* Other minor bytes/Unicode changes

Change-Id: I03b8eff0fd70ab65ac66d6f3221e8ced0a56db17
7 years ago
David Shaughnessy a9a1943fde DSCP QoS rule implementation
This patch adds the front end and back end implementation of QoS DSCP.

Associated patches that are dependent on this one:

* python-neutronclient:
* openstack-manuals:
* API Guide:
* Heat:
  * Spec:
  * QoSDscpMarkingRule resource:
* Fullstack tests:

APIImpact - The API now supports marking traffic egressing from a VM's
            dscp field with a valid dscp value.

Co-Authored-By: Nate Johnston <>
Co-Authored-By: Victor Howard <>
Co-Authored-By: Margaret Frances <>
Co-Authored-By: James Reeves <>
Co-Authored-By: John Schwarz <>
Needed-By: I25ad60c1b9a66e568276a772b8c496987d9f8299
Needed-By: I881b8f5bc9024c20275bc56062de72a1c70c8321
Needed-By: I48ead4b459183db795337ab729830a1b3c0022da
Needed-By: Ib92b172dce48276b90ec75ee5880ddd69040d7c8
Needed-By: I4eb21495e84feea46880caf3360759263e1e8f95
Needed-By: I0ab6a1a0d1430c5791fea1d5b54106c6cc93b937
Partial-Bug: #1468353

Change-Id: Ic3baefe176df05f049a2e06529c58fd65fe6b419
7 years ago
Assaf Muller 1a1ccb8c94 Uniquely identify tunnel interfaces for fullstack tests
OVS agent tunnel interfaces are named via:
'%s-%s' % (tunnel_type, destination_ip)

This means that the tunnel interface name is not unique if
two OVS agents on the same machine try to form a tunnel with a
third agent. This happens during full stack tests that start
multiple copies of the OVS agent on the test machine.

Thus, for full stack tests, to make sure that the tunnel
interface names created by ovs agents are globally unique, they
will have the following format :
'%s-%s-%s' % (tunnel_type, hash of source IP, hash of dest IP)

Since this patch centralizes the formation of the tunnel interface
name in a dedicated method that is monkey patched by the full stack
framework, a unit test has been added for this method.

Co-Authored-By: Mathieu Rohon <>
Closes-Bug: #1467633
Change-Id: I991af6a5f982746cc297f0248454f803dfbb2daf
7 years ago
Bhagyashri Shewale 88e899f7a0 Fix module's import order
Made corrections in import order for built-in, third party and
project specific modules as per OpenStack import standards [1].


Change-Id: I899deefd6ee4732d6c0afd17a5afbe42b0fa37ba
7 years ago
Akihiro Motoki 2d8632e412 Use _ from neutron._i18n
Partial-Bug: #1520094
Change-Id: I874a4aa1d71d1f7034a1ff0b7450b419ef5c6864
8 years ago
Miguel Angel Ajo d56fea0a39 Fix the low level OVS driver to really do egress
It seems that the Queue + QoS + linux-htb implementation was really
limiting ingress by default. So this patch switches the implementation
to the ovs ingress_policing_rate and ingress_policing_burst parameters
of the Interface table.

Later in time we may want to revise this, to make TC & queueing possible,
but this is good enough for egress limiting.

Also, removed the _update_bandwidth_limit del+set on OvS QoS driver for
the bandwidth limit rule update, since that's not needed anymore.

Change-Id: Ie802a235ae19bf679ba638563ac7377337448f2a
Partially-Implements: ml2-qos
8 years ago
Jakub Libosvar 2aac5991ac Update port functional tests for qos agent
Change-Id: I4a1f4ec1ed9a9104fe7e5bbce66147d8ea6c0f27
Partially-Implements: quantum-qos-api
8 years ago
Miguel Angel Ajo af2e56d86c Functional test for QoS policy bandwidth rule update
Creates a port in a policy, and subsequently modifies the
bandwidth limit rule in the policy, then verifies that the
new limits are assigned to the port.

Change-Id: I23fe45ef08618ad91567feb1707028e0a0bfe0d6
Partially-Implements: ml2-qos
8 years ago
Carl Baldwin 8d4cbb3911 Append @randtoken to L3 agent namespaces in full stack tests
Change-Id: Ib180a5836f653385ec2877c50fbca6f850eff351
Closes-Bug: #1446261
8 years ago
Jakub Libosvar d3f13320be Remove neutron.tests.common.agents package
It seems like agents' package content was removed by commit
01a7ba19cf but package itself was left in
the tree.

Change-Id: I651f8010aa7c4af59ce403b099db7bc064364133
8 years ago
Miguel Lavalle 01a7ba19cf Refactor management of namespaces in the L3 Agent
Creates classes representing the 3 types of namespaces handled by the L3 agent:
router, snat and fip.

The scope of this change is:

- Creation and deletion methods are provided for each namespace class
- Creation and deletion of router and snat namespaces are moved to the router
  classes. These namespaces are now members of the corresponding router class
- Invocation of Fip namespace creation and deletion is left in the agent, since
  the agent owns it
- A context manager is provided to move the namespaces (router and snat)
  cleanup code out of the agent

A follow up patchset will add methods to create and delete interfaces in the
namespaces. These methods are intended to be used by the router classes

Change-Id: I54b14e593ded6b2990d57a3ae9d598a699ae133e
Partially-Implements: bp restructure-l3-agent
8 years ago
Assaf Muller 911f8b57f8 Change L3 agent AdvancedService class to be non-singleton
The idea behind the AdvancedServices (Metadata, *aaS) services
to be singletons was to (1) offer a way to get an instance of such
a service in a convinient manner, and to (2) ensure that only one
service registered per service type.

(1) Since the AdvancedService.instance method required an instance
of a L3 agent, this point was kind of missed. If you have a L3 agent
instance in your hand, just use it to access the service you're looking

(2) This is now fulfilled by asserting that only one driver is registered
(per type) in the .add method.

The motivation to make them non-singletons is that:
a) They don't need to be
b) The code is simplified this way
c) I've been facing crazy issues during functional testing where we're
   constantly instantiating L3 agents, and the (singleton) metadata
   service was referencing the wrong configuration object. The service
   was re-initialized during a test, screwing up the configuration of
   other mid-run tests.

Closes-Bug: #1425759
Depends-On: I155aece9b7028f1bdd3d9709facef392b38aec27
Depends-On: I199f6a967c64e921624b397fa5b7dfc56572d63a
Change-Id: I192e4d12380346c53c8ae0246371ccc41a836c4c
8 years ago
Assaf Muller 1e562decdc Update L3 agent drivers singletons to look at new agent
L3 agent drivers are singletons. They're created once, and hold
self.l3_agent. During testing, the agent is tossed away and
re-built, but the drivers singletons are pointing at the old
agent, and its old configuration.

Change-Id: Ie8a15318e71ea47cccad3b788751d914d51cbf18
Closes-Bug: #1404662
9 years ago
Carl Baldwin d7c5306360 Move classes out of
The file has become too large.  This patch is a simple
pure refactor to move some of the functionality in to other files
where things aren't too tangled up.  There is no functional change
with this patch and I avoided gratuitous other fixups in this patch in
order to make it easier to review.

I plan to follow up on the new l3_dvr and l3_agent_router modules with
more restructuring in the near future.

Partially-Implements: bp restructure-l3-agent

Change-Id: I3529fe4146c50c940f41eb26d0b5efc5870b3af9
9 years ago
John Schwarz a8edfc6e6d Test HA router failover
HA failover testing requires the ability for two L3 agents to
co-exist on the same machine. The following changes were made
to support this:

* Add l3 agent ability to simulate nested namespaces by suffixing
  each router namespace with '@host'. For example, if a router
  with id '1' is created, its namespace will be:
* Added test.common submodule which will hold common code which is
  shared between the different kind of tests (unit and funtional).
* Added tests.common.agents submodule which holds test-only agents, and
  added a test L3NAT agent to be used in the HA functional testings.

Co-Authored-By: Assaf Muller <>
Closes-bug: #1374946
Change-Id: I2185fe276d9d32b34c407396e7ed5d6e2801340c
9 years ago