This change moves ConfigDict and ConfigFileFixture in order to use
them in fullstack and functional tests (in a follow-up change).
rabbit_virtual_host, rabbit_hosts, rabbit_userid, and rabbit_password
for oslo_messaging are now deprecated in the DEFAULT config section.
Those settings should be provided under the oslo_messaging_rabbit section
Previously, we used create_all() based on models. We don't use
create_all() in production code and there is no guarantee models and
scripts are in sync even though we have a good functional test that
validates that. There are still pieces that can't be compared by
* The EnvironmentDescription class describes an entire fullstack
environment (as opposed to the currently implemented host-only
descriptions). This will allow future patches to signify that a test
should set up an environment that supports tunneling, l2pop, QoS and
* Now, most fullstack fixtures (config and process ones, at least),
expect both the EnvironmentDescription for the current test and the
HostDescription for the 'host' the config/process is on. This allows
for easier and most robust future changes, as now adding a new
parameter to one of the description objects doesn't mean adding that
argument to a number of other objects which are using it.
* Changed HostDescription's default argument of l3_agent to False, since
adding new configurations and defualting them to True forces the
author to go through ALL the tests and explicitly turn them on/off.
However, defaulting new configurations to False only requires
explicitly turning them on, which we ought to do anyway.
* Full stack tests' fake VMs are represented via a namespace,
MAC, IP address and default gateway. They're plugged to an OVS
bridge via an OVS internal port. As opposed to the current
fake machine class used in functional testing, this new fake
machine also creates a Neutron port via the API and sets the
IP and MAC according to it. It also sets additional attributes
on the OVS port to allow the OVS agent to do its magic.
* The functional fake machine and the full stack fake machine
should continue to share commonalities.
* The fullstack fake machine currently takes the IP address
from the port and statically assigns it to the namespace
device. Later when I'll add support for the DHCP agent
in full stack testing this assignment will look for the dhcp
attribute of the subnet and either assign the IP address
via 'ip' or call a dhcp client.
* Added a basic L2 connectivity test between two such machines
on the same Neutron network.
* OVSPortFixture now uses OVSInterfaceDriver to plug the port
instead of replicate a lot of the code. I had to make a
small change to _setup_arp_spoof_for_port since all OVS ports
are now created with their external-ids set.
* Created a 'resources' subdir and moved all fixture files
* Split ML2ConfigFixture to the server-side ml2 configuration
fixture, and the OVS agent configuration fixture.
* Neutron process logs were using H:M:S format as their file name,
but when starting multiple agents of the same type my machine
was fast enough to do that in the same second so that different
processes were outputting to the same log file. No good!
Added ms to the log name format. I also changed the log time
from UTC to local timezone.
* Renamed and moved 'FullstackFixture' to neutron/tests/fullstack/
* Added a 'Host' abstraction that groups agents that report with
the same 'host' value. Hosts may be interconnected by the
environment via shared bridges.
* The 'Environment' class will accept global
attributes (This will be later filled with stuff like tunneling,
l2pop or other environment-level flags), and in this patch accepts
a list of host attributes (Configuration that may differ between
hosts like the l3 agent mode [legacy, dvr, dvr_snat]).
* Made OVS agent and L3 agent fixtures expose their bridges
so that I could interconnect them.
* Added a super simple L3 HA test to show that this entire thing
There is nothing Linux or agent specific in the function. I need to use
it outside agent code in one of depending patches, hence moving it into
better location while leaving the previous symbol in place, with
deprecation warning, for backwards compatibility.
oslo_utils.timeutils.strtime() is deprecated in version '1.6'.
We use datetime.datetime.isoformat() or datetime.datetime.strftime()
There were two broad issues with fixtures.
Firstly, the 'SafeFixture' workaround for resource leaks in fixtures
<1.3 is not needed if we depend on fixtures>=1.3.1. While testtools
may raise a TypeError when trying to query a fixture that failed to
setup, this is only ever a cascading failure - it will not cause
tests to fail, cause leaks, or cause tests to incorrectly pass. That
will be fixed in testtools soon to stop it happening (but as it cannot
affect whether a test passes or fails or leaks happen there is no
reason to wait for that). Leaks are seen with fixtures 1.3.0 still
because eventlet raises a BaseException subclass rather than an
Exception subclass, and fixtures 1.3.0 didn't handle that - 1.3.1 does.
Secondly, some of the fixtures had race conditions where things were
started and then cleanups scheduled. Where possible I've fixed those,
but some of them require more significant work to fully address.
The NetcatTester is a testing tool that can be used also in fullstack
tests so I think it should go there to avoid imports in fullstack tests
Tests for original helpers module was removed.
Following capabilities were added:
- used transport protocol is passed as a constant instead of bool
- src port for testing was added
- connection can be established explicitly
- change constructor parameters of NetcatTester
As a part of removing bool for protocol definition
get_free_namespace_port() was also modified to match the behavior.
Currenty useFixture(myfixture) ensures to call myfixture.cleanUp only
if myfixture.setUp succeed.
This change defines a workaround to ensure cleanUp call even if setUp
fails until testtools/fixtures support it: SafeFixture which ensures
cleanUp call if setUp fails and replaces fixtures.Fixture use by
SafeFixture. This workaround will be removed when the bug will fixed in
testtools and fixtures.
 testtools.TestCase.useFixture, fixtures.Fixture.useFixture
 see related bugs
mysql-python driver has been replaced by PyMySQL driver in neutron
code but MySQL related functional/fullstack tests try to use
mysql-python driver because of MySQLOpportunisticTestCase and tests
are skipped because mysql-python driver is no more available.
This change provides a backend implementation for mysql+pymysql, a base
base testcase MySQLTestCase using mysql+pymysql implementation
(currently oslo.db provides none of them but will in the future) and
replaces MySQLOpportunisticTestCase with MySQLTestCase.
"neutron-server isn't up yet" logs are useless because if
you time out when waiting for the server to start that information
will be in the trace. When you don't time out, the log is just spam.
Same reasoning for the "There are %d agents running!" log.
Also made the agents_count parameter mandatory for the
wait_until_env_is_up method because having a default of 0, or any
other default makes no sense. There's no reason to ever call that
method without specifying the agents_count. This method used
to be used with agents_count == 0 by the server to make sure
its up (And responding to REST calls), but the Neutron server
fixture now uses server_is_live method (Which calls list_networks)
In rare cases, concurrent workers may attempt to ensure a directory
exists. One may successfully create the directory while the other
gets an oserror that it already exists. This patch detects the
problem and returns successfully in both cases.
The goal of this doc is to communicate what are full stack tests,
how they benefit you and when would you write such a test.
Additionally I'd like to communicate the way forward, and gather
feedback about any areas in the code that can benefit from full
stack tests, and any additional thoughts!
Currently, it's up to the developer who wants to run full-stack on his
machine to make the directory in question (/opt/stack/logs). However,
this also means that the files don't get compressed at the end of a gate
run. Now, each full-stack test will have each own log directory in /tmp.
Once the logs are there, post_test_hook.sh can run 'gzip' on all the log
files before moving them to /opt/stack/logs on its own.
This patch introduces the last step of complete test separation for
full-stack tests - separate rabbitmq queues - in the form of rabbitmq
* This patch also renames EnvironmentFixture to FullstackFixture, for
There has been a problem with iproute package that resulted in errors
when deleting the namespaces, so deleting was turned off by default.
According to tests with iproute version 3.12.0 there is no such issue
so the option could be safely turned on by default.
Creating these utilities allows functional tests to mock them out more
easily to in order to change the namespace identification and cleanup
Currently, the full-stack framework has only one test which only uses
the neutron-server. This patch adds an actual test which makes sure that
once a router is created, an actual namespace is create for it. Since
this test requires 3 processes (neutron-server, l3-agent, ovs-agent),
existing full-stack code is modified to add more streamlined support for
Partially-Implements: blueprint integration-tests
The full-stack framework overrides the database connection string before
every test is started, but after the test it doesn't revert the string
back to what it was originally. Since after the test the database is
deleted, the string is not actually valid once the test finished, and
this conflicts with tests which are ran on the same job (specifically
the retargetable tests - see associated bug). The proposed patch saves
the original connection string and reverts it after the test finishes.
Change Ifca5615680217818b8c5e8fc2dee5d089fbd9532 was intended to
remove the neutron.tests.sub_base module, but a bad rebase means that
it was left in the tree.
This patch introduces the full-stack tests framework, as specified in
the blueprint. In short, this adds the neutron.tests.fullstack module,
which supports test-managed neutron daemons. Currently only
neutron-server is supported and follow-up patches will support for
Implements: blueprint integration-tests
Co-Authored-By: Maru Newby <email@example.com>