Merge "Add devref that explains fullstack testing and its direction"
This commit is contained in:
commit
4dda680cf0
|
@ -0,0 +1,93 @@
|
|||
==========================
|
||||
Neutron Full Stack Testing
|
||||
==========================
|
||||
|
||||
Why?
|
||||
====
|
||||
|
||||
The idea behind "fullstack" testing is to fill a gap between unit + functional
|
||||
tests and Tempest. Tempest tests are expensive to run, difficult to run in
|
||||
a multi node environment, and are often very high level and provide little
|
||||
indication to what is wrong, only that something is wrong. Developers further
|
||||
benefit from full stack testing as it can sufficiently simulate a real
|
||||
environment and provide a rapidly reproducible way to verify code as you're
|
||||
still writing it.
|
||||
|
||||
How?
|
||||
====
|
||||
|
||||
Full stack tests set up their own Neutron processes (Server & agents). They
|
||||
assume a working Rabbit and MySQL server before the run starts. Instructions
|
||||
on how to run fullstack tests on a VM are available at TESTING.rst:
|
||||
http://git.openstack.org/cgit/openstack/neutron/tree/TESTING.rst
|
||||
|
||||
Each test defines its own topology (What and how many servers and agents should
|
||||
be running).
|
||||
|
||||
Since the test runs on the machine itself, full stack testing enables
|
||||
"white box" testing. This means that you can, for example, create a router
|
||||
through the API and then assert that a namespace was created for it.
|
||||
|
||||
Full stack tests run in the Neutron tree with Neutron resources alone. You
|
||||
may use the Neutron API (Keystone is set to NOAUTH so that it's out of the
|
||||
picture). VMs may be simulated with a helper class that contains a container-
|
||||
like object in its own namespace and IP address. It has helper methods to send
|
||||
different kinds of traffic. The "VM" may be connected to br-int or br-ex,
|
||||
to simulate internal or external traffic.
|
||||
|
||||
Full stack testing can simulate multi node testing by starting an agent
|
||||
multiple times. Specifically, each node would have its own copy of the
|
||||
OVS/DHCP/L3 agents, all configured with the same "host" value. Each OVS agent
|
||||
is connected to its own pair of br-int/br-ex, and those bridges are then
|
||||
interconnected.
|
||||
|
||||
.. image:: images/fullstack-multinode-simulation.png
|
||||
|
||||
When?
|
||||
=====
|
||||
|
||||
1) You'd like to test the interaction between Neutron components (Server
|
||||
and agents) and have already tested each component in isolation via unit or
|
||||
functional tests. You should have many unit tests, fewer tests to test
|
||||
a component and even fewer to test their interaction. Edge cases should
|
||||
not be tested with full stack testing.
|
||||
2) You'd like to increase coverage by testing features that require multi node
|
||||
testing such as l2pop, L3 HA and DVR.
|
||||
3) You'd like to test agent restarts. We've found bugs in the OVS, DHCP and
|
||||
L3 agents and haven't found an effective way to test these scenarios. Full
|
||||
stack testing can help here as the full stack infrastructure can restart an
|
||||
agent during the test.
|
||||
|
||||
Short Term Goals
|
||||
================
|
||||
|
||||
* Multinode & Stability:
|
||||
- Interconnect the internal and external bridges
|
||||
- Convert the L3 HA failover functional test to a full stack test
|
||||
- Write a test for DHCP HA / Multiple DHCP agents per network
|
||||
* Write DVR tests
|
||||
* Write L3 HA tests
|
||||
* Write a test that validates L3 HA + l2pop integration after
|
||||
https://bugs.launchpad.net/neutron/+bug/1365476 is fixed.
|
||||
* Write a test that validates DVR + L3 HA integration after
|
||||
https://bugs.launchpad.net/neutron/+bug/1365473 is fixed.
|
||||
|
||||
None of these tasks currently have owners. Feel free to send patches!
|
||||
|
||||
After these tests are merged, it should be fair to start asking contributors to
|
||||
add full stack tests when appropriate in the patches themselves and not after
|
||||
the fact as there will probably be something to copy/paste from.
|
||||
|
||||
Long Term Goals
|
||||
===============
|
||||
|
||||
* Currently we configure the OVS agent with VLANs segmentation (Only because
|
||||
it's easier). This allows us to validate most functionality, but we might
|
||||
need to support tunneling somehow.
|
||||
* How do advanced services use the full stack testing infrastructure? I'd
|
||||
assume we treat all of the infrastructure classes as a publicly consumed
|
||||
API and have the XaaS repos import and use them.
|
||||
* Currently we configure the Neutron server with the ML2 plugin and the OVS
|
||||
mechanism driver. We may modularize the topology configuration further to
|
||||
allow to rerun full stack tests against different Neutron plugins or ML2
|
||||
mechanism drivers.
|
Binary file not shown.
After Width: | Height: | Size: 29 KiB |
|
@ -21,7 +21,8 @@ Developer Guide
|
|||
In the Developer Guide, you will find information on Neutron's lower level
|
||||
programming APIs. There are sections that cover the core pieces of Neutron,
|
||||
including its database, message queue, and scheduler components. There are
|
||||
also subsections that describe specific plugins inside Neutron.
|
||||
also subsections that describe specific plugins inside Neutron. Finally,
|
||||
the developer guide includes information about Neutron testing infrastructure.
|
||||
|
||||
|
||||
Programming HowTos and Tutorials
|
||||
|
@ -51,6 +52,12 @@ Neutron Internals
|
|||
oslo-incubator
|
||||
callbacks
|
||||
|
||||
Testing
|
||||
-------
|
||||
.. toctree::
|
||||
:maxdepth: 3
|
||||
|
||||
fullstack_testing
|
||||
|
||||
Module Reference
|
||||
----------------
|
||||
|
|
|
@ -0,0 +1 @@
|
|||
Please see neutron/doc/source/devref/fullstack_testing.rst.
|
Loading…
Reference in New Issue