Add devref that explains fullstack testing and its direction

The goal of this doc is to communicate what are full stack tests,
how they benefit you and when would you write such a test.

Additionally I'd like to communicate the way forward, and gather
feedback about any areas in the code that can benefit from full
stack tests, and any additional thoughts!

Change-Id: Ifd4ff9be0ed0184a49df6566d238c31a328cd23f

doc/source/devref/fullstack_testing.rst

@@ -0,0 +1,93 @@
+==========================
+Neutron Full Stack Testing
+==========================
+
+Why?
+====
+
+The idea behind "fullstack" testing is to fill a gap between unit + functional
+tests and Tempest. Tempest tests are expensive to run, difficult to run in
+a multi node environment, and are often very high level and provide little
+indication to what is wrong, only that something is wrong. Developers further
+benefit from full stack testing as it can sufficiently simulate a real
+environment and provide a rapidly reproducible way to verify code as you're
+still writing it.
+
+How?
+====
+
+Full stack tests set up their own Neutron processes (Server & agents). They
+assume a working Rabbit and MySQL server before the run starts. Instructions
+on how to run fullstack tests on a VM are available at TESTING.rst:
+http://git.openstack.org/cgit/openstack/neutron/tree/TESTING.rst
+
+Each test defines its own topology (What and how many servers and agents should
+be running).
+
+Since the test runs on the machine itself, full stack testing enables
+"white box" testing. This means that you can, for example, create a router
+through the API and then assert that a namespace was created for it.
+
+Full stack tests run in the Neutron tree with Neutron resources alone. You
+may use the Neutron API (Keystone is set to NOAUTH so that it's out of the
+picture). VMs may be simulated with a helper class that contains a container-
+like object in its own namespace and IP address. It has helper methods to send
+different kinds of traffic. The "VM" may be connected to br-int or br-ex,
+to simulate internal or external traffic.
+
+Full stack testing can simulate multi node testing by starting an agent
+multiple times. Specifically, each node would have its own copy of the
+OVS/DHCP/L3 agents, all configured with the same "host" value. Each OVS agent
+is connected to its own pair of br-int/br-ex, and those bridges are then
+interconnected.
+
+.. image:: images/fullstack-multinode-simulation.png
+
+When?
+=====
+
+1) You'd like to test the interaction between Neutron components (Server
+   and agents) and have already tested each component in isolation via unit or
+   functional tests. You should have many unit tests, fewer tests to test
+   a component and even fewer to test their interaction. Edge cases should
+   not be tested with full stack testing.
+2) You'd like to increase coverage by testing features that require multi node
+   testing such as l2pop, L3 HA and DVR.
+3) You'd like to test agent restarts. We've found bugs in the OVS, DHCP and
+   L3 agents and haven't found an effective way to test these scenarios. Full
+   stack testing can help here as the full stack infrastructure can restart an
+   agent during the test.
+
+Short Term Goals
+================
+
+* Multinode & Stability:
+    - Interconnect the internal and external bridges
+    - Convert the L3 HA failover functional test to a full stack test
+    - Write a test for DHCP HA / Multiple DHCP agents per network
+* Write DVR tests
+* Write L3 HA tests
+* Write a test that validates L3 HA + l2pop integration after
+  https://bugs.launchpad.net/neutron/+bug/1365476 is fixed.
+* Write a test that validates DVR + L3 HA integration after
+  https://bugs.launchpad.net/neutron/+bug/1365473 is fixed.
+
+None of these tasks currently have owners. Feel free to send patches!
+
+After these tests are merged, it should be fair to start asking contributors to
+add full stack tests when appropriate in the patches themselves and not after
+the fact as there will probably be something to copy/paste from.
+
+Long Term Goals
+===============
+
+* Currently we configure the OVS agent with VLANs segmentation (Only because
+  it's easier). This allows us to validate most functionality, but we might
+  need to support tunneling somehow.
+* How do advanced services use the full stack testing infrastructure? I'd
+  assume we treat all of the infrastructure classes as a publicly consumed
+  API and have the XaaS repos import and use them.
+* Currently we configure the Neutron server with the ML2 plugin and the OVS
+  mechanism driver. We may modularize the topology configuration further to
+  allow to rerun full stack tests against different Neutron plugins or ML2
+  mechanism drivers.

doc/source/devref/index.rst

@@ -21,7 +21,8 @@ Developer Guide
 In the Developer Guide, you will find information on Neutron's lower level
 programming APIs. There are sections that cover the core pieces of Neutron,
 including its database, message queue, and scheduler components. There are
-also subsections that describe specific plugins inside Neutron.
+also subsections that describe specific plugins inside Neutron. Finally,
+the developer guide includes information about Neutron testing infrastructure.
 
 
 Programming HowTos and Tutorials
@@ -51,6 +52,12 @@ Neutron Internals
    oslo-incubator
    callbacks
 
+Testing
+-------
+.. toctree::
+   :maxdepth: 3
+
+   fullstack_testing
 
 Module Reference
 ----------------

neutron/tests/fullstack/README

@@ -0,0 +1 @@
+Please see neutron/doc/source/devref/fullstack_testing.rst.