50 Commits

Author SHA1 Message Date
Jakub Libosvar
648936b8c8 docs: Update TESTING.rst about openvswitch requirements
Since we switched to use Ubuntu Cloud Archives in upstream gate, the
note in TESTING.rst is not longer valid for the gate. The condition to
trigger the test still remains in order to avoid false negative test
results for those that run functional tests with older openvswitch
version.

Change-Id: Ibff983efe0f452864d003913205cbbfa39ee7374
2017-04-19 07:57:17 +00:00
Jakub Libosvar
b96029f6a2 docs: Update TESTING.rst with gate information about ovs
We used to compile from source for functional and fullstack jobs. For
functional job we skip single test and avoid compiling. We still do
compile from source for fullstack job. This patch adds justification to
TESTING.rst document.

Change-Id: I60c272bba6fa9eeefa29dfa4faa69433989efde2
2017-04-12 16:31:31 +00:00
yuhui_inspur
d5c3361753 Fix some grammatical errors in TESTING.rst
TrivialFix.

Change-Id: I1a0c2b2da9d6ae0a52028420a98f5ae17e8d6855
2017-04-09 19:31:26 +00:00
Jenkins
e2550837bb Merge "TrivialFix: Modify the spelling mistake" 2017-02-03 09:59:05 +00:00
Yushiro FURUKAWA
e1f43a673e Fix typo
Trivial fix

Change-Id: Ia6b25c8b91db5efa839e1077c224c5c679c1f07e
2017-01-04 14:37:29 +09:00
Armando Migliaccio
76c9984426 Fix docs headers
Ic68858aaf1a519085dff394e6edc8c8fe6aeee6a forgot to move these
sections one level up. Without this, tox -edocs barfs.

Change-Id: I649ba20013ccbe25023b2c8c3a2014fb7be47d97
2016-11-30 11:54:34 -08:00
Jenkins
f6199da7c9 Merge "Deprecate run_tests.sh" 2016-11-22 06:08:21 +00:00
Armando Migliaccio
f5e30abcc5 Add rally hook to neutron devstack plugin
This allows us to configure neutron when running the rally job in
the gate. This effort stems from patch [1]. Blame Kevin for not
wanting to squash the two together.

[1] I12aaf6121b677e9696131601b3539a7091e2858c
Change-Id: I006957784ac7900021bcfee57cbc83b5a6c533c4
2016-11-10 07:44:28 +00:00
Henry Gessau
e8e4e5d7ab Deprecate run_tests.sh
We support tox for testing.
We don't maintain anything else.
In the Pike release we will remove the script.

Change-Id: Ic68858aaf1a519085dff394e6edc8c8fe6aeee6a
2016-11-08 03:50:20 +00:00
lilintan
26050562ca TrivialFix: Modify the spelling mistake
Change-Id: I07bd96de0dd1f89b66b3a4fd2bed0f222c237fb8
2016-10-27 11:52:17 +00:00
Ihar Hrachyshka
31e1aeb66b Forbid importing neutron.tests.* from outside tests subtree
neutron-sanity-check tool was importing neutron.tests.base module, which
may be not present on some systems (f.e. RDO splits neutron/tests/
subtree in a separate python-neutron-tests package). It made the tool
not usable in some setups.

https://bugzilla.redhat.com/show_bug.cgi?id=1374282

This is not the first time when we by mistake import from
neutron.tests.* and break distributions. It's time to stop it by
proactively forbidding that pattern via a new hacking check.

Some functions were moved from neutron.tests.base to
neutron.common.utils to fulfill the need requirement. They were moved
using debtcollector, no current consumers should be affected.

Closes-Bug: #1621782
Change-Id: I790777ddcbd1b02218b3db54ae3d5c931d72d4fa
2016-09-15 18:42:45 +00:00
Carl Baldwin
98838c491a Add information about using file based sqlite for unit tests
This is helpful when you want to inspect the database when sitting at
a break-point in a unit test.

Change-Id: Id4e25ef980437fe160822c3598135aa220127ac8
2016-07-25 23:54:38 +00:00
Itzik Brown
707363fea0 Adding basic connectivity scenario to Neutron
A Basic scenario:
- Creates an internal network and a subnet
- Creating a key pair
- Creating a router, setting the gateway and adding an
  internal interface
- Lauching an instance with a Nic connected to the internal network
- Adding rules to the tenant's default security group to allow SSH
  and ICMP
- Creating and associating a Floaing IP to the instance
- Checking SSH connectivity to the instance Floating IP address

Change-Id: Ica6fef4763b6f98c7795629b99ab392e6f7b6e59
Co-Authored-By: John Schwarz <jschwarz@redhat.com>
2016-06-02 13:39:25 -04:00
Assaf Muller
8adc737cd1 Delete post_test_hook.generate_test_logs
Log files as .txt files, don't zip them, and put them where
they need to be instead of copy them there in the post gate
hook. The benefit to doing this is that we'll get logs
for tests even if the job timed out.

Change-Id: I4bfd27534c827aed3cbd7b43d7d1289480ea4806
Related-Bug: #1567668
2016-05-04 11:11:46 -04:00
Armando Migliaccio
965e1d18a7 Move away from invoking api tests explicitly
Since adoption of the Tempest plugin framework, this
extra plumbing is no longer necessary in tree.

Related-bug: 1555356

Depends-on: If5f7ea82c45494953452f728fabb7403d0b6e14e
Change-Id: I5377f33c5a7b29f97f00878e048d154378a4fe34
2016-04-15 17:13:59 +00:00
Daniel Mellado
c56918027a Use tempest plugin interface
Make use of the Tempest plugin interface instead of copying Neutron
files into Tempest. This will remove the burden to port Neutron
tests onto Tempest master recurrently.

It uses neutron/tests/tempest/ as new top folder for all Tempest
tests. It follows the model of Heat [1].

[1]: https://github.com/openstack/heat/tree/master/heat_integrationtests

Partially implements bp external-plugin-interface

Change-Id: Ia233aa162746845f6ae08a8157dcd242dcd58eab
2016-04-14 15:19:14 +02:00
Jenkins
4f8050b172 Merge "Fix typos in Neutron documentation" 2016-04-02 19:11:00 +00:00
Jenkins
5bee125bff Merge "Add fullstack logging" 2016-03-30 14:38:03 +00:00
Nguyen Hung Phuong
0f15c461d7 Fix typos in Neutron documentation
This patch fixes typos in Neutron documentation.

Change-Id: I3a7d7e7c6c9623412781dab021e4de95a46b7851
2016-03-25 16:26:04 +00:00
Nate Johnston
c064258886 Note fullstack depends on released neutronclient
Note in TESTING.rst that fullstack tests that depend on changes to
python-neutronclient to execute need to wait until there is a released
version of python-neutronclient before they will work in the gate.

Change-Id: I1d32c65f0c941cf8f163a441258faa34b50ff88f
2016-03-24 11:10:19 -04:00
Assaf Muller
8603ea6340 Add fullstack logging
We currently don't log everything being output from the test
runner, only what the processes themselves are logging.

Change-Id: Id5fb9cd44a0ed677a03da1d153ee3079fd5b7975
2016-03-23 11:37:56 -04:00
Assaf Muller
ddd5df9526 Delete 118~ API tests from Neutron
Deleted tests that duplicate tests from Tempest, according to the
new criteria specified in TESTING.rst.

Follow up steps are detailed here:
https://etherpad.openstack.org/p/neutron-tempest-defork

For reviewers, here's how to get a complete list of network
tests from Tempest. From a Tempest directory, execute:
testr list-tests tempest.api.network | cut -d"[" -f1 | cut -d"." -f4-
I verified that every test removed here actually exists in that list.

Here's a list of patches that modified tests after the fork:
https://etherpad.openstack.org/p/neutron-tempest-defork-patches-since-initial-sync

And the list of tests we care about:
https://etherpad.openstack.org/p/neutron-tempest-defork, Ctrl-F for:
'Tests that should be synced from Neutron'

Related-bug: #1552960

Change-Id: I685291058f221a7ef0b5b7485280cdabceaa4af3
2016-03-10 15:34:08 -05:00
Sławek Kapłoński
2254acef06 Add fullstack resources for linuxbridge agent
This patch adds resources, fixtures and base test_connectivity tests
for host with linuxbridge agent.

Model of host with linuxbridge agent is made in separate namespace
named "host-XXX". It has got connectivity with rabbit via ovs port.
Same port is used also to provide connectivity with different "hosts"

Co-Authored-By: Mathieu Rohon <mathieu.rohon@gmail.com>

Change-Id: I6230d8d09f77bd20674bf6c3be69fbc1627d66f8
Closes-bug: #1518675
2016-02-19 13:12:25 +00:00
Jakub Libosvar
00f7313162 Compile OVS for functional tests
The patch takes OVS with conntrack support, compiles it and loads its
kernel modules. The patch steals parts from networking-ovn
project [1]. To run compilation of OVS, an environment variable
COMPILE_OVS must be set to True. Optional environment variables
OVS_REPO and OVS_BRANCH can be defined as a source of bits.

Once Neutron starts using neutron-lib, next steps will be taking
content of devstack/lib/ovs and putting it into neutron-lib project
in order to let other projects like networking-ovn to re-use the
code.

[1] https://github.com/openstack/networking-ovn/blob/master/devstack/plugin.sh#L175

Change-Id: Id2f51853a6339da72dbde7a82f94f99a4c306565
2016-02-04 11:06:22 +00:00
Manjeet Singh Bhatia
d32a405ed7 Fix typo in test path in Testing.rst
Change-Id: I8e9f250bb2f9d2781cfcaf7058fbc77ace42f063
2016-01-13 20:29:47 +00:00
Jenkins
68276dc961 Merge "Allow tox to pass more arguments to ostestr command" 2016-01-12 22:20:10 +00:00
gong yong sheng
240facf9e6 Allow tox to pass more arguments to ostestr command
Co-Authored-By: Kevin Benton <blak111@gmail.com>
Change-Id: Ie9969e0a5254ab5dd33b287f6d3120362b73edd7
Closes-bug: #1490389
2016-01-05 14:53:19 -08:00
Assaf Muller
cfe8d23c58 Add explanations and examples to TESTING.rst
* Added motivation paragraph
* Added definitions and an explanation about the scope of the
  test vs. the test infrastructure it uses
* Added content and examples to unit & functional tests sections

Change-Id: If78bd01d0fd7d899d4e6fbcdc8a33c441d2a1609
2015-12-09 16:12:43 -05:00
Assaf Muller
41d0498dc4 Moved fullstack test doc content to TESTING.rst
* Removed long term goals documentation (I don't see a need
  to document these).
* Added and rearranged short term goals.

Change-Id: If494533cb6507f18b84a41b3f1daf42cd10d9f51
2015-12-03 17:02:54 -05:00
armando-migliaccio
19f06c67e4 Add pointers to access Neutron test coverage details
Being aware of what the coverage looks like should help
writing more comprehensive tests.

Change-Id: Idd31edb2695d6b2b3475bfd3b2e9eafd6c12ba8e
2015-10-08 12:26:27 -07:00
Assaf Muller
11da3bbe8b Add testing coverage .rst, missing test infrastructure to-dos
Change-Id: I1f3bc57ebc196a1c50b731c68cecb5aad3cda21d
2015-08-10 20:37:23 -04:00
Assaf Muller
e152f93878 TESTING.rst love
Change-Id: I64b569048f8f87ea2fe63d861302b4020d36493d
2015-08-10 12:15:09 -04:00
Henry Gessau
fef79dc7b9 Consistent layout and headings for devref
The lack of convention for heading levels among the independently
written devref documents was starting to make the Table of Contents
look rather messy when rendered in HTML.

This patch does not cover the "Neutron Internals" section since its
layout is reasonably OK for now.

Change-Id: I827c105599f05773bda7e4fc0a941ce04ebd51fa
2015-08-09 21:46:57 -04:00
Assaf Muller
408af3f7da Prepare for full stack CI job
Related-Bug: #1467275

Change-Id: I90f4794f48ae151a888f37df26c087a7fdcd9d31
2015-06-22 11:47:54 -04:00
John Schwarz
2fcbbf3031 Move full-stack logs post-tests
Currently, it's up to the developer who wants to run full-stack on his
machine to make the directory in question (/opt/stack/logs). However,
this also means that the files don't get compressed at the end of a gate
run. Now, each full-stack test will have each own log directory in /tmp.
Once the logs are there, post_test_hook.sh can run 'gzip' on all the log
files before moving them to /opt/stack/logs on its own.

Change-Id: I5c04d0af0b9858722ae0c4baf0ee478ffb078e02
2015-05-28 12:58:40 +03:00
armando-migliaccio
cc904070cc Fix formatting errors in TESTING.rst
There were a few errors that went undetected.

TESTING.rst:266: SEVERE: Title level inconsistent:
TESTING.rst:67: ERROR: Unknown target name: "test".
TESTING.rst:74: ERROR: Unknown target name: "test".

Change-Id: Iad225e95c23b7460d228ba5447f4a361aa68d5dc
2015-04-14 23:09:36 +00:00
Gal Sagie
77df532e10 Additions to TESTING.rst
Small addition on how to run pep8 tests only for latest
patch set.

Change-Id: I07fa2c633d17acd1284ccd726a99a46414100ba3
2015-04-13 14:27:19 +03:00
Maru Newby
3e83a26e66 Enhance TESTING.rst
Add detail about api testing and provide better visual separation
between the different types of testing.

The current testing guidelines are mainly about running tests, and
this change does little to fix that.  The intention is to add detail
about writing tests in subsequent changes.

Change-Id: I39d0439c91e5c6edb1d48d4da310443c99fb6d9e
2015-04-07 16:17:21 +00:00
Maru Newby
1105782e39 Reorganize unit test tree
This change ensures that the structure of the unit test tree matches
that of the code tree to make it obvious where to find tests for a
given module.  A check is added to the pep8 job to protect against
regressions.

The plugin test paths are relocated to neutron/tests/unit/plugins
but are otherwise ignored for now.

Change-Id: If307593259139171be21a71c58e3a34bf148cc7f
Partial-Bug: #1440834
2015-04-06 23:28:31 +00:00
John Schwarz
e0ea5edc12 Add full-stack tests framework
This patch introduces the full-stack tests framework, as specified in
the blueprint. In short, this adds the neutron.tests.fullstack module,
which supports test-managed neutron daemons. Currently only
neutron-server is supported and follow-up patches will support for
multiple agents.

Implements: blueprint integration-tests
Co-Authored-By: Maru Newby <marun@redhat.com>
Change-Id: Iff24fc7cd428488e918c5f06bc7f923095760b07
2015-03-26 20:21:40 +02:00
Maru Newby
30c6f56365 Add script to copy neutron api tests from tempest
This change adds a script to automate the copying of api tests from
tempest in the style of oslo_incubator.  The target path will be
neutron/tests/tempest and no manual modifications should be made to
this path until such time as neutron api test development is frozen in
tempest and development can proceed in the neutron tree.  Until that
occurs, a policy of manual once-daily synchronization is suggested.

The target path includes the name 'tempest' as a clear indication that
this is not part of neutron, and that once development is allowed to
proceed, its contents should be rewritten and removed until there is
nothing left.

So long as the tests exist in both the tempest and neutron trees,
testing effort will be duplicated.  The larger goal is to have the
tests in question removed from tempest as per the qa guidelines [1],
so this should be temporary.

1: https://wiki.openstack.org/wiki/QA/Tempest-test-removal

Change-Id: I3cd55983e610a1d61aae565f88fe5017edba1090
2015-03-06 19:57:27 +00:00
Ihar Hrachyshka
754b99668f devref: consider sphinx warnings as failures
We previously introduced issues in our documentation due to not checking
for sphinx warnings [1], so let's enforce documentation validation.

This commit also fixes minor syntax issues caught by sphinx. Among
those, contribute.rst is now properly attached to ToC.

[1]: I48ab899077458d99eee6f643577d574106c1ca88

Change-Id: I9170883a20137c343f03d8b052399eacab4ae130
2015-02-23 18:48:18 +00:00
Maru Newby
540e4d791f Automate host configuration for functional testing
This change adds a new script, configure_for_func_testing.sh, that
automates configuration of a host to support functional testing.  The
script's functionality is consumed by a refactored version of
gate_hook.sh, and both minimizes runtime and removes the previous
dependency on the devstack-gate repo.

Additionally, the dsvm-functional tox env is no longer dependent on
devstack to deploy neutron's rootwrap configuration system-wide.
Rootwrap configuration is now deployed to the target tox venv on each
tox invocation.

Change-Id: Iaf43be458bdf3c4535f95ee5a3a3b47a744020a0
2015-02-14 05:13:03 +00:00
Maru Newby
930b4e3d7b Replace mention of nose with nose2 in devref
Documentation for the nose test runner was previously included in the
testing section of the developer reference.  Due to nose's lack of
support for the load_tests protocol - required to support
testscenarios-based generative testing - the documentation has been
updated to suggest the load_tests-supporting nose2 instead.

Change-Id: I5beb8619d5045372472021ed7eb19bddd0de5862
2015-01-06 00:46:47 +00:00
Jeremy Stanley
25ef9258a8 Workflow documentation is now in infra-manual
Replace URLs for workflow documentation to appropriate parts of the
OpenStack Project Infrastructure Manual.

Change-Id: I77aa4152a903a0f1f47de20bb1a006d29f5a1bf2
2014-12-05 03:30:37 +00:00
Miguel Angel Ajo
d37ff165f4 Support pudb as a different post mortem debugger
Post mortem debugger starts pdb at the assert failure place
when OS_POST_MORTEM_DEBUGGER is set and an exception or assert
failure happens.

This patch adds a neutron env variable OS_POST_MORTEM_DEBUGGER
to allow invocation of pudb instead of pdb. pudb module is not
a hard requisite as it will be only imported if you set
OS_POST_MORTEM_DEBUGGER=pudb.

The old OS_POST_MORTEM_DEBUG env variable is removed now.

Change-Id: I5d40913add439cf9c30305bafc98af9f8cd4d12f
2014-11-14 12:30:20 +01:00
Jenkins
4309020f8e Merge "Add a gate-specific tox env for functional tests" 2014-07-19 02:48:16 +00:00
Maru Newby
37086a8f17 Add a gate-specific tox env for functional tests
The functional gate jobs (*-neutron-dsvm-functional) need special
configuration (e.g. OS_SUDO_TESTING, OS_ROOTWRAP_CMD) to run tests.
This patch adds a new gate-specific tox env (tox -e dsvm-functional)
that sets the appropriate values and allows easy duplication of the
results of the upstream job on a host that has successfully run
devstack with neutron enabled.

A new configuration option, OS_FAIL_ON_MISSING_DEPS, is also set by
the dsvm-functional tox env to ensure that dependency failures will
result in failures rather than skips when run in the gate.

Change-Id: Idbfef1ba72ed129a3021509c7969e2685b8a6d09
Partial-bug: #1336172
2014-07-09 06:05:11 +00:00
Ralf Haferkamp
4675b462b6 Fix example for running individual tests
The ':' syntax doesn't seem to work anymore. Additionally fix the "tox" call to
not try to run it from the current working directory. It's usually installed
somewhere in $PATH.

Closes-Bug: #1334134
Change-Id: Ie608cee2ac131941d6642e2560a84db55415654b
2014-07-01 09:06:13 +02:00
Sean M. Collins
72398f1f69 Developer documentation
* Turns TESTING into a rst file, that we include in the developer
  documentation, for instructions on how to run the unit tests.
* Link to a Vagrant project that sets up Neutron inside a VM.
* Adds a section for how to debug with Nose
* Add new section for Neutron Internals
* Neutron L2 Agent documentation - currently only OVS
* Make the Security Group API extension an example of how an API
  extension is implemented

Implements bp developer-documentation

Change-Id: I9b452abc9da3b1a41ae65cff727967de0eab12fe
2014-02-26 11:03:46 -05:00