Commit Graph

251 Commits

Author SHA1 Message Date
Ihar Hrachyshka
a99897ffb3 Use rootwrap for fullstack test runner
We plan to switch to devstack-gate for fullstack job, and it revokes
direct sudo calls before executing tests, so we can't rely on sudo
working anymore.

This also moves functional-testing.filters to a more generic filename
(testing.filters) because the filters are now deployed and used by
fullstack target too.

Related-Bug: #1557168
Related-Bug: #1693689

Change-Id: I1718ea51836adbb8ef8dea79822a722dcf111127
2017-06-08 18:42:47 +00:00
Jenkins
0479f0f9d2 Merge "fullstack: Don't let dhcp agents failover" 2017-04-21 17:56:46 +00:00
Jakub Libosvar
bc979efdb8 fullstack: Don't let dhcp agents failover
It turned out dhcp tests work only because agents are considered dead
after 10 seconds while they report to server every 60 seconds. This led
to calling network resync after agent revival and hiding the fact dhcp
agent is not capable of receiving any amqp messages.

This patch sets the report interval of agents to the half of
agent_down_time on server side and uses eventlet dhcp agent in order to
trigger eventlet monkey patching code.

Eventlet was behind the failure with messages not getting processed. As
[1] notes: "Note: If the “eventlet” executor is used, the threading and
time library need to be monkeypatched."

Because each port calls dhclient to obtain IP address and each dhclient
instance overwrites /etc/resolv.conf there was added a script that
generates fullstack-dhclient-script from an existing dhclient-script
before starting fulltstack tests. This generated script is passed to
each dhclient process running in fake fullstack machine using -sf
parameter.

[1] https://docs.openstack.org/developer/oslo.messaging/server.html

Related-bug: 1453350

Change-Id: I0336176b9c364fe3a95be5cef9e7a3af1ef9d7e9
2017-04-20 14:04:50 +00:00
Jenkins
55be865d9a Merge "Add launchpad bug update support to abandon script" 2017-04-20 09:06:05 +00:00
Jenkins
e0eb0d6bfc Merge "Replaces yaml.load() with yaml.safe_load() in neutron" 2017-04-17 06:56:53 +00:00
Kevin Benton
514580692c Add launchpad bug update support to abandon script
Change-Id: If6db7c8f39665dfdef9b3936fd75e33bd9b76e03
2017-04-16 02:44:03 -07:00
gengchc2
edb6766294 Replaces yaml.load() with yaml.safe_load() in neutron
Yaml.load() return Python object may be dangerous if you receive
a YAML document from an untrusted source such as the Internet.
The function yaml.safe_load() limits this ability to simple Python
objects like integers or lists.

Reference:
https://security.openstack.org/guidelines/dg_avoid-dangerous-input-parsing-libraries.html

Change-Id: I026355f3e71b7fd3aeee2fe5c7920a1c6306ab02
2017-03-10 01:09:16 +00:00
Brian Haley
a0552e75e3 Fix bashate warnings
Three ((value++)) lines are generating this bashate warning:
E043: Arithmetic compound has inconsistent return semantics

This will now be an error when running pep8 checks.

Trivialfix

Change-Id: If7a63e550d5d27fe6716cc63c85426cd2250732f
2017-03-07 13:32:04 -05:00
Victor Morales
a14439549f Add bashate support
Bashate is a style checker program for bash scripts.  This addition
improves the quality of the current bash scripts and ensures that
any future change will follow the same standards.

Change-Id: Ia346f77632d4ac7beb288fa3aacea221d7969c87
2017-02-25 06:29:55 +00:00
Jenkins
f760bd792f Merge "configure_for_func_testing.sh: Remove neutron-legacy inclusion" 2017-02-03 21:02:24 +00:00
Jakub Libosvar
d5c07fe512 ovsfw: Support protocol numbers instead of just tcp and udp
Neutron API accepts also protocol numbers as protocols for security
groups. This patch makes support for it in OVS firewall driver. iptables
driver already supports it.

Fullstack test covering SCTP connection was added and it requires
ip_conntrack_proto_sctp kernel module in order to make conntrack work
with SCTP.

Change-Id: I6c5665a994c4a50ddbb95cd1360be0de0a6c7e40
Closes-bug: 1625516
2017-01-20 11:17:40 -05:00
Jenkins
edd0c130d9 Merge "Replace nc command with ncat" 2017-01-07 03:15:48 +00:00
Sławek Kapłoński
a12de8975e Fullstack tests for DHCP agent HA
This patch adds fullstack tests for DHCP agent HA functionallity.

There are two tests added:
* One is checking if network will be properly
  rescheduled to new agent if existing DHCP agent will be down,
* Second is checking if network handled by 2 DHCP agents will have properly
  working DHCP service even if one DHCP agent will be down

Change-Id: Iaad373cafd6f83f2c1f8e7ac58dc70070e7aabaf
2017-01-04 22:48:46 +00:00
Jakub Libosvar
8f37f2d34e Replace nc command with ncat
Ncat has better capabilities like SCTP which nc doesn't have.

Change-Id: I7a3a955465aa05032ead63459191167cfe3f98bd
2017-01-03 16:09:14 +00:00
YAMAMOTO Takashi
74e7145b60 configure_for_func_testing.sh: Remove neutron-legacy inclusion
After devstack change (see Depends-On below),
it isn't necessary anymore.

Depends-On: I3ff136fc8330c92007cdfe91b77d7f9865eabd8d
Change-Id: Iec6f1fb59bf1f656100947f50311afcee4636ca2
2016-12-01 22:40:02 +09:00
YAMAMOTO Takashi
c15c4e4ede configure_for_func_testing.sh: Source lib/neutron as well
Also, set ENABLED_SERIVICES before sourcing devstack files
because they uses it.  This is a preparation for an upcoming
change in devstack. [1]

[1] I3ff136fc8330c92007cdfe91b77d7f9865eabd8d

Change-Id: Ida1f83b6b3ef9b76be13c063c7e35a8703214078
2016-12-01 20:40:19 +09:00
Jenkins
ac834fd040 Merge "Remove duplicated tool" 2016-11-11 22:26:52 +00:00
Henry Gessau
b0e56ac646 Retire deprecations for constants, exceptions, attributes
Subprojects have had enough time to switch to neutron-lib for
the deprecated items from these modules.

Change-Id: Ib2f4acf7e0153667389c57a52f2db33a23758b24
2016-11-09 23:43:38 -05:00
Ihar Hrachyshka
80eb375ba1 Fixed functional iptables firewall tests for newer kernels
Iptables functional tests fail on Xenial and other newer kernels if
br_netfilter kernel module is not loaded, in which case sysctl knobs to
enable bridge firewalling are not available, and attempt to set them
with _enable_netfilter_for_bridges fails.

We should load the kernel module before running those tests. Luckily,
devstack has a function for just that (plus more).

Change-Id: I602d8cd02c73b18e9d719b19998e36059ae28cd8
Depends-On: Id6bfd9595f0772a63d1096ef83ebbb6cd630fafd
Related-Bug: #1622914
2016-09-29 13:39:18 +00:00
Armando Migliaccio
9339117c24 Remove duplicated tool
This is hosted in reviewday

Change-Id: I485440ea448c092f020fc12900d558e9513cee19
Depends-on: I2d9034a31a2492a5723db6d8e10e495b20696db6
2016-09-06 13:13:19 -07:00
Jenkins
5f849165a6 Merge "Make milestone-review-dash script pull all Stadium projects" 2016-09-01 11:37:44 +00:00
Armando Migliaccio
40f5846280 Make milestone-review-dash script pull all Stadium projects
Change-Id: I2ba04d777001ebac12e5cbeaf5d530c932cc99f7
2016-08-31 18:08:49 -07:00
Henry Gessau
99cfd671be debtcollector for globals
The deprecation shim created by Doug turns out to be rather useful.
It emits a warning when an global (attribute of a module) is
referenced but that global has been moved to another module.

This update makes the following changes to the shim:

 - Rename it to _MovedGlobals to better describe what it is a
   debtcollector for.

 - Use inspect to get the original reference and to check that
   _MovedGlobals is called from the last line of a module.

 - Save the old reference automatically in the instance to prevent it
   from getting garbage collected.

 - Beef up the _moved_global() method for moving/renaming individual
   globals, allowing it to move and rename or rename in place.

Change-Id: I868aa4a3129dd05467a103364088efbb86bc5d0f
2016-08-30 10:18:19 -04:00
Henry Gessau
961c5e73e9 Add tool to list moved globals
This tools list the constants and exceptions that have moved
to neutron-lib. Add it to pep8 to raise awareness of moved
globals.

Related Blueprint: neutron-lib

Change-Id: Ic6439f98d4bf8f566debf26eca3337f359119d5a
2016-08-18 10:14:42 -04:00
Jenkins
05f722c0c2 Merge "Pull stadium projects from governance.o.o in utility script" 2016-07-09 00:41:22 +00:00
Sindhu Devale
0a7895996f Grammar error fixed
Change-Id: I1b90140dd150a2d23d30d39623047868d55b204b
2016-07-06 18:48:59 +00:00
Armando Migliaccio
acb6a950ed Pull stadium projects from governance.o.o in utility script
For housekeeping reasons we have abandon_old_reviews.sh that is
used to purge blocked reviews from Gerrit. The project query
should be extended to all stadium projects. To avoid hard coding,
fetch the list from governance.o.o, parse it and feed it to
the rest of the script.

Change-Id: Iad40d78b7a72a4435382406373bb9f15c0ffc4a7
2016-07-05 20:58:41 -07:00
Miguel Angel Ajo
40e10a3150 Add sanity check for idempotent id's in tempest tests
Idempotent id's for tempest tests must be unique, as it's used to trace
tests even in the event of refactors [1].

This patch adds a new check using check-uuid tool from tempest to
validate id uniqueness. This adds a new test-only dependency to pull in
tempest from PyPI to make pep8 tox target work.

Also, the duplicate idempotent id's in our tests have been fixed.

[1]
http://docs.openstack.org/developer/tempest/HACKING.html#test-identification-with-idempotent-id

Co-Authored-By: Ihar Hrachyshka <ihrachys@redhat.com>
Change-Id: Ie92e4576d9ce6c26f19bdd668426c71a62409e7a
2016-06-08 23:51:15 +02:00
Kevin Benton
b53a63c9e5 Remove tools/pecan_server.sh
This isn't used anymore and may not even work.
The server runs as part of the previous neutron-server binary
now with an option to control whether pecan or the legacy
code is used.

Change-Id: Ic5236ec305ca21a5e06b3149c7270eb2d62d2606
Closes-Bug: #1579924
2016-05-06 18:06:25 -07:00
Jenkins
0c88a3085c Merge "milestone-review-dash detect current milestone" 2016-04-12 05:06:32 +00:00
Ihar Hrachyshka
c25ece63bd Cleaned up tox_install.sh
Now that we don't need the script to pull tempest from git, it became
redundant. It's better to clean it up.

Change-Id: I8881840813339c20e4d5c59b98c2eff63f4feffb
2016-04-04 11:41:05 +02:00
Armando Migliaccio
8a6913c534 Fix zuul_cloner errors during tox job setup
Since [1], Tempest is a pip installable package, and that prevents zuul_cloner
to work correctly. This change moves away from the existing logic in tox_install,
and adds tempest as an explicit requirement for the api job. This is in fact
the only tox target that needs Tempest to work.

tox_install.sh has become less important now, but cleanup is left as follow
up, to speed up gate salvation.

[1] I25eac915c977ebaedced66ac896c5dd77259d193

Depends-on: I1d1ca087982d7bdda983686775736ed258026aeb
Change-Id: I00d882dde77a687ecb57ec200a34fd96256ad87a
2016-04-02 02:02:52 +00:00
rossella
79ab61d56c milestone-review-dash detect current milestone
This patch makes the milestone arg optional. If it's not passed
the script will detect and use the current milestone.
This is required to run this script in a periodic job, since infra
won't accept hard-code parameters.

Change-Id: I8c2ffbc68fd36759924e9ab2a1eb64be390d4756
2016-03-22 16:09:32 +00:00
Michael Johnson
44ef44c0ff Update devstack plugin for dependent packages
Recent changes to the gate base images [1] removed a package
neutron requires (netcat-openbsd). This patch installs the
required package.

[1] https://review.openstack.org/#/c/292573

Closes-bug: #1558397

Change-Id: I4041478ca09bd124827782774b8520908ef07be0
2016-03-16 23:45:02 -07:00
Jenkins
205e86a7f2 Merge "Remove unused -constraints tox targets" 2016-03-07 23:12:31 +00:00
Ihar Hrachyshka
317f5b2dfc Remove unused -constraints tox targets
Now that gate does not rely on -constraints targets [1], we should be
safe to remove them.

[1] Iea3a0b1f740d5679ebfa135f2bf5b89c52008716

Change-Id: I59855f1b15d54323dcc7c4759904be7124a1df1a
2016-03-07 17:21:05 +01:00
Ihar Hrachyshka
de182d672e constraints: fixed typo in tox_install.sh
We should not use -ne with strings; instead, != should be used.

This bug in the script effectively left all tox targets unconstrained.

Change-Id: I867a3c7a2049e1d8e5cde76d8eb60d15880c430d
2016-03-07 14:21:06 +01:00
Ihar Hrachyshka
e297d7d7db Make all tox targets constrained
Since neutron gate already switched to -constraints target, we need to
keep those targets until we switch gate back to using targets without
the postfix.

This change effectively makes both types of targets identical.

Note: override install_command for venv, releasenotes, and cover targets
since corresponding gate jobs do not provide constraints file yet.

Also removed cover-constraints target since it would never work in gate
anyway.

Co-Authored-By: Ihar Hrachyshka <ihrachys@redhat.com>
Co-Authored-By: Andreas Jaeger <aj@suse.com>
Change-Id: Ica2d509df0c54496aba7cb3162565384cc5d34c8
2016-03-04 20:11:56 +01:00
Jenkins
3b6832032a Merge "Use tools_path/venv environment variables in install_venv" 2016-02-29 20:00:41 +00:00
Sławek Kapłoński
2254acef06 Add fullstack resources for linuxbridge agent
This patch adds resources, fixtures and base test_connectivity tests
for host with linuxbridge agent.

Model of host with linuxbridge agent is made in separate namespace
named "host-XXX". It has got connectivity with rabbit via ovs port.
Same port is used also to provide connectivity with different "hosts"

Co-Authored-By: Mathieu Rohon <mathieu.rohon@gmail.com>

Change-Id: I6230d8d09f77bd20674bf6c3be69fbc1627d66f8
Closes-bug: #1518675
2016-02-19 13:12:25 +00:00
Qiaowei Ren
3ad46e1c40 Add necessary executable permission
Some files should be marked as executable, and this patch
just 'chmod +x' to them.

Change-Id: If2b7a9c703ae2fcfdcffedac0528437281bb23e5
2016-02-15 15:11:57 +08:00
Doug Wiegley
828068181c Get tempest via zuul-cloner if needed and it is available
Change-Id: I70f402202f99a989ef5e95213669f4ca9f7b706f
2016-02-03 23:46:02 -07:00
Assaf Muller
a2fe5fd2e0 Delete Tempest fork, import from tempest and tempest_lib
A while ago we copied Tempest networking API tests in to the
Neutron repo, and along came thousands of lines of code of Tempest
testing infrastructure (neutron.tests.tempest). For a while we
periodically refreshed our fork via:
tools/copy_api_tests_from_tempest.sh

I think it's time we move away from that model by eliminating
the fork. We do this by deleting unused code and importing the
rest from tempest_lib. There's some Tempest code still not
moved from Tempest to tempest_lib in tempest.common. I think
it's preferable to import that code than to copy it, and Tempest
cores mostly agree. Manila and Ironic also do the same.
To be able to import from tempest I added it as a requirement:
Since Tempest is not on PyPi, I had to get it from git. Only the api
tests environment needs Tempest, so instead of adding it to
test-requirements, I added it specifically to the api and
api-constraints venvs.

neutron.tests.tempest.test and neutron.tests.tempest.common.*
still remain. These are tighly coupled with one another, and
sadly since Neutron forked Tempest code, Tempest has made significant
changes to those files that also require changes to the test files.
I aim to get rid of the Neutron fork of these files in a follow up
change.

Also fixed import grouping in test files so that it's std libs,
3rd party libs, and then Neutron code.

* Removed neutron.tests.tempest.config:
  - We only added one option after the fork. I created a new group
    called 'neutron_plugin_options' and moved the new option to that
    group. This is in preperation for the Tempest plugin architecture,
    where you're supposed to add new config options to a new group
    and not to existing configuration groups. Note that this is
    obviously a backward incompatible change, but it's to an option
    added in the same cycle.
* Removed neutron.tests.tempest.test and neutron.tests.tempest.common.
  - This introduced an API change to the way we access Keystone,
    which required mechanical changes to a few tests (create_tenant
    calls need a different client now).
  - The way Tempest manages primary, admin and alternative tenant
    credentials was changed after we forked, which required another
    mechanical change to a few tests.
* Cut all of the Keystone clients we don't need. We only need
  to create/delete tenants, the other clients were used in Tempest by
  actual Keystone tests.
* Changed neutron.tests.api.base.BaseNetworkTest:
  - Re-implemented get_client_manager so that it returns the Neutron
    clients manager and not the one in the Tempest repo.
  - Updated it from the Tempest repo so that it uses the new way
    to manage credentials (Since it now uses the Tempest test base
    class and not our out of date forked copy).

Change-Id: I4f9193dfe26f2d36985cb480a98709ec182a2f7b
2016-01-29 11:31:27 -05:00
rossella
f585662a8d Neutron review tool use message instead of topics for bugs
After the upgrade of Gerrit the message field works as expected
so the review tool can use it instead of filtering using the
topic. This implies that having the bug number in the commit
message is enough for the patch to be included in the dashboard.

Change-Id: I6e61643fe862f842d731abaacfd1b18881e0c45e
2016-01-26 11:52:24 +00:00
Henry Gessau
e29975b799 Remove obsolete plugin stuff
Some spring cleaning:
 - Remove obsolete cisco vpn configs
 - Remove obsolete references from UT check script
 - Remove obsolete script for cleaning up plugins
 - Remove obsolete mock of heleosapi module

Change-Id: I0051e0b75b17279ffb61de328d7ccc7a12dc99dc
2016-01-21 23:16:20 +00:00
Einst Crazy
858389db7c Use tools_path/venv environment variables in install_venv
In the run_tests.sh, it export tools_path/venv/venv_path.
In the install_venv.sh, to get venv value in environ values. In
the with_venv.sh, to get correct envirion values.

Change-Id: Ia041937eaa8cf2194a929c846a7a96312a641ac5
Partial-Bug: #1535514
2016-01-20 02:07:12 +00:00
Doug Wiegley
875175ae9c Add --dry-run mode to code review abandon tool
Usually I edit the script every time, to see what it would do
before the actual run. Add that mode via a flag.

Change-Id: I3ea50b905217947a43e4437b4925bb79c45a3d18
2016-01-13 16:07:12 -06:00
Jenkins
68276dc961 Merge "Allow tox to pass more arguments to ostestr command" 2016-01-12 22:20:10 +00:00
Martin Hickey
074268f74c Add test for Neutron object versions
Adds a test to Neutron to check object versions for any changes
to objects. It prompts the developer to update the version of the
changed object. It uses oslo.versionedobjects.

Change-Id: I99454b28ae0b5fa663354eeccdf709d4030a280b
Co-Authored-By: Ryan Rossiter <rlrossit@us.ibm.com>
2016-01-06 10:34:03 +00:00
gong yong sheng
240facf9e6 Allow tox to pass more arguments to ostestr command
Co-Authored-By: Kevin Benton <blak111@gmail.com>
Change-Id: Ie9969e0a5254ab5dd33b287f6d3120362b73edd7
Closes-bug: #1490389
2016-01-05 14:53:19 -08:00