29 Commits

Author SHA1 Message Date
Matt Riedemann
85a1dd338b Convert legacy nova-live-migration and nova-multinode-grenade to py3
This makes these legacy devstack-gate-based jobs run with python3.

Change-Id: Id565a20ba3ebe2ea1a72b879bd2762ba3e655658
2019-11-14 16:06:02 +00:00
Matt Riedemann
14ca6f62e3 Remove the TODO about using OSC for BFV in test_evacuate.sh
With OSC 4.0.0 we could now use the --boot-from-volume option
to create a volume-backed server from the provided image. However,
that option leaves the created root volume around since
delete_on_termination defaults to false in the API. So while we
could use that option and convert from nova boot to openstack
server create, it would mean we'd have to find and manually delete
the created volume after the server is created, which is more work
than it's worth to implement the TODO so just remove it.

Change-Id: I0b70b19d74007041fc2da55a4edb1c636af691d6
2019-11-07 14:33:32 -05:00
melanie witt
6ea945e3b1 Remove redundant call to get/create default security group
In the instance_create DB API method, it ensures the (legacy) default
security group gets created for the specified project_id if it does
not already exist. If the security group does not exist, it is created
in a separate transaction.

Later in the instance_create method, it reads the default security group
back that it wrote earlier (via the same ensure default security group
code). But since it was written in a separate transaction, the current
transaction will not be able to see it and will get back 0 rows. So, it
creates a duplicate default security group record if project_id=NULL
(which it will be, if running nova-manage db online_data_migrations,
which uses an anonymous RequestContext with project_id=NULL). This
succeeds despite the unique constraint on project_id because in MySQL,
unique constraints are only enforced on non-NULL values [1].

To avoid creation of a duplicate default security group for
project_id=NULL, we can use the default security group object that was
returned from the first security_group_ensure_default call earlier in
instance_create method and remove the second, redundant call.

This also breaks out the security groups setup code from a nested
method as it was causing confusion during code review and is not being
used for any particular purpose. Inspection of the original commit
where it was added in 2012 [2] did not contain any comments about the
nested method and it appeared to either be a way to organize the code
or a way to reuse the 'models' module name as a local variable name.

Closes-Bug: #1824435

[1] https://dev.mysql.com/doc/refman/8.0/en/create-index.html#create-index-unique
[2] https://review.opendev.org/#/c/8973/2/nova/db/sqlalchemy/api.py@1339

Change-Id: Idb205ab5b16bbf96965418cd544016fa9cc92de9
2019-10-14 18:54:43 +00:00
melanie witt
7c41365f19 Add regression test for bug 1824435
This adds a regression test in our post test hook. We are not able to
do a similar test in the unit or functional tests because SQLite does
not provide any isolation between transactions on the same database
connection [1] and the bug can only be reproduced with the isolation
that is present when using a real MySQL database.

Related-Bug: #1824435

[1] https://www.sqlite.org/isolation.html

Change-Id: I204361d6ff7c2323bc744878d8a9fa2d20a480b1
2019-10-14 06:03:44 +00:00
Zuul
6ee5cfb397 Merge "Test heal port allocations in nova-next" 2019-10-03 03:28:30 +00:00
Balazs Gibizer
0044702e0d Test heal port allocations in nova-next
This patch extends the existing integration test for
heal_allocations to test the recently implemented port
allocation healing functionality.

Change-Id: I993c9661c37da012cc975ee8c04daa0eb9216744
Related-Bug: #1819923
2019-10-02 11:15:36 +02:00
Balazs Gibizer
2cf9a5f9fa Add cold migrate and resize to nova-grenade-multinode
Changes in [1] could potentially break a mixed-compute-version
environment as we don't have grenade coverage for cold migrate and
resize. This adds that coverage to the nova-grenade-multinode
job.

[1]https://review.opendev.org/#/c/655721/10

Change-Id: I81372d610ddf8abb473621deb6e7cb68eb000fee
2019-08-30 15:35:46 -04:00
Balazs Gibizer
3c1d9dab85 Move live_migration test hooks under gate/
This patch resolves a TODO in the .zuul.yaml about using common
irrelevant files in our dsvm jobs. To be able to do that we need to move
the test hooks from nova/tests/live_migraton under gate/.

Change-Id: I4e5352fd1a99ff2b4134a734eac6626be772caf1
2019-08-29 14:45:48 -04:00
melanie witt
f5c2430876 Remove unused args from archive_deleted_rows calls
As of commit 1c9de9c7779b1faf9d9542b3e5bd20da70067365, we no longer
pass any args to the archive_deleted_rows function, so we can remove
the argument list from the function.

Change-Id: I73b2f716908088b137102631f9360939a1d7341a
2019-08-28 05:14:18 +00:00
melanie witt
1c9de9c777 Verify archive_deleted_rows --all-cells in post test hook
We are already running archive_deleted_rows in the gate, but we are
not verifying whether all instance records, for example, were actually
successfully removed from the databases (cell0 and cell1).

This adds the --all-cells option to our archive_deleted_rows runs and
verifies that instance records were successfully removed from all cell
databases.

It is not sufficient to check only for return code 0 because
archive_deleted_rows will still return 0 when it misses archiving
records in cell databases.

Related-Bug: #1719487

Change-Id: If133b12bf02d708c099504a88b474dce0bdb0f00
2019-08-27 06:16:24 +00:00
melanie witt
f32671359e Make a failure to purge_db fail in post_test_hook.sh
Currently, the 'purge_db' call occurs before 'set -e', so if and when
the database purge fails (return non-zero) it does not cause the script
to exit with a failure.

This moves the call after 'set -e' to make the script exit with a
failure if the database purge step fails.

Closes-Bug: #1840967

Change-Id: I6ae27c4e11acafdc0bba8813f47059d084758b4e
2019-08-21 19:21:55 +00:00
Matt Riedemann
cee072b962 Convert nova-next to a zuul v3 job
For the most part this should be a pretty straight-forward
port of the run.yaml. The most complicated thing is executing
the post_test_hook.sh script. For that, a new post-run playbook
and role are added.

The relative path to devstack scripts in post_test_hook.sh itself
had to drop the 'new' directory since we are no longer executing
the script through devstack-gate anymore the 'new' path does not
exist.

Change-Id: Ie3dc90862c895a8bd9bff4511a16254945f45478
2019-07-23 11:32:35 -04:00
Matt Riedemann
87365c760e Add integration testing for heal_allocations
This adds a simple scenario for the heal_allocations CLI
to the post_test_hook script run at the end of the nova-next
job. The functional testing in-tree is pretty extensive but
it's always good to have real integration testing.

Change-Id: If86e4796a9db3020d4fdb751e8bc771c6f98aa47
Related-Bug: #1819923
2019-06-29 11:03:55 +00:00
Dan Smith
0685139ed8 Make nova-next archive using --before
Change-Id: I4fbd0cb73c73ab680af3f341d6069addb57393fb
2019-06-05 07:42:23 -07:00
Matt Riedemann
bed9d49163 Pass --nic when creating servers in evacuate integration test script
Devstack change Ib2e7096175c991acf35de04e840ac188752d3c17 started
creating a second network which is shared when tempest is enabled.
This causes the "openstack server create" and "nova boot" commands
in test_evacuate.sh to fail with:

  Multiple possible networks found, use a Network ID to be more specific.

This change selects the non-shared network and uses it to create
the servers during evacuate testing.

Change-Id: I2085a306e4d6565df4a641efabd009a3bc182e87
Closes-Bug: #1822605
2019-04-01 09:58:01 -04:00
Sean Mooney
30550d3d94 update gate test for removal of force evacuate
micro-version 2.68 removed force evacuation, this chage
updates gate/test_evacuate.sh to use micro-version 2.67

Closes-Bug: #1819166

Change-Id: I44a3514b4b0ba1648aa96f92e896729c823b151c
2019-03-08 16:31:31 +00:00
Zuul
99e7df0795 Merge "Remove placement perf check" 2018-12-08 04:56:56 +00:00
Matt Riedemann
3b1463b968 Use tempest [compute]/build_timeout in evacuate tests
Waiting 30 seconds for an evacuate to complete is not enough
time on some slower CI test nodes. This change uses the
same build timeout configuration from tempest to determine
the overall evacuate timeout in our evacuate tests.

Change-Id: Ie5935ae54d2cbf1a4272e93815ee5f67d3ffe2eb
Closes-Bug: #1806925
2018-12-05 10:46:06 -05:00
Chris Dent
84182d0aa2 Remove placement perf check
gate/post_test_perf_check.sh did some simplistic performance testing of
placement. With the extraction of placement we want it to happen during
openstack/placement CI changes so we remove it here.

The depends-on is to the placement change that turns it on there, using
an independent (and very small) job.

Depends-On: I93875e3ce1f77fdb237e339b7b3e38abe3dad8f7
Change-Id: I30a7bc9a0148fd3ed15ddd997d8dab11e4fb1fe1
2018-11-30 15:12:48 +00:00
Matt Riedemann
2023f46015 Add volume-backed evacuate test
This adds a volume-backed instance evacuate scenario
to the test_evacuate post-test script.

Change-Id: I37120d9ce02de6dadbd279de195d2f289c891123
2018-10-25 16:15:56 -04:00
Matt Riedemann
8327011f91 Add post-test hook for testing evacuate
This adds a post-test bash script to test evacuate
in a multinode job.

This performs two tests:

1. A negative test where we inject a fault by stopping
   libvirt prior to the evacuation and wait for the
   server to go to ERROR status.

2. A positive where we restart libvirt, wait for the
   compute service to be enabled and then evacuate
   the server and wait for it to be ACTIVE.

For now we hack this into the nova-live-migration
job, but it should probably live in a different job
long-term.

Change-Id: I9b7c9ad6b0ab167ba4583681efbbce4b18941178
2018-10-25 16:15:56 -04:00
Chris Dent
28937be947 Add trait query to placement perf check
This updates the EXPLANATION and sets the pinned version placeload
to the just release 0.3.0. This ought to hold us for a while. If
we need to do this again, we should probably switch to using
requirements files in some fashion, but I'm hoping we can avoid
that until later, potentially even after placement extraction
when we will have to moving and changing this anyway.

Change-Id: Ia3383c5dbbf8445254df774dc6ad23f2b9a3721e
2018-08-16 18:32:12 +01:00
Chris Dent
e6754e1b9e Add explanatory prefix to post_test_perf output
The pirate on crack output of placeload can be confusing
so this change adds a prefix to the placement-perf.txt log
file so that it is somewhat more self-explanatory.

This change also pins the version of placeload because the
explanation is version dependent.

Change-Id: I055adb5f6004c93109b17db8313a7fef85538217
2018-08-16 18:21:47 +01:00
Chris Dent
8b4fcdfdc6 Add placement perf info gathering hook to end of nova-next
This change adds a post test hook to the nova-next job to report
timing of a query to GET /allocation_candidates when there are 1000
resource providers with the same inventory.

A summary of the work ends up in logs/placement-perf.txt

Change-Id: Idc446347cd8773f579b23c96235348d8e10ea3f6
2018-08-14 15:42:08 +01:00
Dan Smith
fd59fbd4d1 Make nova-manage db purge take --all-cells
This makes purge iterate over all cells if requested. This also makes our
post_test_hook.sh use the --all-cells variant with just the base config
file.

Related to blueprint purge-db

Change-Id: I7eb5ed05224838cdba18e96724162cc930f4422e
2018-03-08 09:26:49 -08:00
Dan Smith
ae241cc68f Add simple db purge command
This adds a simple purge command to nova-manage. It either deletes all
shadow archived data, or data older than a date if provided.

This also adds a post-test hook to run purge after archive to validate
that it at least works on data generated by a gate run.

Related to blueprint purge-db

Change-Id: I6f87cf03d49be6bfad2c5e6f0c8accf0fab4e6ee
2018-03-07 10:35:32 -08:00
Dan Smith
64635ba18d Run post-test archive against cell1
Change-Id: I4af326fe66f0cf24ede8a8b7a8ce0e528c4f437c
2018-03-07 10:35:32 -08:00
Matt Riedemann
e8e8941d25 Check for leaked server resource allocations in post_test_hook
The post_test_hook.sh runs in the nova-next CI job. The 1.0.0
version of the osc-placement plugin adds the CLIs to show consumer
resource allocations.

This adds some sanity check code to the post_test_hook.sh script
to look for any resource provider (compute nodes) that have allocations
against them, which shouldn't be the case for successful test runs
where servers are cleaned up properly.

Change-Id: I9801ad04eedf2fede24f3eb104715dcc8e20063d
2018-02-24 02:27:38 +00:00
Sean Dague
95441ef896 move gate hooks to gate/
We prevent a lot of tests from getting run on tools/ changes given
that most of that is unrelated to running any tests. By having the
gate hooks in that directory it made for somewhat odd separation of
what is test sensitive and what is not.

This moves things to the gate/ top level directory, and puts a symlink
in place to handle project-config compatibility until that can be
updated.

Change-Id: Iec9e89f0380256c1ae8df2d19c547d67bbdebd65
2017-01-04 11:05:16 +00:00