617 Commits

Author SHA1 Message Date
bhagyashris
046f34290d Fix invalid import order
Made corrections in import order as per OpenStack import standards [1].

[1] http://docs.openstack.org/developer/hacking/#import-order-template

TrivialFix

Change-Id: Ieb63c7fc5becc68db8d68b3e6847321e77ceeed3
2016-07-04 16:24:10 +05:30
Jenkins
d68acca919 Merge "Add ability to select specific tests for py34" 2016-06-29 16:31:23 +00:00
Jens Rosenboom
0196f4676e Add ability to select specific tests for py34
For py27 you can simply run specific tests by calling

    tox -e py27 nova.tests.unit.test_something

Add a similiar option for the py34 environment. This will skip the
blacklist of tests that would otherwise be ignored for python3.

Change-Id: I9bf5901afed4abf592e1aaff2f4820b9e3abc671
2016-06-28 05:24:48 +02:00
liangjingtao
a0f5a56a15 Make print py3 compatible
In PY3,remove the print "", join the print () function to achieve
the same function,this patch is to fix it.

Change-Id: Ie860e912b0c7101fd9d1d5c8373db8688f99122c
Closes-Bug: #1595773
2016-06-25 08:19:27 +00:00
Sean Dague
cdf74c57a6 remove db2 support from tree
This removes db2 support from tree completely. This is an oddball non
open database that made doing live data migrations difficult. It is
used by 0% of users in the OpenStack User Survey.

Supporting commercial software that doesn't have users at the cost of
delivering features and fixes to our community is the wrong
tradeoff. This corrects that.

It rewrites migrations, which we typically don't ever do, but it is
better if newton fresh environments fail early in db creation instead
of very deep in the data migration process.

Change-Id: Ifeb9929e4515e3483eb65d371126afd7672b92a4
2016-05-23 07:33:20 -04:00
Sujitha
8fc7d30600 Config options: Centralize compute options
This change moves the config options in nova/compute/
manager.py to a central location "nova/conf/compute.py".

Change-Id: Iac75f966571d1224fcd9487b110cb5a4eaa29219
Implements: blueprint centralize-config-options-newton
2016-04-20 19:00:25 +00:00
Anusha Unnam
424ab495e7 Config options: Centralize xenapi driver options
The config options of
"nova/virt/xenapi/driver.py" got moved
to the new central location
"nova/conf/xenserver.py”

Change-Id: I87f6163c7a3b69ef8ede947132b771f32742781e
Implements: blueprint centralize-config-options-newton
2016-04-05 16:34:02 +00:00
Jenkins
91010e1b50 Merge "Add a tool for reserving migration placeholders during release time" 2016-02-10 19:48:44 +00:00
Dan Smith
526ec7ca65 Add a tool for reserving migration placeholders during release time
This adds a tool to make it easy to reserve placeholder migrations
after we do a release. This is something we have to do each time we
do a release, and it can be somewhat silly in terms of mechanical
work. This adds a script to make it trivially easy.

Change-Id: I34c11f1aee64ad36887779f1c9ca4491a25580ec
2016-02-09 10:54:08 -08:00
Jenkins
9ee136a98c Merge "ebtables/libvirt workaround" 2016-02-08 13:10:20 +00:00
Chet Burgess
17264ee6a6 ebtables/libvirt workaround
Idealy nova is run with libvirt 1.2.11 or later to guarantee that
libvirt is calling ebtables with --concurrent. Since we can't
always guarantee this we have created this workaround.

The workaround is extremely hacky and not recommend but for those
who simply have no other way to address this bug the following
should be done.

 * Copy /sbin/ebtables to /sbin/ebtables.real
 * Copy the ebtables.workaround script to /sbin/ebtables

Caution: Future OS level updates and packages way overwrite the
above changes. Its recommend users upgrade to libvirt 1.2.11.

The work around script was copied from devstack and originally
written by sdague.

Change-Id: Icdffc59d68b73a6df22ce138558d6e23e1c96336
Closes-Bug: #1501366
2016-02-04 09:38:34 -05:00
Bob Ball
a40ee2ce1b XenAPI: Fix race on rotate_xen_guest_logs
There is a race between where Xen creates the domain and starts logging
and when XAPI returns to say that the domain has started.

If the cronjob to rotate the guest logs runs in this period, it can
delete the console logs before we set the last_dom_id which is used to
preserve logs.

This change adds a fixed delay before deleting any log files, allowing
the VM start operation to return after the domain has been successfully
created.

Change-Id: I82ae3e008974f33b60256366f171abf1f0369e60
Closes-Bug: 1535616
2016-01-19 10:15:08 +00:00
Jenkins
3fdb41323a Merge "XenAPI: Fix race in rotate_xen_guest_logs" 2016-01-05 17:33:36 +00:00
hgangwx
27ed157b8b Misspelling in message
Misspelling in the message:
"You have uncommited changes"
Should be: "You have uncommitted changes"

Totally 2 occasions in Nova base code.

Change-Id: If0ad3163f3b9db3f31f7e9fa40245282c1a0acb4
2015-12-27 19:22:07 +08:00
Bob Ball
61813e69f7 XenAPI: Fix race in rotate_xen_guest_logs
It's possible that a log file is created while this script is
running, and if being created after we get the valid last_dom_ids
then the log file may be deleted incorrectly.

Also add some logging which can be used to trace any future race
conditions

Change-Id: I1df1a0775a7dae41105d6c41b046c86002eb5eec
Closes-Bug: 1516721
2015-12-17 13:15:16 +00:00
Ravi Shekhar Jethani
b93d214442 Simplified boolean variable check
Currently 'test_list' variable is being checked
for empty list like 'if test_list == []:'.
The more standard approach is like 'if not test_list:'.

TrivialFix

Change-Id: I0b389c11b992c93529b305b1ebce6b181f9a9a30
2015-11-18 23:45:28 -08:00
Sulochan Acharya
d731a04b60 xapi-tools: fixes cache cleaner script
Fixes cache cleaner script by removing import of
driver and using session directly to call vm_utils.

There is no test included for this. Since this is out of nova
tree, adding tests is tracked in 1481705

Closes-Bug: #1481693

Change-Id: I25ee22777baecaf64de63a39d19e8f8f514f70b3
2015-10-02 10:20:51 +00:00
Daniel P. Berrange
a7471dd773 tox: make it possible to run pep8 on current patch only
The 'tox -epep8' command takes a long time as it checks
every single file in the Nova git repository (~1,400 at
time of writing).

This makes tox use a simple wrapper around flake8 which
can be told to restrict the check to only files changed
in the current command. This can be invoked in a simple
manner with 'tox -epep8 -- -HEAD'. Since most commits
only touch a handful of files, this will usually be
far faster than checking all 1,400 source files.

To check an entire branch for bisectability it can be
automated via

  git rebase -i master -x 'tox -epep8 -- -HEAD'

Change-Id: I157d1ccb883ca02402eee51fd7d6a50f86079389
2015-07-24 16:15:38 +01:00
Jenkins
d8ab58339c Merge "XenAPI: Refactor rotate_xen_guest_logs to avoid races" 2015-07-20 13:00:18 +00:00
Jenkins
275c9ace5c Merge "Switch from MySQL-python to PyMySQL" 2015-07-15 15:53:35 +00:00
Matthew Treinish
62575dd40e
Add tool to build a doc latex pdf
The sphinx latex generation generates invalid latex that won't compile
when you try to use it.[1][2] This commit adds a helper script to generate
the sphinx latex and then modify it so it'll work. It depends on
ImageMagick convert and sed being available to work.

[1] https://github.com/sphinx-doc/sphinx/issues/1907
[2] https://github.com/sphinx-doc/sphinx/issues/1959

Change-Id: Id289c10907aaddae2483f18b39063852ec699d66
2015-07-14 16:13:47 -04:00
Matt Riedemann
9a84211729 Add DB2 support
Notes on migration changes/tests:

1. The initial havana migration does not create any foreign keys which
   require the instances.uuid column since it's nullable and DB2 won't
   allow creating a unique constraint over a nullable column. To create
   the foreign keys on instances.uuid, this change depends on commit
   e07a2b2d4be0503ad8dad75daa632be27cf83320 to make instances.uuid
   non-nullable and create a unique constraint on that column. Then
   the new migration here creates the missing instances.uuid
   related foreign keys for DB2.

2. Commit b930fb3a6b0ab8cbe0c19eb3ab8ba33d60d147be changed
   test_migrations.py to use oslo.db's opportunistic test fixture
   which currently only supports sqlite/mysql/postgresql. Unit test
   support for DB2 therefore needs to be added to oslo.db and then
   hooked into nova, but that's targeted for the 2016.1 'M' release
   so is not part of this change.

3. The 247_nullable_mismatch migration is updated for DB2 because
   making the pci_devices.deleted column nullable=True fails since
   the column is in a UniqueConstraint already (DB2 would require
   it to be in a unique index instead for the column to be nullable).
   There is a foreign key involved (created in 246) so for DB2 to
   have the same FKey it requires a UC and that requires the deleted
   column to be non-null, which doesn't work with the SoftDeleteMixin.
   So in this case the schema will diverge for DB2 until we make
   the deleted column non-nullable for all tables.

4. The 252_add_instance_extra_table migration is updated for DB2
   because a foreign key is created between the instance_extra
   table and the instances.uuid column (which is nullable at the
   time) so we have to handle that foreign key constraint creation
   in the 296 migration with the other foreign keys to
   instances.uuid.

5. The 271 migration is updated to add ibm_db_sa the same as
   sqlite and postgresql since it works the same in this case for
   DB2.

6. The API migrations need checks for DB2 to avoid a duplicate index
   SQL0605W error since DB2 implicitly creates an Index when a
   UniqueConstraint is created and will fail if trying to add a
   duplicate index on the same column(s) that are in the UC.

Implements blueprint db2-database

Change-Id: Ic4224e2545bcdfeb236b071642f9f16d9ee3b99f
2015-07-08 16:45:37 -07:00
Jeremy Stanley
650c811cdf Switch from MySQL-python to PyMySQL
As discussed in the Liberty Design Summit "Moving apps to Python 3"
cross-project workshop, the way forward in the near future is to
switch to the pure-python PyMySQL library as a default.

https://etherpad.openstack.org/p/liberty-cross-project-python3

Change-Id: I590c26523f4dc8d14c45e61bb89555486629b64d
2015-07-03 22:09:36 +00:00
Bob Ball
f985a11bc6 XenAPI: Refactor rotate_xen_guest_logs to avoid races
Moving console logs to a temporary file is a clear race when Nova tries to
access a file it might not be there.

We need to keep the readable file in place at all times.  Atomic operations
like mv are fine as if the file was opened before the mv operation Linux
will keep the contents available until the file is closed by Nova's plugin.

Futher, we need to be able to report the console logs from domains that have
been shutdown but still exist on the host.  Ensure we keep track of the domain
that we last booted with

Change-Id: Icfff9886d79f788be871bf28c8b729a57dcd94d2
Fixes-bug: 1466914
2015-06-26 12:17:38 +01:00
Davanum Srinivas
390fab8da4 Use oslo-config-generator instead of generate_sample.sh
Oslo team is actively getting rid of the old style generator.py
in openstack/common/config. So we should switch to the newer
oslo-config-generator which is the way forward. For simplicity's
sake, i used one single list_opts in opts.py. However we can
split this up and add entrypoints in setup.cfg if we decide to
do that instead.

blueprint oslo-config-generator

Change-Id: Ib7c72af7be47df08983708ac6da0ce103518b78c
2015-06-18 22:00:18 +00:00
Victor Stinner
51f443ec4e Don't use dict.iterkeys()
Iterating on a dictionary iterators on its keys, so calling its
iterkeys() method is not needed.

Moreover, the dict.iterkeys() method is gone in Python 3. This change
makes the modified code compatible with Python 3.

Blueprint nova-python3
Change-Id: I140f499d2054bb7d5486e4c61a2f886be6649cbb
2015-05-21 14:52:53 -07:00
Davanum Srinivas
e7b51eb27f Avoid MODULEPATH environment var in config generator
MODULEPATH is apparently a well known variable from
environment-modules package. So let's just use another
one as we don't care about the specific var name itself.

Closes-Bug: #1437904
Change-Id: Id79bbdaad498c771dc4c032256bd8160b5663ef9
2015-03-31 00:00:57 +00:00
Davanum Srinivas
a412e038dd Switch to uuidutils from oslo_utils library
Get rid of our copy of uuidutils.py from oslo-incubator

Change-Id: Idca3581475bcd4a04ce8d3420a1b7763db15b390
2015-02-25 20:05:49 -05:00
Jenkins
45e4d3a574 Merge "work around for until-failure" 2015-02-07 13:35:28 +00:00
Davanum Srinivas
af2d6c9576 Switch to using oslo_* instead of oslo.*
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.

This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware

Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
2015-02-06 06:03:10 -05:00
Jenkins
a834cd62a6 Merge "include python-novaclient in abandon policy" 2015-01-23 20:45:21 +00:00
Sean Dague
7604179d1d work around for until-failure
testr run --until-failure --subunit does not actually fail when a test
fails, upstream bug filed at
https://bugs.launchpad.net/testrepository/+bug/1411804

Work around this by reverting to old style version when running in
--until-failure mode.

Change-Id: Ic2b26c423e4e6c729bb1357bc901fdf1afdb083f
2015-01-16 14:24:29 -05:00
Daniel P. Berrange
72de6ac4f0 remove all traces of pylint testing infrastructure
The pylint tests have been broken for several months and
the Jenkins job has also been disabled. There has been
no sign of attempts to address this, so just delete what
remains of the pylint support so developers don't mistakenly
think it is supposed to work with patches they are submitting.

Change-Id: Ifacc99e45acb0e9a72be6c31113e0cf073354f02
2015-01-08 11:56:45 +00:00
Matthew Treinish
7a543bcec8 Switch to tempest-lib's packaged subunit-trace
This commit removes the local copy of subunit-trace in nova and uses
the packaged version of the utility in tempest-lib.

Change-Id: I1ed2fea9ff3d8194628b722684492f8ba7a3cabb
2015-01-06 21:25:56 +00:00
Jenkins
eeee1ede12 Merge "Use oslo db concurrency to generate nova.conf.sample" 2014-12-18 02:56:46 +00:00
Joe Gordon
6859dd61a8 Use oslo db concurrency to generate nova.conf.sample
Make sure we include database and concurrency options in the
sample config file.

Change-Id: I66e68ec120cba2cee86e36aa26720199876c40a4
Closes-Bug: #1391782
2014-12-15 12:08:30 -08:00
Sean Dague
e471f5b5d8 include python-novaclient in abandon policy
python-novaclient had some 6 month old stalled patches, add it to the
overall abandon cleanup policy.

Change-Id: I28831acc1c0cee846e79cc63138b7bc2ce8ee03d
2014-12-08 16:11:08 -05:00
Jenkins
e430443f8b Merge "add abandon_old_reviews script" 2014-12-02 13:28:41 +00:00
Balazs Gibizer
687e8664a2 Enable pep8 on ./tools directory
This patch fixes the pep8 issues of the tool scripts and
configures tox to run pep8 on the tools directory.

Two hacking rules have been modified to ignore the tool directory as both
rules are checking oslo usage which is not critical in case of tools
- N310 use timeutils instead of datetime module
- N324 use jsonutils instead of json module

The tools/xenserver directory is still excluded from pep8 as it contains
scripts that are expected to run on dom0 with python2.4 therefore the
following rules cannot be meaningfully followed
- H231  Python 3.x incompatible 'except x,y:' construct
- H233  Python 3.x incompatible use of print operator

Change-Id: Icfd9e708a2ca1e421676b410e41807b2e630718f
2014-11-25 11:42:02 +01:00
Sean Dague
900e6abdd4 add abandon_old_reviews script
This is an automated script to abandon the reviews that we agreed as a
community are beyond stale.

No activity for 4w with a -2 on them or No activity for 4w and Jenkins
doesn't have a +1 or better on it.

Can be run by any core reviewer.

Change-Id: I19636f04b691c0b74af306709d54b26978c19282
2014-11-22 08:24:14 -05:00
Balazs Gibizer
92d38e7695 Adapting pylint runner to the new message format
Pareseable output format and --include-ids parameter are deprecated
and ignored in the pylint version used on the gate.
These functions are replaced by the --msg-template parameter.

This patch changes the runner to use the new parameter so that the
job can run properly on the gate

Closes-Bug: #1394606
Change-Id: I1de424bae4c2b634cd0bc804cb7e81624c96a195
2014-11-21 09:46:27 +01:00
Sean Dague
a2d8734881 remove use of explicit lockutils invocation in tests
Instead of passing lockutils loading explicitly in the test run
command line, provide the support for individual test classes to ask
that lock_path is setup with the REQUIRES_LOCKING = True class
parameter.

Augment all classes that currently fail without this with
REQUIRES_LOCKING = True. Hopefully many of those can be individually
removed in the future so we can get out of needing external locking in
any of our unit tests.

Change-Id: I56229e93ad61ac823646a4119a035d36c3400173
2014-10-29 14:22:45 -04:00
Sean Dague
476700d2d8 error if we don't run any tests
This makes subunit-trace error out if it finds that there were no test
results in the subunit stream. This guards against the case where
something breaks the test toolchain and we stop running tests
entirely. Copied over from equivalent code in Tempest where this has
served us well.

Change-Id: I765395335a68dfc7eb754317f8c45c3dfdd85949
2014-10-28 09:03:00 -04:00
Sean Dague
cef1fea688 Revert "Switch Nova to use oslo.concurrency"
This patch made unit tests stop running (always runs 0 unit tests) we
should revert and put a check in place to prevent that from happening
so that we don't regress the rest of Nova trying to figure out the
oslo.concurrency issue.

This reverts commit 382294762b08abb1921a77b00608ac941478ce05.

Change-Id: I98b04ac2a0ba67ba8af44867799317fdf95b8aff
2014-10-28 08:51:21 -04:00
Davanum Srinivas
382294762b Switch Nova to use oslo.concurrency
Let's switch to the newly released oslo library for
the processutils and lockutils.

Change-Id: Icf7831261322abd0a6fc5439fe09b1d4cb614cb7
2014-10-27 16:56:13 +00:00
Jenkins
edcaee462c Merge "Remove obsolete vmware/esx tools" 2014-10-20 03:16:10 +00:00
Davanum Srinivas
0bfade69eb Sync with latest oslo-incubator
Changes include:
55ca7c3 Split cliutils
1b44601 Allow dictionary lookup in credentials with dot notation
6c706c5 Delete graduated serialization files
3edbfb3 remove caching param from prettytable call
9ce1d96 Fix i18n import
5d40e14 Remove code that moved to oslo.i18n
a4be4ed Remove graduated config modules
6ff6b4b Switch oslo-incubator to use oslo.utils and remove old modules
70189c9 Remove graduated test and fixtures libraries
8b71f57 Delete graduated db files
6ede600 rpc, notifier: remove deprecated modules
6b65a41 Fix build break - switch from str to hash for lookup
f76f44c Delete the token and endpoint on expiry of token of client
aebb58f Fix typo to show correct log message
1131b56 Enabled mask_password to handle byte code strings
6b048e7 Let oslotest manage the six.move setting for mox

Change-Id: Ic588aec0fc0291440feaac3439836054bd3608cd
2014-10-11 21:25:52 -04:00
Davanum Srinivas
3ef4f63ea7 Remove obsolete vmware/esx tools
Our Docs have updated information in and the README is stale.
http://docs.openstack.org/trunk/config-reference/content/vmware.html#vmware-prereqs

The guest_tool.py was once used a long time ago to setup
networking by pulling information from esx and setting
the OS details in the VM. This is no longer used or needed as
we have better ways to do the same using config drive or metadata service
http://docs.openstack.org/trunk/config-reference/content/vmware.html#VMware_networking
http://docs.openstack.org/user-guide/content/config-drive.html
http://cloudinit.readthedocs.org/en/latest/topics/datasources.html#config-drive

Change-Id: I8c2ba52cffbcf9be50fd0e2f7b2bb1da8089e324
2014-10-08 09:52:17 -04:00
Davanum Srinivas
323fa6fef7 Use oslo.utils
oslo.utils library now provides the functionality previously in
oslo-incubator's excutils, importutils, network_utils, strutils
timeutils, units etc. Some modules already moved to oslo.utils
will still be around since other code in nova/openstack/common/
are using it and will be removed in a subsequent commit.

Change-Id: Idc716342535fdfa680963e0e073ddb46f5f1eb34
2014-10-06 21:41:17 -04:00
Sean Dague
9c448bfb5f support TRACE_FAILONLY env variable
In order to get similar behavior to the way testr out of the box
works, only displaying failing results, all the user to define
TRACE_FAILONLY environment variable (any value will do). After this
point only failures, and things leaking out of stderr will end up on
the screen.

We also need to remove the suppression of the inline fail dumps for
this to have the impact that you'd expect.

Change-Id: I72911adc8a03450522429c52db661ec0a1bc4678
2014-10-01 15:08:03 -04:00