361 Commits

Author SHA1 Message Date
Jenkins
318e64e0ff Merge "fix TypeError with unstarted threads in nova-network" 2012-04-10 21:52:19 +00:00
Yuriy Taraday
559a4df121 Improved localization testing.
Moved localization tests to tools/hacking.py.

Change-Id: I903b90dfb09a46a72b1c64c30301f90661999f5b
2012-04-10 22:01:55 +04:00
Johannes Erdfelt
5f2b4c1ec7 Cleanup xenapi driver logging messages to include instance
Also, use the logging instance kwarg so it gets logged in a consistent
format.

Change-Id: I4429e9660ff72e1f79d889f89b5abbf1949720de
2012-04-06 15:07:43 +00:00
Ionuț Arțăriși
ebdef01425 fix TypeError with unstarted threads in nova-network
(nova): TRACE: File "/usr/lib64/python2.6/site-packages/nova/utils.py",
line 860, in __init__
(nova): TRACE: self.tname = "-%x-%x" % (ident & 0xffffffff, gident &
0xffffffff)
(nova): TRACE: TypeError: unsupported operand type(s) for &: 'NoneType'
and 'int'

Fixes: bug 971629

Change-Id: I3d34816c36a11cd6fc2dd9c889c93c5872133a2c
2012-04-02 17:29:33 +02:00
Pádraig Brady
e3992a6e68 ensure atomic manipulation of libvirt disk images
This pattern could probably be used elsewhere,
but only libvirt disk images are considered for now.
This change ensures there are no stale files left
anywhere in the path from glance, through the libvirt image cache.
These could cause subsequent operational errors either
directly or indirectly through disk wastage.

* nova/utils.py: Add a new remove_path_on_error() context manager
that is used to remove the passed PATH on a raised exception.
* nova/virt/images.py: Ensure temporary downloaded and
converted images are protected.
* nova/virt/libvirt/connection.py: Ensure all the images in
the image cache and instance dirs are protected.

Change-Id: I81a5407665a6998128c0dee41387ef00ebddeb4d
2012-03-24 02:51:38 +00:00
Rick Harris
9807bb1944 Add periodic_fuzzy_delay option.
Fixes bug 962665

This random delay is intended to reduce the stampeding behavior
associated with periodic tasks when compute workers are restarted in
unison across a cluster.

Change-Id: Ie3771d94af29049061c129b8ea562ee447a61771
2012-03-23 21:03:37 +00:00
MotoKen
4d536585fd Fixes bug 957708.
Work around bug in netaddr 0.7.5
https://github.com/drkjam/netaddr/issues/2

Change-Id: I3e064860c3865d5029481e6d63c5284d275a0081
2012-03-20 14:55:21 +08:00
Jenkins
c8e2979b9b Merge "various cleanups" 2012-03-18 23:20:04 +00:00
Vishvananda Ishaya
6bd4a551f1 Workaround issue with greenthreads and lockfiles
* Adds a GreenLockFile that always works in greenthreads
 * Adds test to verify that regular Lockfile is broken
 * Adds test to verify that GreenLockfile works
 * Adds note about limitation of external locks
 * Adds test showing limitation of nested locks
 * Fixes bug 956313

Change-Id: I11cd1206611aa4862dadd2fcc077c4c2e0f798f6
2012-03-16 13:56:17 -07:00
Pádraig Brady
c052b03350 various cleanups
No functional changes

Change-Id: I19e118e6e0c47fccfe5a817e727f3ab27e26aa79
2012-03-16 04:36:26 +00:00
Vishvananda Ishaya
064fc1f8d7 Refix mac change to work around libvirt issue
* fixes bug 921838

Change-Id: I11278a03c4429686499b2f62c66a7f442258f5a6
2012-03-14 10:35:18 -07:00
Jenkins
e8cdf673e6 Merge "Use a high number for our default mac addresses" 2012-03-10 04:18:07 +00:00
Mark McLoughlin
b435fcc734 Add pybasedir and bindir options
Add a pybasedir option so that it can be used for interpolation in the
default values of other options. This helps eliminate hard-coded paths
from the sample config file.

Also add a bindir option for similar reasons, but it also helps with
packaging.

Change-Id: Iadc746dcf2a24adbdf9bac945b5b330f01faeeb5
2012-03-09 11:23:33 +00:00
Vishvananda Ishaya
b4df4ff3a3 Use a high number for our default mac addresses
* FE has the locally administered bit set and multicast bit unset
 * fixes bug 921838

Change-Id: Id33a06985c4150da4c5367c700f894590fdac2b9
2012-03-08 15:36:31 -08:00
Monsyne Dragon
f88b5595c9 Add adjustable offset to audit_period.
Allows audit periods to be offset, so daily periods can begin at local
midnight, instead of UTC, monthly periods can begin on the 15th
instead of the 1st, etc.

Fixes bug 948601

Change-Id: I38f6af0a5a513f888b791a4b9ca39467030105f2
2012-03-07 16:32:47 +00:00
Doug Hellmann
91e3524cfc fix restructuredtext formatting in docstrings that show up in the developer guide
blueprint sphinx-doc-cleanup
bug 945160

- Correct parameter declarations, list formatting, cross-references, etc.
- We don't need "let" in generate_autodoc_index.sh since we aren't doing math.
- Change conf.py to not prefix class and function names with full namespace
  in generated output to save width on the screen.

Change-Id: I9adc8681951913fd291d03e7142146e9d46841df
2012-03-05 16:18:53 -05:00
Johannes Erdfelt
1827563701 Only raw string literals should be used with _()
Fix a number of places where formatted strings were used with _() (causing
gettext to not match the string) or variables with _() (causing xgettext
to not extract a string)

Also, there's no value in internationalizing an empty string

Change-Id: Iac7dbe46eeaa8ddf03c2a357ecd52f69aa8678aa
2012-03-04 22:10:59 +00:00
Rick Harris
e3584cb8bd Refactor spawn to use UndoManager.
UndoManager provides a mechanism for automatically rolling back on
exceptions.  An additional benefit of this approach is that undo code is
spatially close to the roll-forward code.

This patch should be considered an intermediate step towards a more
complete Command pattern based approach down the road.

Change-Id: Ib429420d67324422a5d13cdea6872fd9c57c857e
2012-03-01 22:13:03 -06:00
Thierry Carrez
251f52a845 Add missing filters for new root commands
Add missing rootwrap filters for 'ovs-ofctl', 'cp' and 'mkfs'.
Do not run 'rm' as root since it's unnecessary.
Add documentation to try to prevent future misses.
Fixes bug 943293.

Change-Id: Ia680048a28a75f661a136d8447ff0aaf195649ba
2012-02-29 20:30:21 +01:00
Russell Bryant
4bb182a243 Use constant time string comparisons for auth.
Fix bug 942644.

Use constant time string comparisons when doing authentication to help
guard against timing attacks.

Change-Id: Iaaefb13f7618b06834630d9ccb97aff056b4bf4c
2012-02-28 14:56:44 -05:00
Jenkins
2a1c1c846c Merge "OS X Support fixed, bug 942352" 2012-02-28 19:17:07 +00:00
Johannes Erdfelt
558f37b1d4 Add utils.tempdir() context manager for easy temp dirs
Fixes bug 883323 (and others)

Users of tempfile.mkdtemp() need to make sure the directory is cleaned
up when it's done being used. Unfortunately, not all of the code does
so at all, or safely (by using a try/finally block).

Change-Id: I270109d83efec4f8b3dd954021493f4d96c6ab79
2012-02-28 05:56:58 +00:00
Joe Gordon
c1132ff5c7 OS X Support fixed, bug 942352
Change-Id: I3aa28ba46b0418310637dfeacbdea60361a7a280
2012-02-27 17:57:33 -08:00
Rick Harris
cb8fe3ebe7 Adds temporary chown to sparse_copy.
`sparse_copy` needs read and write access to the devices. Since we
cannot shell out to a run-as-root command here, we temporarily take
ownership of the device.

Change-Id: I891c38dbcba7177286dca729684c88ac065bd085
2012-02-27 21:14:48 +00:00
Mike Pittaro
a7400b41b0 Clean stale lockfiles on service startup : fixes bug 785955
Adds cleanup_files_locks() to nova/utils, which cleans up stale locks
left behind after process failures.

Adds a call to clean up locks on service startup for nova-api, nova-cert,
nova-compute, nova-network, nova-objectstore, and nova-scheduler.

Adds tools/clean_file_locks.py, which can be used to manually clean
stale locks.

Change-Id: I752e0b24d3c7fc5f1dc290da355cbd7f430789b8
2012-02-24 15:10:33 -08:00
Jenkins
bd088ace2d Merge "Add attaching state for Volumes" 2012-02-23 18:32:53 +00:00
Liam Kelleher
af753c7fc0 Add attaching state for Volumes
bug 897726 Add "attaching" status for volumes and clear state on error
This closes a gap where multiple calls of attach for the same volume
can happen before the volume is actually attached.

Change-Id: I59d22ceda83729c1a455af9994c9ffec1912e23b
2012-02-20 14:21:25 +00:00
Nick Bartos
e88925f065 Escape apostrophe in utils.xhtml_escape() (lp#872450)
Also remove comment which is no longer relevant.

Change-Id: I33a951d08a34510b2a9dbacb9fb3ebf6bee978b4
2012-02-20 04:57:44 -08:00
Eoghan Glynn
bbac2bd05e Support non-UTC timestamps in changes-since filter
Fixes nova aspect of lp 837464

Prevously only Zulu time was supported in the changes-since filter,
i.e. timestamps formatted as %Y-%m-%dT%H:%M:%SZ

We now support arbitrary timezones, with the offset from UTC expressed
via the ISO 8601 ±hh:mm notation.

Microsecond accurracy is also optionally supported in timestamps.

Notes:

- nova.utils.parse_isotime(), isotime() & normalized_time() are
  prime candidates for promotion to openstack-common, as these
  methods were duplicated from my corresponding glance patch:

    https://review.openstack.org/#change,4198

- this patch introduces a new dependency on python-iso8601, which
  has already been packaged for Fedora, EPEL and Ubuntu/Debian.

Change-Id: I89b45f4f3d910606c578d927420f78cea94f4e3b
2012-02-17 20:40:51 +00:00
Jason Kölker
77492777b0 Allow file logging config
* Fixes lp904305
* remove module level log functions (getLogger to rule them all)
* Move specific Environment logging to the one place it is used
* Wrap getLogger to return a logger wrapped in a NovaContextAdapter
* Do not overwrite the root logger
* save_and_reraise_exception logs via error for passing exc_info
* Uses CommonConfigOptions for compatability across Openstack Projects
* Prefers CommonConfigOptions over legacy options
* Install a NullHandler on the root logger if configured by FLAGS
* Include example logging config file to mimic Nova defaults

Change-Id: Ie59c3f755c142e2b7dc3b94b4e82e142e157bfac
2012-02-17 11:24:32 -06:00
Jason Kölker
67959068ac Standardize logging delaration and use
* Make modules use getLogger(__name__) and log to the result

Change-Id: Ib6d69b4be140ec89affc86ed11e65e422d551df1
2012-02-14 13:02:43 -06:00
Mark McLoughlin
707b6141d7 Remove the last of the gflags shim layer
Make FLAGS a ConfigOpts instance and fix up all the places where we
expected FlagValues behaviour.

Change-Id: I8f96f42e0d8d30ba6b362d29861e717cf0fa9e89
2012-02-10 06:31:56 +00:00
Jenkins
00173e15c0 Merge "Backslash continuations (misc.)" 2012-02-10 00:39:50 +00:00
Johannes Erdfelt
1a9cb8e679 Use named logger when available
Cleanup a handful of places where named loggers are created for a
particular module but aren't used consistently. Also fix a couple
of log entries that aren't internationalized

Change-Id: I38186d47e7f31626b3ead779707d8ee5a15f56ac
2012-02-08 21:53:55 +00:00
Zhongyue Luo
674fa33253 Backslash continuations (misc.)
Fixes bug #925166

This patch for packages which have few backslash continuations.

Follow up patches will be for packages network, scheduler, virt,
db/sqlalchemy, tests, and api/openstack.

Change-Id: I4200010b47b33fa8b9115b5d379b543200f6668d
2012-02-08 08:52:43 +08:00
Johannes Erdfelt
119edb84cc Fix support for --flagfile argument
Recent changes broke support for the --flagfile argument. It would attempt
to find the argument and then pull off the filename portion, but it would
only skip one letter, instead of the full argument name. The resulting
error would be:

Change-Id: Ic252b0c778774a7ea3e24a9ed8b182deb5987f65
OSError: [Errno 2] No such file or directory: 'lagfile=../nova.conf'
2012-02-07 15:49:13 +00:00
Jenkins
06a12f39be Merge "Allows nova to read files as root" 2012-02-06 17:18:45 +00:00
Vishvananda Ishaya
a139c4309d Allows nova to read files as root
* Adds a rootwrap filter to limit readable files
 * Adds a utils method to read a file as root
 * adds tests to verify the additions work

Change-Id: Ic1ffb6f72f9b73795d5f39fac719842e6bdf16dd
2012-02-03 16:49:42 -08:00
Soren Hansen
375f0693ce Re-run nova-manage under sudo if unable to read conffile
Having to manually sudo to the nova user to make things work is
tedious. Make it so that if we can't read the conffile, we just
re-exec under sudo.

Fixes bug 805695

Change-Id: I322cece80ca757c69147fb3f8474ad137d9bff82
2012-02-04 00:26:26 +01:00
Mark McLoughlin
8e26006b8e Move cfg to nova.openstack.common
Move it here so that it can be kept in sync with openstack-common using
the new update.py script for code in openstack-common's incubation area.

See here for more details:

  http://wiki.openstack.org/CommonLibrary#Incubation

Note: this commit just moves the existing code in Nova with no other
changes. A subsequent commit will sync it with latest openstack-common
so that it is easier see the new changes.

Change-Id: If88d678b1b9bad3d37117de7f7159d7fea8ab4c8
2012-02-03 19:21:54 +00:00
Michael Still
1b92784908 blueprint nova-image-cache-management phase1
Phase 1 of the image-cache-management blueprint. Implement
checksumming and verification of base images, as well as
removal of base images which are not in use. Removal is
enabled with a flag.

Change-Id: Ib04a02075a103c2fef6716fd0ac87c41a3535d37
2012-02-03 09:53:15 +11:00
Mark Washenberger
e8785ced97 Optionally disable file locking.
File locks in nova have a bad habit of lingering if the process that
created them is killed, and there isn't a good automated way to fix this
behavior. This option allows a deployer to avoid the problem if they
know they are only running a single nova process on a machine.

Change-Id: I1ae20cc54a4614b200093ffd581d3ab21d7c241b
2012-02-02 11:41:12 -05:00
Brad Hall
e05a2b9de5 Add support for pluggable l3 backends
This will allow us to support backends other than linux_net (i.e.
quantum L3 when it is available) for defining L3 connectivity.

Change-Id: I0b2ece2278bd68166741107a88cedd106d1ab651
2012-02-01 15:44:41 -08:00
Jenkins
aedcfa5351 Merge "Fix VPN ping packet length." 2012-02-01 21:49:49 +00:00
Jenkins
d992216b38 Merge "lockfile.FileLock already appends .lock" 2012-02-01 20:42:33 +00:00
Johannes Erdfelt
cbdb863922 lockfile.FileLock already appends .lock
Otherwise you end up with files like nova-iptables.lock.lock

Change-Id: Ia7dd82c3721f89d438c1266e7ec3535a0d89334e
2012-02-01 20:08:29 +00:00
Trey Morris
b65659d186 Ties quantum, melange, and nova network model
get_instance_nw_info() now returns network model, and keeps the network
info cache up to date.
virt shim and translation in place for virts to get at the old stuff

Change-Id: I070ea7d8564af6c644059d1c209542d250d19ddb
2012-02-01 13:29:14 -06:00
Russell Bryant
baf5f4c866 Fix VPN ping packet length.
Fix bug 906346.

This patch addresses the typo pointed out in bug 906346.  The ping being
sent was 15 bytes long when it should have been 14.  Removing a pad byte
from the format string resolves this issue.  I verified that the format
described in the code comments (and now the code) was correct using an
OpenVPN connection setup packet capture.

Change-Id: Idbc5e48ede4a8d2836dd1b102a9a0e172540776c
2012-01-31 20:30:50 -05:00
Russell Bryant
5feaa44e38 Remove utils.runthis().
Fixes bug 884825.  The bug was pointing out some unused variables
noticed by pylint.

utils.runthis() was only used in one place and doesn't really provide
much value over utils.execute(), so just remove it.

Change-Id: I3dcc34bd840a128418b58a8a66acb62adf3fc9cc
2012-01-31 14:52:55 -05:00
Deepak Garg
3e55b3e9e7 Blueprint xenapi-provider-firewall and Bug #915403.
1. Provides dom0 IPtables driver to implement the Provider firewall rules.
  2. Existing libvirt code has been refactored to reduce the amount of duplicated code to a minimum
  3. The three provider apis in ec2/admin.py file are now fixed the following way:
    a.    remove_external_address_block returned 'OK' on removing blocks which didn't exist. This is now fixed.
    b.    block_external_addresses raised exception earlier on duplicate network blocks. Now the exception is logged and failed status message is returned.
    c.  all the three provider apis now logs for invalid and improper inputs and return uniform (a dictionary ) and proper status messages for all cases.
  4. appropriate unit tests added to cover the same

Change-Id: I27d83186f850423a6268947aed0c9a349d8f8d65
2012-01-24 22:31:35 -08:00