362 Commits

Author SHA1 Message Date
guillaume-thouvenin
d5b8d5b18d XenAPI: Add the support for updating the status of the host.
We added a function into the xenhost plugin to get information
about PCI devices. Roughly we run the lspci command on dom0. This
information will be used to get the list of pci devices  that are
passed on the pciback.hide dom0 command line.

  The hide option is used to hide the devices from the normal guest
drivers and assign them to the pciback kernel driver at boot on dom0
instead of their normal driver. We will parse the output of the lspci
command to find which device is using the pciback kernel driver and
thus to know if it has been passed to the pciback.hide option.

  This information will be used to perform the match with the
list of pci devices provided in pci_whitelist into /etc/nova/nova.conf.

Implements: blueprint pci-passthrough-xenapi

Change-Id: I465fc5d29f3c47ab0079adcfcc2d7d6501bd4b20
2014-02-17 11:56:18 +01:00
Bob Ball
270d4f1d6b XenAPI: Monitor the GC when coalescing
Add a new host plugin to monitor whether the garbage collector is running.
This is then used by the wait_for_vhd_coalesce function to know whether
waiting any more is likely to produce the expected results.

Increased the number of times we loop waiting for coalesce as the loop
now has knowledge that something is still happening.

DocImpact
Change to parameter default.

Change-Id: Idcc0738945bd1aee8cdb52c9cfabd798f85c31db
Closes-bug: 1258169
2014-02-05 23:54:37 +00:00
liu-sheng
74f953a1d7 Remove vi modelines
We don't need to have the vi modelines in each source file,
it can be set in a user's vimrc if required.

Also a check is added to hacking to detect if they are re-added.

Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6
Closes-Bug: #1229324
2014-02-03 14:19:44 +00:00
John Garbutt
89b0d540d5 xenapi: stop hang during glance download
It seems there are cases when the server gets stuck in a building state
because the download from glance fails, but the nova-compute doesn't
find out.

After some code inspect, a likely cause seems to related to having no
socket timeout inside urllib2 when we download the image from glance. A
workaround for this issue is to configure the default socket timeout as
described here:
http://docs.python.org/3.1/howto/urllib2.html#sockets-and-layers

Fixes bug 1251650
Change-Id: I8b34d95d3524f3825b28e61cbbb59290653d3711
2013-12-03 18:42:24 +00:00
Brian Elliott
f20523aeca xenapi: Fix agent update message format
Fix the agent update message send so Windows guests will work

Message must be formatted: "url,md5sum"

Change-Id: I675cc5a7ca522696eab1820c14c48361ff7e0f74
Closes-Bug: 1250165
2013-11-12 20:19:25 +00:00
Joe Gordon
df4eb86234 Remove xenapi rpm building code
Nova shouldn't be building any types of packages. And if we are we should
be testing this code in gate somewhere.

Change-Id: I70d82a6173b5eda183d4e49d330ae97dd14f9078
2013-10-15 14:38:40 -07:00
Bob Ball
24fd331b8c XenAPI: Add versioning for plugins
Because the plugins live on a host seperate to Nova we need an interface
to test whether they are the expected version.  We can't use pip or other
requirement systems as they are cross-machine.

Closes bug 1226622

Change-Id: I58ab669061f51bd87071e2cf0d93d33021001309
2013-10-03 13:08:56 +00:00
Matt Riedemann
016e39734e Fix several flake8 issues in the plugins/xenserver code
Due to some issues in the gate, several flake8 errors got merged
recently. This patch fixes the most recent issues found.

For the H304 relative import errors, we have to skip those because they
are put into /etc/xensource/scripts which is the current working
directory when running the plugin.

For the H231 incompatible python 3 'except x,y' construct, we have to
skip those because this code is written to run on python 2.4 and 'except
x as y' does not work with python 2.4.

Note that in cleaning up some of the H304 failures for relative imports,
I also re-arranged the imports to follow the hacking guide of doing
standard library packages first, then third party packages, and finally
nova-specific packages.

Closes-Bug: #1229753

Change-Id: I1c2211fd6a10d43d7e65cdb4e18530397788cf2c
2013-09-24 09:04:21 -07:00
Jenkins
fc07ac93ba Merge "xenapi: Cleanup pluginlib_nova" 2013-09-24 00:27:12 +00:00
Rick Harris
097f9a34a2 xenapi: Cleanup pluginlib_nova
* Removes dead code
* Prefixes internal functions with underscore
* Removes a few one-off functions in favor of inlining

Change-Id: Ic2f8c2e512df270f3b1b0e35ded98832a5152c72
2013-09-23 18:49:03 +00:00
jmeridth
2266c04458 xenapi: fix pep8 violations in nova plugins
Removed plugins/xenserver from tox.ini excludes list, ran tox -e pep8, and
fixed violations.

Fixes: bug 1197893
Change-Id: I8e0116304b07d54c49a01202b3ce6d3426097bb4
2013-09-17 16:44:59 +00:00
Jenkins
7e8f27e1eb Merge "Adds Instance UUID to rsync debug logging" 2013-08-30 22:52:24 +00:00
Mate Lakat
36be3145c7 Add env to make_subprocess
Due to a previous commit, a client of make_subprocess is making a call
with an env parameter. As make_subprocess does not have such a
parameter, image upload fails. This patch adds the env keyword argument
to make_subprocess, and use that in the Popen call.

Fixes bug 1216267

Change-Id: I8098b484067b54c15fe95f2bf84c5890db7ebcb3
2013-08-27 10:43:32 +01:00
Jenkins
c1742adfe1 Merge "xenapi: Added iPXE ISO boot support" 2013-08-26 17:00:23 +00:00
Zaina Afoulki
1df56a5c4a Adds Instance UUID to rsync debug logging
This commit adds the instance uuid to the rsync debug output in
the debug logs. The issue we were seeing is that when multiple
migrations are ongoing at the same time, it's not easy to
track the rsync progress of each migration.
Fixes: bug #1215629

Change-Id: Iec662bdc0aed1c6f54ecaa062335dc5504ac030f
2013-08-22 16:09:09 -05:00
Teran McKinney
9a24ae88da nova.conf configurable gzip compression level
Currently only used by XenAPI.

Sets GZIP environment variable before spawning tar -cz to force the
compression level. Defaults to none, which means the GZIP environment
variable is not set and the default (usually -6) is used.

Implements: blueprint image-compression-mode

Change-Id: I02e136b7215a4f2cefc259f40bde9c1b205737ed
2013-08-22 17:22:36 +00:00
Josh Kearney
614c4a2359 xenapi: Added iPXE ISO boot support
This patch adds support for a new kind of Glance image, a specially crafted ISO
which supports iPXE booting, giving customers a means to roll their own image.

Two virt-layer modifications were needed. The first was adding configurations
for the iPXE ISO feature (network to use, boot menu, mkisofs_cmd). The second
was the ability to inject networking info into the ISO after it was
downloaded.

To use this feature, operators should enable the `ipxe_boot` image-property.

DocImpact

Implements blueprint xenapi-ipxe-iso-boot-support

Change-Id: I33acf9dfdff0a5ed9797723a142bc451348e8549
2013-08-21 22:17:33 +00:00
Jared Culp
47a20a9f44 xenapi: send identity headers from glance plugin
Send identity headers while uploading/downloading images from glance. Related
to:

1) Add identity headers while calling glanceclient from nova
Ife02059abbdce7920f1f408b71e9745d777fa770

2) Pass identity headers received in glanceclient to nova
Ifbef582aa4e64a2e7a46db43a9cc6cf8c3531dbd

3) Pass identity headers received by glance api to glance registry
Ie5f07ed6dfeaa8428de4f79c4d40d182328e6ab4

NOTE: since this is a change to a Dom0 plugin API, it requires the lock-step
upgrade of both the compute-manager and the dom0 plugin.

DocImpact

Change-Id: I6d5e3448d2c0acc392d18e0b88cec25cf313da5b
2013-08-08 17:51:22 +00:00
Jenkins
3475662e02 Merge "xenapi: no image upload retry on certain errors" 2013-08-08 16:02:56 +00:00
Brian Lamar
6d3d641de5 Remove * import from xenserver plugins
Recently a change was made to how/where the '_' method is imported. This
seems to have broken most xenserver plugins. These plugins shouldn't be
doing * imports anyway so I've gone in and cleaned up all places where
"from pluginlib_nova import *" was used and replaced with "import
pluginlib_nova".

Fixes bug 1207107

Change-Id: I3b2a299cf60543e0f766becfeb4740af791b3576
2013-07-31 18:17:45 -04:00
Sulochan Acharya
7aa6f8465c xenapi: no image upload retry on certain errors
There are a few upload errors that glance plugin should
not retry upload on, as it will fail again. This patch changes
upload to not retry on the following conditions in addition
to the 401 error that it already handles:

- CONFLICT
- REQUEST_ENTITY_TOO_LARGE
- PRECONDITION_FAILED
- FORBIDDEN

Fixes bug 1203726

Change-Id: I538a4f9b133169b2e398e7dfcb3af58fef204f03
2013-07-31 17:03:41 -05:00
Alex Gaynor
cd9d8e45c6 Remove the monkey patching of _ into the builtins
Previous _ was monkey patched into builtins whenever
certain modules were imported. This removes that and
simply imports it when it is needed.

Change-Id: I0af2c6d8a230e94440d655d13cab9107ac20d13c
2013-07-26 07:31:17 -07:00
Jenkins
e6812fc4e6 Merge "xenapi: Pass string arguments to popen" 2013-07-25 23:08:41 +00:00
John Garbutt
8e81e069a1 xenapi: no glance upload retry on 401 error
It is possible for a token to expire during a very long image upload.
If this happens, there is little point in retrying the upload, as it
is most likely to fail again for the same reason.

Fixes bug 1199454

Change-Id: Id699135a8ec4c603671b16f0127d0b2cd2a55855
2013-07-18 08:06:40 +00:00
Rick Harris
3cc2264eea xenapi: Pass string arguments to popen
The current BitTorrent code was passing integer arguments to `make_subprocess`
which would in turn pass those to `popen`. This would cause a failure because
the semantics of the underlying `execve` function is to only accept a list of
strings.

The solution is to cast the arguments to a string before passing them into
`make_subrpocess`.

Fixes bug 1201594

Change-Id: I8372795ca032420641c449cb07bb47f9252ff4ef
2013-07-17 22:06:08 +00:00
John Garbutt
d871738859 xenapi: glance plugin should close connections
Current the xenapi glance plugin does not close
httplib connections when exceptions are raised.

Note this code is run under python 2.4 so with
is not available.

Fixes bug 1197846

Change-Id: I5cfb067320e7506109c301532a95fe83aa4fd0f9
2013-07-09 17:40:50 +00:00
Euan Harris
61ef64f48f xenapi: Tidy up Popen calls to avoid command injection attacks
* Rewrite xenhost._run_command and xenstore._run_command to use
   utils.make_subprocess and utils.finish_process
 * Change exception raised by utils.finish_process to retain information
   needed by calls in xenhost and xenstore

Change-Id: Idcdb50bededf0acde92f1774d6752043ba8f97ce
Fixes: bug #1074087
2013-07-03 10:05:45 +01:00
Brian Elliott
eb52c017a1 xenapi: Make BitTorrent url more flexible
Change xenapi bittorrent plugin to not assume that torrent file
is under CONF.xenapi_torrent_base_url + image_id + ".torrent".

This allows for sources images from arbitrary torrent URLs.

Change-Id: Ica9462a51c6299b5affc861781f3054d18866458
2013-06-26 15:02:28 +00:00
Brian Elliott
1fcebf22a6 xenapi: Improve cross-device linking error message.
Catch invalid-cross device linking exceptions raise from os.rename
and provide a more intelligible error message.

Change-Id: I22b6a145141431b6d4e4cb0176881f0cf0dd609f
2013-06-26 15:02:28 +00:00
Chris Behrens
ee1accfe03 Fix xenstore-rm race condition
If an entry is already gone from xenstore, any other xenstore-rm will
raise.

When we make calls to nova-agent, we wait for the response in Xenstore.
After a timeout period, we attempt to remove it.  If the DomID has
changed (due to a Windows VM rebooting at the right time via sysprep,
etc)... or if the agent picked up the command and didn't respond in a
timely fashion, the removal attempt will fail.  It would be normal for
the entry to be gone in this case.

This fixes the plugin to ignore when the entry is already
deleted.

Fixes bug 1193720

Change-Id: I8585a9f147cad3c5abe531620872b475eb4128c9
2013-06-23 02:13:06 +00:00
John Garbutt
6e93fefade xenapi: implement get_console_output for XCP/XenServer
If an administrator has enabled the logging of guest
consoles on XCP/XenServer, this enables nova to return
the last MB of those logs to the user.

The management of the logs on the server is a little
tricky, and will be sorted in a later patch.

Change was based on this previous idea:
https://review.openstack.org/#/c/17959/

DocImpact
Part of blueprint xenapi-server-log
Change-Id: I23c83bcf8c648cc2714a0c78951acc29a16d5c31
2013-06-12 14:39:49 +00:00
Jenkins
04c446f575 Merge "Improve message and logging for corrupt VHD footers" 2013-05-14 07:32:34 +00:00
Matthew Sherborne
9ef6d6874e Improve message and logging for corrupt VHD footers
* Handles the output of vhd-util check a bit more intelligently.
 * Changes it to use LOG like in other modules
 * Adds a necessary import (errno)

Change-Id: I75d6b9e081159bc29732a721deb7a557e5f2ee06
2013-05-10 22:03:18 +10:00
Bob Ball
e507094eef Enable live block migration when using iSCSI volumes
Fix for bug 1160323.

DocImpact
Depends on a version of XAPI supporting relax-xsm-sr-check.
Currently no release versions support this, so the change at
https://github.com/xen-org/xen-api/pull/1116 would need to be applied
to the source to enable this.  XenServer 6.2 is expected to
support this mode, and possibly some future hotfixes for XenServer 6.1.

It also depends on a configuration change which must be documented and added to devstack:
* Set "relax-xsm-sr-check = true" in /etc/xapi.conf

This commit makes the following changes:
* Attach the SR on the destination host prior to migrate
* Returns the SR-ref from the pre_migrate call (API change)
* Populates the XS VDI map with the sr-ref on the destination host
* Removes the connection to the SR from the source host post-migrate
* Added plugin to determine if iSCSI block migration is enabled

Associated tempest test at https://review.openstack.org/#/c/26478/

Change-Id: I917d9cf094190d636f4b9e14f6c8e728ff85af0e
Fixes: bug 1160323
2013-05-07 14:39:59 +00:00
Mate Lakat
f8d25feed1 xenapi: rpmbuild fixes
Fixes bug 1157695

This fix removes the parted dependency from the rpm specification file,
and also gets rid of the assumption, that the build-rpm.sh is executed
at a specific directory.

Change-Id: I1dcf9096a783a88bc09af0f4f4d5541a7e505748
2013-03-20 15:29:09 +00:00
Jenkins
07694d4ffa Merge "xenapi: Adding logging for migration plugin" 2013-03-19 16:27:03 +00:00
Andy Hill
3c0f4d0c3d xenapi: Adding logging for migration plugin
With this change administrators will be able to examine hypervisor
logs for real-time progress of rsync operations (resizes, migrates).

Change-Id: I463b574be1021e141fe2c8380b9c07f6cd860c78
2013-03-18 13:58:57 -05:00
Dan Prince
7f1b1b69d7 Revert 1154253 causes XenServer image compat issue
In 1154253 we:

  Drop gzip flag from tar command for OVF archives

While support more OVF file formats is a worthy goal the approach
in 1154253 is going to cause serious issues for existing
Nova XenServer users who are already zipping their OVS images.

I would also point out that our utils.py xenserver plugins still
zips tarballs we create (see create_tarball) so the new code
causes internal inconsistencies as well.

This should fix SmokeStack.

Fixes LP Bug #1155783.

Change-Id: Iff6db314c0dec6df2e48d98db2a547f6624b9d9a
2013-03-15 16:57:47 -04:00
Greg Ball
f5a2890621 Drop gzip flag from tar command for OVF archives
Fixes bug 1154253

Change-Id: I3880d9b187dc25d2ff44fe502eec4d1e0bce4218
2013-03-13 10:39:26 -05:00
Jenkins
b3e097a0f4 Merge "Adds retry on upload_vhd for xapi glance plugin" 2013-03-04 16:54:13 +00:00
Nicholas Kuechler
2db1a0f62b Adds retry on upload_vhd for xapi glance plugin
Add a retry for the xapi glance plugin to handle transient issues
when uploading the vhd.

An example issue we're seeing is a connection timeout:

['XENAPI_PLUGIN_FAILURE', 'upload_vhd', 'error',
		"(110, 'Connection timed out')"]

To work around transient issues such as a connection timeout, we
should retry based on glance_num_retries before outright failing.

Change-Id: Ice6fdd3dd39ef40e5997d69209aaafa66bff5d6e
Fixes: bug #1134493
2013-02-27 17:01:01 -06:00
Kurt Taylor
d17f9ab13d Update OpenStack LLC to Foundation
Update all references of "LLC" to "Foundation".

Change-Id: I009e86784ef4dcf38882d64b0eff484576e04efe
2013-02-26 19:15:29 -05:00
Rick Harris
0a93b563e9 Harmonize PEP8 checking between tox and run_tests.sh
Tox and run_tests.sh were running PEP8 checks against different
file-sets. This patch refactors the logic to determine which files to
run PEP8 checks on into `tools/run_pep8.sh` where it can be called by
both tox and `run_tests.sh`.

Additional fixes:

Some of our Python XenAPI Dom0 plugins don't end in *.py but should
still be checked by PEP8. This patches fixes the hacking.py violations
in the files and adds them back to the srcfiles list.

Merged tools/unused_imports.sh into tools/run_pep8.sh

Change-Id: Id5edd1acb644ab938beffc3473494a179d9d8cda
2013-02-15 00:39:58 +00:00
Pádraig Brady
f379db545c fix misspellings in logs, comments and tests
Flagged with: https://github.com/lyda/misspell-check
Run with: git ls-files | misspellings -f -

Fixes bug: 1100083
Change-Id: Icf1f844fea8ad0a1101d1dc64b9a126608e9536e
2013-01-21 23:14:13 +00:00
Sean Dague
5fca1d9e97 fix new N402 errors
fix the N402 errors that have slipped in in the last 48 hrs since
starting this patch series.

fix an N401 error that our scanner current can't find because it
only looks for doc strings on classes and defs.

this is the xeno's paradox of patch series, but we're getting close.

Change-Id: I4a763bb4c812335d853eae05c72464f18ab93297
2013-01-10 08:57:09 -05:00
Monsyne Dragon
65644195fe Fix bandwidth polling exception.
When the bandwidth polling task tries to update bw usage for an
instance that does not have an entry for the last two periods already
in the bw_usage_cache table, it will throw an exception in the
polling task. This was just a misses 'if' statement.

Also:
xenserver: fix swapped bw data in xs plugin.

This fixes bug 1075255

Change-Id: I44bb143039fcdfc8dacb13b67ae8f79dc5f38777
2012-11-05 17:46:55 +00:00
Rick Harris
216a83da47 Add ability to download images via BitTorrent.
This patch adds a new dom0 plugin which supports downloading images via
BitTorrent. Torrent metadata files are assumed to be served from a
webserver which is specified by the `torrent_base_url` config.

Under the hood, the dom0 plugins calls out to rasterbar's libtorrent via
Python bindings in order perform the initial download as well as the
seeding thereafter.

Implements BP xenserver-bittorrent-images

Change-Id: I824720a6e3a37317080a22cd7405d2a88172c3ef
2012-10-03 21:29:46 +00:00
Jenkins
d399c75568 Merge "xenapi: Removing legacy swap-in-image." 2012-10-01 21:56:20 +00:00
Brian Lamar
84a9af7966 Update RPM SPEC to include new bandwidth plugin.
RPM builds will fail until this line is added to include
the new plugin.

Fixes bug 1059772

Change-Id: I06d78cf199d4cb580003e4c220af3efc8dbe437c
2012-10-01 14:26:19 -04:00
Jenkins
2377d6a7b6 Merge "Fix XenServer's ability to boot xen type images" 2012-09-30 07:35:54 +00:00