12424 Commits

Author SHA1 Message Date
Chris Behrens
ea4a4cda45 Sync nova with oslo DB exception cleanup.
DB exceptions have moved to openstack/common/db/exception module so that
they can be shared with multiple DB implementations.

Deadlock checking was also added to oslo to consolidate with
DuplicateKey checking.  This allows us to clean up our
_retry_on_deadlock decorator in sqlalchemy/api.py

Fixes unrelated pep8 issue with duplicate test in test_compute also.

Change-Id: I7e985b384d1ef345e0d67c919b84b4faff869699
2013-02-22 16:37:28 +00:00
Devananda van der Veen
710c0d8969 Fix exception handling in baremetal API.
In patch ab9f8667c63d901f37d1662c5204fb2938be44fe, several
baremetal/db/api methods' exceptions were changed from InstanceNotFound
to NodeNotFound. The API extension for baremetal was not updated to
catch these, and this was not caught by unit testing. This resulted in
unhandled exceptions within nova-api if any baremetal node lacked an
associated interface.

While fixing that bug, a few other unit tests for the baremetal API
were added, and a missing exception was added to db/api bm_node_destroy.

Fixes bug 1131430.

Change-Id: I15f7624723754f9d7b217b609663a2d709acb056
2013-02-21 13:56:25 -08:00
Rick Harris
25a3558e0b BM Migrations 2 & 3: Fix drop_column statements
`drop_column` takes a string not a `Column` object.

Change-Id: I01d537f78e40a20b762439087a13e104b9d18d46
2013-02-21 23:12:38 +00:00
Stanislaw Pitucha
196c757748 Remove function redefinitions
A number of tests have never been run due to another function having
the same name. Duplicate names have been changed and additionally
pyflakes check will now fail in such situations.

Also fix ovs ethernet driver test which was incorrect.

Change-Id: Icd93528395e65a24194a6ec5dfb8025a820d29b5
2013-02-21 20:01:48 +00:00
Chris Behrens
bbab1acabb Move some context checking code from sqlalchemy
Move some context related code from sqlalchemy into nova/context.py
where it can be used outside of sqlalchemy.

Change-Id: I6522a072132e27b42561435cb4fd671a2ece4867
2013-02-19 00:20:39 +00:00
Devananda van der Veen
a2e66182a7 Baremetal driver returns accurate list of instance
Add 'instance_name' to bm_nodes table so that baremetal driver is able
to return the names of all instances it believes are still running.

Previously, baremetal.driver.list_instances was fetching all allocated
instances from baremetal database, then calling VirtAPI to get the
instance name. This would raise an InstanceNotFound exception for
deleted instances. This prevented ComputeManager from ever detecting
a running-but-deleted baremetal instance, and could leave baremetal
instances in an undeletable state.

Fixes bug 1096723.

Change-Id: Ifae532e8e70e97e48c589608cb3c7000bb6a7609
2013-02-10 12:49:53 -08:00
Devananda van der Veen
0e91860e80 Identify baremetal nodes by UUID.
- add a new 'uuid' column to 'bm_nodes' table
- provide a new db/api method for accessing nodes by their uuid
- return this to Nova as the nodename / hypervisor_hostname

In this way, a baremetal node is uniquely identified to the user,
whether they use a per-compute baremetal db or global baremetal db.

It will also allow for an instance to be booted on a specific baremetal
node using a means akin to the current force-hosts (TBD in later patch).

Also, creates two new Exceptions to disambiguate between
InstanceNotFound and NodeNotFound / NodeNotFoundByUUID.

Change-Id: I81105a201588fdef31cffabdae260bb43017bcd1
2013-02-13 18:25:29 -08:00
Devananda van der Veen
9d4c933825 Improve performance of baremetal list_instances.
This patch adds two new methods to baremetal/db/api:
 - bm_node_get_associated
 - bm_node_get_unassociated
which return a list of nodes either associated or not associated to a
Nova instance, as determined by the instance_uuid field, respectively.

Now, list_instances and get_available_nodes methods only fetch assocated
and unassociated nodes from the baremetal database, instead of fetching
all nodes.

Change-Id: Ib875d26d7b85a9dbf08c925a89802e2cd614e363
2013-02-09 11:51:20 -08:00
Devananda van der Veen
ed47731e12 Better error handling in baremetal spawn & destroy
Improve the exception handling in baremetal driver spawn and destroy.

Adds unit tests for destroy to check that bm_node is disassociated
from instance_uuid only when destroy succeeds, and is set to ERROR state
(but still associated) if any exception is not handled.

Change-Id: I75853210d4883b352da9876a682dbb7288689541
2013-02-09 10:35:16 -08:00
Devananda van der Veen
9f514596c2 Wait for baremetal deploy inside driver.spawn
Previously, baremetal driver.spawn returned as soon as the
machine power turned on, but before the user-image was deployed to the
hardware node, and long before the node was available on the network.
This meant the nova instance was marked as ACTIVE before provisioning
had actually finished. If the deploy failed and the baremetal node was
set to an ERROR state, the nova instance could still be left as ACTIVE
and the user was never informed of the error.

This patch introduces a LoopingCall to monitor the deployment status in
the baremetal database. As the deployment is performed by
nova-baremetal-deploy-helper, the database record is updated. Once the
deployment is complete, driver.spawn() sets the baremetal node status
and the nova instance status is also set properly. If an error occurs
during the deployment, an exception is raised within driver.spawn()
allowing nova to follow the normal cleanup and notify paths.

This also allows the baremetal PXE driver to delete cached image files
when a baremetal deployment fails.

Fixes bug 1088655.

Change-Id: I4feefd462fd956c9780995ec8b05b13e78278c8b
2013-02-08 20:36:19 -08:00
Devananda van der Veen
0214e43479 Add better status to baremetal deployments.
This patch introduces a few new baremetal states, which are used to
track the deploy process. Now, nova-baremetal-deploy-helper updates the
bm_nodes record directly when it begins and finishes deploying an image
to that node.

The next patch will add a LoopingCall inside driver.spawn() to wait for
the deploy to complete.

Also, since there can not be >1 active deployment per node, there
is no need to have a separate table for storing them. This patch drops
the table bm_deployments and adds the important information it contained
to bm_nodes. Since the previous behavior was to mark a deployment as
deleted once it completed, there is no need to copy any data from
bm_deployments prior to dropping the table -- assuming that no active
deployments are in process when the migration is run.

Since this is the first migration for the baremetal database, it also
adds a new test class, TestBaremetalMigrations, and refactors the
test_migrations.py file to allow for multiple test classes.

partially implements fix for bug 1096723

Change-Id: Iad30b462d49c88fc19babed43a2fb8540b1fad30
2013-02-19 12:39:55 -08:00
Mark McLoughlin
af276eb0b2 Use oslo-config-2013.1b4
The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.

Add the 2013.1b4 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.

Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.

Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.

Make tools/hacking.py include oslo in IMPORT_EXCEPTIONS like it already
does for paste. It turns out imp.find_module() doesn't correct handle
namespace packages.

Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).

Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
2013-02-15 22:30:16 +00:00
Arata Notsu
b2d77bcf1d Delete baremetal interfaces when their parent node is deleted
Change-Id: I504cbed0adea86cab840c40ea9b9f96faf299f73
2013-02-15 14:45:28 +09:00
Devananda van der Veen
ee76aba082 VirtualPowerDriver catches ProcessExecutionError.
Baremetal virtual_power_driver now catches ProcessExecutionError coming
from ssh_execute and logs an exception. Requested actions (eg, power_on)
return an error state instead of raising an exception.

Change-Id: I8f66ba843e12de13f9a14d736f7bef2eda4ef85c
2013-02-12 15:08:38 -08:00
Devananda van der Veen
380bca8ad5 Don't modify injected_files inside PXE driver
Make a local copy of injected_files so that we don't modify the
original.

Change-Id: I508cfc0baec15a091a1717eb2aebb9dd39278e5b
2013-02-12 18:16:30 -08:00
Devananda van der Veen
bfb4b8a5de Remove nova.db call from baremetal PXE driver
Baremetal PXE driver was fetching instance_type from the nova db.
Now that no-db-compute is done, that's not possible. Also, there's a
more efficient way -- just call extract_instance_type(instance)!

Baremetal unit tests should check for instance_types with swap = 0 and
with swap > 0. This required changing test/utils to include a
real copy of instance_types in instance['system_metadata'].

Change-Id: I2a91e5c026782946f6b01e4189ec85d30ba87583
2013-02-07 16:42:29 -08:00
Chris Krelle
09de325519 Add a virtual PowerDriver for Baremetal testing
This virtual_power_driver provides the ability for baremetal driver to
start/stop/reboot VMs within common dev/test environments. Its goal is to
approximate the same functionality as other PowerDrivers (eg. IPMI) within
a desktop development environments and within the devstack-gate environment.

Authored-by: Chris Krelle <nobodycam@gmail.com>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>

Change-Id: I9935a9a30a166a9c28a9abb139c49f219a32ddd7
2013-01-16 18:40:21 -08:00
Rafi Khardalian
d93b8109ae Recache or rebuild missing images on hard_reboot
The primary purpose of this change is to provide the ability to
re-cache missing backing files on hard_reboot. The old
pre_block_migration function was already performing a very similar
operation. That function has been refactored to be idempotent and
renamed to _create_images_and_backing. The pre_block_migration
function is a wrapper, with some additional checking, around the
renamed function.

Image backend was also adjusted to look for either a missing backing
file or disk image, recaching or creating accordingly. It should
also be idempotent, never clobbering existing images.

Change-Id: Icf4c488d6db59e732b463d08d0606b428ee1e7b9
2013-02-05 09:40:15 +00:00
Eric Windisch
3d07dfe4e4 Use oslo database code
Bring in the new database code from oslo.

Uses get_session() from oslo as well as changing NovaBase to derive from
a common class.

Remove test_sqlalchemy.py now that this code is test in oslo.

Implements blueprint db-common.

Change-Id: I090754981c871250dd981cbbe1a08e7181440120
2013-01-17 15:38:02 -05:00
Zhongyue Luo
c8386dbefe Fixes 'not in' operator usage
Change-Id: I1e26a8fcb9fa564308e63c11a72aaa55119e4eee
2013-01-31 13:56:42 +08:00
Joe Gordon
1017d78950 Make sure there are no unused import
Remove all currently unused imports
Prevent future unused imports

Change-Id: I6ac26d5c71b79952a7732db300355a00310c712e
2013-01-25 20:34:03 +00:00
Joe Gordon
e515a666f3 Enable N302: Import modules only
Fix all N302 issues, and re-enable.

Change-Id: Ic94d144c915b228b7ff2fd9c5951875e159ffcdd
2013-01-25 12:10:16 -05:00
Arata Notsu
c511d4dbb5 Correct a format string in virt/baremetal/ipmi.py
"%(err)%s" to "%(err)s"

Change-Id: I5ef1d3218c6121fed7c5d2b4d318da7c6d884776
2013-01-23 20:48:19 +09:00
Arata Notsu
1396d2ffbc Add REST api to manage bare-metal nodes
* create/delete/list/show bare-metal nodes
* add/remove interfaces to/from bare-metal nodes

blueprint general-bare-metal-provisioning-framework

Change-Id: I1e76f7e3f7f74087e844cfb23dc92154f4c3e127
2013-01-23 17:32:46 +09:00
Devananda van der Veen
fd6120c877 Baremetal/utils should not log certain exceptions
unlink_without_raise was logging exceptions when it failed to unlink a
file, which is confusing since it is often called on non-existing files.

create_link_without_raise was also logging exceptions when it failed
to create a symlink.

This patch corrects this behaviour; both functions now explicitly check
for the type of error they expect, and then suppress that. If another
type of OSError is encountered, a warning is logged. This patch also
adds unit tests for both functions.

fixes bug 1097931.

Change-Id: Ie36f59dce34a6c67765770c5f968c003003acc88
2013-01-17 13:30:58 -08:00
Devananda van der Veen
1c9111eaa2 PXE driver should rmtree directories it created
Baremetal PXE driver was failing to delete the per-instance tftpboot and
image directories which it created when the instance was deleted. This
happened partly because of dangling files within the directory, and
partly because 'unlink' does not remove directories and the error was
squelched.

Now, when destroy()ing an instance, PXE driver will call shutil.rmtree()
on the per-instance directories which it created.

Fixes bug 1101048.

Change-Id: I682d0264288add488ea23e5d5200357b7be52dd9
2013-01-18 09:40:37 -08:00
Davanum Srinivas
84d8b43626 Add support for Option Groups in LazyPluggable
Use @markmc's suggestion to enhance LazyPluggable with an
optional config group. Also fix the baremetal database
backend option to use the "baremetal" config group.

Fixes LP #1093043

Change-Id: I28cf51a2962f516fcef4ced19e30c985220e86dc
2013-01-21 21:18:34 -05:00
Robert Collins
5b04310d99 Remove obsolete baremetal override of MAC addresses.
Now that the hypervisor driver can specify what MAC addresses it
needs, overriding the MAC address during bare metal instance
provisioning is no longer appropriate.

Change-Id: I2b0790753297104ddd40a410c8acb5fdac97ad15
2013-01-16 13:20:47 +13:00
Devananda van der Veen
40177796d1 PXE driver should not accept empty kernel UUID.
Baremetal PXE driver was aborting only if the kernel and ramdisk
specified in image metadata were None, but not if they were another
non-true value, such as "". Now, exception is raised if they are any
non-true value.

Fix bug 1100589.

Change-Id: I9783a14cc242e1b9db04d797c61d54b06d52c680
2013-01-17 09:09:32 -08:00
Matt Joyce
8f09391466 Correcting improper use of the word 'an'.
Change-Id: I032509539c61f2f5fda2001e2db5a3298e30c561
2013-01-16 11:41:37 -08:00
Robert Collins
7b7c28f939 Export the MAC addresses of nodes for bare-metal.
The bare-metal hypervisor needs to let the network layer know the MAC
address of each node it has, or TFTP boot will fail, as the MAC
addresses quantum / nova dynamically allocate will not match the
actual MAC of the node as it boots. This change exports the MAC
addresses to the manager, which passes them onto the network driver in
use.

With this change administrators should register all the MAC addresses
of a given node with Nova bare-metal as interfaces, even though that
may duplicate the MAC for PXE boot provisioning. Long term the
dedicated provisioning MAC address will be removed.

Change-Id: I55f6031294a2c5d31975462f868aa27441e11ad2
2013-01-15 14:16:57 +13:00
Robert Collins
2448914cfb Break out a helper function for working with bare metal nodes.
This makes it easier to get a bare metal internal node id when called
from the manager.

Change-Id: I1b39a7be5bd21d5b3060f11a74b9eaf79b107210
2013-01-15 13:44:21 +13:00
Sandy Walsh
47af92f3c0 Keep self and context out of error notification payload.
Back in the day, having self and context in the error notifications
was handy for debugging. Now, there is a lot of confidential stuff
stored in these objects (especially when self = ComputeManager) ...
like passwords, etc.

This patch strips it out.

Also removes dead wrap_exception calls (which did nothing since they
did not specify a notifier).

Change-Id: Ieab7bd79b64e01c7bca18dbce97455e50094871c
2013-01-08 16:20:12 -06:00
Mikyung Kang
fefe7d99ae Tests for PXE bare-metal provisioning helper server.
a part of blueprint general-bare-metal-provisioning-framework.

The deploy helper was landed without tests. This patch reinstates the
tests.

Change-Id: I690ec077e175dc37c2cc5f18170513b0e5bca103
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>
2012-11-10 08:07:13 +09:00
Robert Collins
7e683ab992 Change ComputerDriver.legacy_nwinfo to raise by default.
This makes non-updated hypervisors visible by grepping for 'def
legacy_nwinfo' rather than only the updated ones being visible, and
when new hypervisors are added, it will be clear whether they use the
legacy format or not.

Out of tree hypervisors will be broken by this, but the fix is
trivial::

def legacy_nwinfo(self):
    # XXX TODO Update to use the non-legacy format.
    return True

Change-Id: If5b461bc5d8e8dc21de3ca9cf521e7b341724900
2013-01-11 10:20:43 +13:00
Sean Dague
6d7fff01d9 fix new N402 errors
fix the N402 errors that have slipped in in the last 48 hrs since
starting this patch series.

fix an N401 error that our scanner current can't find because it
only looks for doc strings on classes and defs.

this is the xeno's paradox of patch series, but we're getting close.

Change-Id: I4a763bb4c812335d853eae05c72464f18ab93297
2013-01-09 16:00:36 -05:00
Mark McLoughlin
69f6994ecc Remove unused baremetal PXE options
These options were introduced by commit a2f260f and appear to never to
have been used.

Change-Id: Idf1387fd453b7b1c392db764934c148dcbfc5d66
2013-01-09 10:43:50 +00:00
Mark McLoughlin
8e21f48b01 Move global service networking opts to new module
The my_ip, host and use_ipv6 options are used all over the codebase
and they're pretty well related to each other. Create a new netconf
module for them to live in.

There are now no options registered globally in nova.config!

blueprint: scope-config-opts
Change-Id: Ifde37839ae6f38e6bf99dff1e80b8e25fd68ed25
2013-01-08 09:48:03 +00:00
Sean Dague
e0761c2f52 Fix N402 for nova/virt
Fix N402 errors (single line docstring should end in a period)
for nova/virt, part of a larger attempt to stop ignoring our own
hacking.py tests.

Change-Id: I523ce41bd2b38c73cf3fdb031101ccc0695f2488
2013-01-08 16:25:23 -05:00
Robert Collins
c09985122b Cope better with out of sync bm data.
The baremetal hypervisor tracks references to instances which are
deleted asynchronously from the commit to the nova bm. As such, its a
normal but rare condition for a bare metal node to refer to a deleted
(and even garbage collected) nova instance. We should treat such
instances are deleted rather than erroring on any call through
list_instances(), permitting starting new instances and listing
instances to work - fixing bug: 1096722.

Pathologically, the database may suffer permanent skew, which means we
need a way to fix it (and thats tracked as a separate bug).

Change-Id: Ic21ff66b1fc0ad64bb5feff26291873b96d20e4e
2013-01-07 15:17:42 +13:00
Devananda van der Veen
2032c91507 Fix baremetal VIFDriver
Commit Id287f7e661 removed the nova/virt/vif.py file and the VIFDriver
base class, and erroneously stated in its commit message that baremetal
driver did not use it. This patch repeats the work of that earlier patch
by removing the import of nova.virt.vif from baremetal/vif_driver.py.

blueprint general-bare-metal-provisioning-framework

Change-Id: Ifac9510a72a028dc5d3574148f3c4a6410b1d249
2013-01-08 09:39:37 -08:00
Mikyung Kang
af631b6a46 CLI for bare-metal database sync.
Part 3 of 6: blueprint general-bare-metal-provisioning-framework.

Change-Id: Ia19ce00edb84aa924c2ab2c9c2217f6b49073d69
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
2013-01-08 22:56:14 +09:00
Arata Notsu
2f772ae031 attach/detach_volume() take instance as a parameter
Previously the methods take instance['name'] as a parameter.
With this change, ComputeDriver can lookup informations about
the instance from hypervisor (in bare-metal driver, it is a local DB)
by any other attributes of the instance.

blueprint general-bare-metal-provisioning-framework

Change-Id: Ibd0567f34ed5053909ce1a408f9cbf87516ba597
2012-12-20 21:29:13 +09:00
Devananda van der Veen
ff7e294721 Convert short doc strings to be on one line
This patch just cleans up some one-line doc strings which were
split across 3 lines needlessly.

Change-Id: I3c2011d20fb7c3b8714d7b5a32151e427bf8a311
2013-01-03 11:27:22 -08:00
Arata Notsu
d96094b5a0 Check admin context in bm_interface_get_all()
This patch puts missing @require_admin_context before
the function.

blueprint general-bare-metal-provisioning-framework

Change-Id: I07d78f75b7f98a225398ca67b3bda251bc6c0440
2013-01-07 17:53:23 +09:00
Devananda van der Veen
5327259765 Provide a PXE NodeDriver for the Baremetal driver
This patch implements a PXE NodeDriver class within the Baremetal
provisioning framework, which provides a means for deploying machine
images using TFTP and PXE.

This patch relies on functionality provided by the nova-baremetal-deploy-helper
utility, implemented in review 15830.

blueprint general-bare-metal-provisioning-framework.

Change-Id: I8d849601186e3dc13f10382857ff2bbc1ff1026d
2012-12-27 17:23:29 -08:00
Michael Still
335f0f2eab Refactor periodic tasks.
This review allows periodic tasks to be enabled or disabled in the
decorator, as well as by specifying an interval which is negative.

The spacing between runs of a periodic task is now specified in
seconds, with zero meaning the default spacing which is currently 60
seconds.

There is also a new argument to the decorator which indicates if a
periodic task _needs_ to be run in the nova-compute process. There is
also a flag (run_external_periodic_tasks) which can be used to move
these periodic tasks out of the nova-compute process.

I also remove the periodic_interval flag to services, as the interval
between runs is now dynamic based on the number of seconds that a
periodic task wants to wait for its next run. For callers who want to
twiddle the sleep period (for example unit tests), there is a
create() argument periodic_interval_max which lets the period
periodic_tasks() specifies be overridden. This is not exposed as a
flag because I cannot see a use case for that. It is needed for unit
testing however.

DocImpact. Resolves bug 939087.

Change-Id: I7f245a88b8d229a481c1b65a4c0f1e2769bf3901
2012-12-24 15:00:52 +11:00
Mark McLoughlin
3cff674671 Add helper methods to nova.paths
We only ever do one of two things with the global path options:

  1) Reference the option in the default of another option so that the
     value can be interpolated it

  2) Use the value of the option to build a path

Add helper methods for both these cases - e.g. basedir_def() for the
former case and basedir_rel() for the latter case. This makes it much
more obvious how and where these options are used.

Change-Id: I7fd94a329fe911761d02d94e5381e950c6668d56
2013-01-04 17:36:29 +00:00
Mark McLoughlin
cf0363115a Move global path opts in nova.paths
Move the global path config options (i.e. state_path, pybasedir and
bindir) into a new nova.paths module. A new module may seem like
overkill but some utility methods associated with these options follow
in a later commit.

Moving them to nova.paths means they are no longer globally defined
and it's more obvious which modules require these options.

Change-Id: I381d23f1bbe36dc6967a38a65062b0983e1661aa
2013-01-04 17:32:36 +00:00
Zhongyue Luo
b144e6d1d7 Removes unused imports
Change-Id: I76d243329f810cdcd6f8a5459eb66fddd5871e33
2013-01-03 18:21:36 +08:00