11394 Commits

Author SHA1 Message Date
Arata Notsu
8e05dbfb33 Define LOG globally in baremetal_deploy_helper
Since LOG = getLogger() has been moved from the global scope
to the function scope (main), the rest of the module could
not find LOG. This patch makes LOG global again.

Fixes bug 1165262

Change-Id: I2607802dcbed64135f75011a02ebce95519e4ea1
2013-04-06 09:57:33 +09:00
Davanum Srinivas
832fca9615 Only call getLogger after configuring logging.
Logging needs to be setup properly before we try to log
something. Fix a test case that loads the code but does
not execute the code that results in the LOG being setup
properly.

Fixes LP# 1161031

Change-Id: I9d5bc6b87cfb25243f00e17b532d4485dc4454e4
2013-04-04 17:33:20 -04:00
Arata Notsu
f5d9ded9c4 baremetal: Integrate provisioning and non-provisioning interfaces
Originally, baremetal pxe/tilera driver managed two types of network
interfaces in two tables; provisioning interfaces in bm_nodes table
and non-provisioning (normal) interfaces in bm_interfaces table.
But, now actually both types are handled in the same way and there is
no difference between them except for which table they are in.

This patch moves the provisioning interfaces to bm_interfaces. However
it does not drop prov_mac_address in bm_nodes since VirtualPowerManager
still uses it to identify a target in a list of VMs returned by the
VM's host.

Change-Id: I5ce940e7127aa3b29ba7802612938dc5dbc3152f
2013-04-04 19:08:49 +09:00
Monty Taylor
c2e75cad16 Move console scripts to entrypoints.
As part of the move of plugins to entrypoints, take advantage of the
entrypoints based console scripts, which will make our command line scripts
available for unittesting.

Part of blueprint entrypoints-plugins

Co-authored-by: Michael Still <mikal@stillhq.com>

Change-Id: I5f17348b7b3cc896c92263dd518abb128757d81f
2012-08-15 15:02:51 -04:00
Arata Notsu
2c2741a44e baremetal: Drop unused columns in bm_nodes
Drop these two columns:
* bm_nodes.prov_vlan_id
* bm_nodes.registration_status

And update related code, tests and api_samples.

Change-Id: Iadefc83a00ad9ae5a3bb39f357080299875bdf09
2013-02-27 03:30:36 +09:00
Arata Notsu
aa48766bb0 Remove print statements
Change-Id: I40a805bee3f10d5989188971423cbb6c07f740c8
2013-04-02 10:45:45 +09:00
Arata Notsu
99d7b91862 Delete tests.baremetal.util.new_bm_deployment()
The function is not used anymore. It creates an instance of
BareMetalDeployment, which was removed in c20110d1.

Change-Id: I40b634f8cf0d4369f09cb61aca6f95a059e85d6d
2013-04-01 20:20:03 +09:00
Mikyung Kang
a27b7f99fe Adds Tilera back-end for baremetal
blueprint add-tilera-to-baremetal.

The baremetal driver is a hypervisor driver for Openstack Nova
Compute. Within the Openstack framework, it has the same role
as the drivers for other hypervisors (libvirt, xen, etc). With
this patch set of tilera-backend, provisioning and management
of non-PXE tilera physical hardware is accomplished using
common cloud APIs and tools.

Change-Id: I356c06a07ace463779e3b84836d5000331e24814
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>
2013-03-26 17:58:04 -04:00
Devananda van der Veen
123826e516 Change type of ssh_port option from Str to Int
The type of CONF option virtual_power_ssh_port was incorrectly defaulted
to Str. This can cause Paramiko to raise when casting to %d.

Fixes bug 1157824.

Change-Id: I30ddd1ff0da45f8392085249f1bd2a539b201a7e
2013-03-20 09:19:44 -07:00
Chris Krelle
7bfbc19021 Virtual Power Driver list running vms quoting error
This patch corrects a issue with VPD when configured for virsh the
list running vms function attempts to execute $2 instead of the actual
command.

Fixes Bug 1154280

Change-Id: I6d06cc58fb0f2935c4341761908b914106bda916
Authored-by: Chris Krelle <nobodycam@gmail.com>
2013-03-12 12:41:12 -07:00
Rick Harris
ed3251715c xenapi: Fix reboot with hung volumes
If a volume becomes inoperable (e.g. the ISCSI connection is severed)
and the user goes to reboot, the instance may enter a permanently halted
state.

The root cause is that a VBD that points to 'bad' volume prevents VM
operations ('reboot', 'start') from completing under XenServer.

The work around is to detect which volumes are bad, detach in the
virt-layer, retry the operation (or in the case of reboot, just 'start'
the halted instance), and then notify the compute manager via a
callback so it can detach the volume in Cinder.

Fixes bug 1148614

Change-Id: Id4e8e84bb5748cfa267c2a418f9405fd86829e8f
2013-03-06 05:28:41 +00:00
Boris Pavlovic
fb457c820e Make bm model's deleted column match database
In Models by default type of deleted column is Integer, but bm tabels
use Boolean type for deleted columns.This produce bug in postgresql
(type mismatch).

At this moment we are not able to do any db migrations, so there is
only one way to fix this bug. Change type of deleted columns in Models
from Integer to Boolean.

In Havana we will be able to change deleted columns types to type of id
for bm tables and use soft_delete and UC as in rest of nova.

Fixes bug 1140294

Change-Id: Id56c4bbff7048ed5c746dc995380be9ba0a12c39
2013-03-08 02:12:22 +04:00
Chris Krelle
c05474c43b Correct substring matching of baremetal VPD node names
This path corrects a issue where is_power_on function can incorrectly match a
node if its name is a substring of another node.

I have also added a test for this.

fixes bug 1152676
Change-Id: Ic3b0b952d53dcd4464098f98879a84684efe22b2
Authored-by: Chris Krelle <nobodycam@gmail.com>
2013-03-08 09:22:19 -08:00
Devananda van der Veen
5dd679aad2 Read baremetal images from extra_specs namespace.
Baremetal PXE driver should read deploy_kernel_id & deploy_ramdisk_id
from the 'baremetal:' namespace within instance_type['extra_specs']
so that it doesn't conflict with ComputeCapabilitiesFilter any more.

Fixes bug 1129485.

Change-Id: I84b3acb2ed83dc2b1ff8f1a21ca1d95f7d25751a
2013-03-04 11:03:40 -08:00
Devananda van der Veen
049312e2c0 Compute manager should remove dead resources
While most hypervisors return a single - and constant - value from
driver.get_available_nodes, baremetal does not. When a node is deleted
from the baremetal database, it is no longer returned from
driver.get_available_nodes. However, Nova's compute_node record is not
directly updated.

This patch allows Compute Manager to detect missing nodes within
update_available_resources. It then invokes resource_tracker to update
the dead node and remove it from compute.

This in turn allows the ServiceGroup API to properly update the
servicegroup when a baremetal node is no longer in service.

Fixes bug 1138184

Change-Id: Icfff3f8e3099668806633a6a58a152b32ec8b49b
2013-03-01 14:05:35 -08:00
Chris Krelle
1a25d3d5ac Add ssh port and key based auth to VPD.
This patch adds the ablity to set ssh port, and use key based
authentication for VPD.

This patch adds config options:
    virtual_power_ssh_port: port to ssh to on virtual power host
    virtual_power_host_key: private key file for ssh auth

Note: Key file must be able to be read by user nova is running as.

      This patch switches the default virtual_power_type from 'vbox'
      to 'virsh' to match the most common usage.

Fix Bug 1136010

Change-Id: Id89878d63d26602bb3eacc2a2ce7902d9bd33d61
Authored-by: Chris Krelle <nobodycam@gmail.com>
2013-02-28 08:11:54 -08:00
Dan Smith
a42d83d2be Add instance_type_get() to virt api
...and remove the use of instance['extra_specs'] from the libvirt and
baremetal virt drivers. Also remove the hack in instance_update()
which places them there in the first place.

Fixes bug 1133572

Change-Id: I39e9fabb28b48dc52ec47f58d76b0bf2c6ee0204
2013-02-27 14:50:50 -05:00
Rick Harris
3246f3051d Don't blindly skip first migration.
The existing code would, no matter-what, skip the first migration when
running through the migrations tests. This makes the code
slightly counterintuitive (why skip the first one?), and is not very
flexible.

The reworked logic lets a migration raise a `NotImplementedError` to
signal that it doesn't support a `downgrade`. This lets the
walk-versions code treat the first-migration just like any other.

Change-Id: I19fd8ebc6075392cf9e573b8b4166291584b4a83
2013-02-28 00:24:22 +00:00
Rick Harris
cfae6726ab BM Migration 004: Actually drop column
SQLAlchemy-migrate appears to support passing in a Column or a
column-name string into `drop_column`. In practice though, only the
column-name form actually works.

Change-Id: I1bcc28511d652df44f7168fb84c8be7dacd60cfd
2013-02-27 23:04:40 +00:00
Kurt Taylor
7e3b9f0c26 Update OpenStack LLC to Foundation
Update all references of "LLC" to "Foundation".

Change-Id: I009e86784ef4dcf38882d64b0eff484576e04efe
2013-02-22 09:13:07 -05:00
Chris Behrens
ea4a4cda45 Sync nova with oslo DB exception cleanup.
DB exceptions have moved to openstack/common/db/exception module so that
they can be shared with multiple DB implementations.

Deadlock checking was also added to oslo to consolidate with
DuplicateKey checking.  This allows us to clean up our
_retry_on_deadlock decorator in sqlalchemy/api.py

Fixes unrelated pep8 issue with duplicate test in test_compute also.

Change-Id: I7e985b384d1ef345e0d67c919b84b4faff869699
2013-02-22 16:37:28 +00:00
Devananda van der Veen
710c0d8969 Fix exception handling in baremetal API.
In patch ab9f8667c63d901f37d1662c5204fb2938be44fe, several
baremetal/db/api methods' exceptions were changed from InstanceNotFound
to NodeNotFound. The API extension for baremetal was not updated to
catch these, and this was not caught by unit testing. This resulted in
unhandled exceptions within nova-api if any baremetal node lacked an
associated interface.

While fixing that bug, a few other unit tests for the baremetal API
were added, and a missing exception was added to db/api bm_node_destroy.

Fixes bug 1131430.

Change-Id: I15f7624723754f9d7b217b609663a2d709acb056
2013-02-21 13:56:25 -08:00
Rick Harris
25a3558e0b BM Migrations 2 & 3: Fix drop_column statements
`drop_column` takes a string not a `Column` object.

Change-Id: I01d537f78e40a20b762439087a13e104b9d18d46
2013-02-21 23:12:38 +00:00
Stanislaw Pitucha
196c757748 Remove function redefinitions
A number of tests have never been run due to another function having
the same name. Duplicate names have been changed and additionally
pyflakes check will now fail in such situations.

Also fix ovs ethernet driver test which was incorrect.

Change-Id: Icd93528395e65a24194a6ec5dfb8025a820d29b5
2013-02-21 20:01:48 +00:00
Chris Behrens
bbab1acabb Move some context checking code from sqlalchemy
Move some context related code from sqlalchemy into nova/context.py
where it can be used outside of sqlalchemy.

Change-Id: I6522a072132e27b42561435cb4fd671a2ece4867
2013-02-19 00:20:39 +00:00
Devananda van der Veen
a2e66182a7 Baremetal driver returns accurate list of instance
Add 'instance_name' to bm_nodes table so that baremetal driver is able
to return the names of all instances it believes are still running.

Previously, baremetal.driver.list_instances was fetching all allocated
instances from baremetal database, then calling VirtAPI to get the
instance name. This would raise an InstanceNotFound exception for
deleted instances. This prevented ComputeManager from ever detecting
a running-but-deleted baremetal instance, and could leave baremetal
instances in an undeletable state.

Fixes bug 1096723.

Change-Id: Ifae532e8e70e97e48c589608cb3c7000bb6a7609
2013-02-10 12:49:53 -08:00
Devananda van der Veen
0e91860e80 Identify baremetal nodes by UUID.
- add a new 'uuid' column to 'bm_nodes' table
- provide a new db/api method for accessing nodes by their uuid
- return this to Nova as the nodename / hypervisor_hostname

In this way, a baremetal node is uniquely identified to the user,
whether they use a per-compute baremetal db or global baremetal db.

It will also allow for an instance to be booted on a specific baremetal
node using a means akin to the current force-hosts (TBD in later patch).

Also, creates two new Exceptions to disambiguate between
InstanceNotFound and NodeNotFound / NodeNotFoundByUUID.

Change-Id: I81105a201588fdef31cffabdae260bb43017bcd1
2013-02-13 18:25:29 -08:00
Devananda van der Veen
9d4c933825 Improve performance of baremetal list_instances.
This patch adds two new methods to baremetal/db/api:
 - bm_node_get_associated
 - bm_node_get_unassociated
which return a list of nodes either associated or not associated to a
Nova instance, as determined by the instance_uuid field, respectively.

Now, list_instances and get_available_nodes methods only fetch assocated
and unassociated nodes from the baremetal database, instead of fetching
all nodes.

Change-Id: Ib875d26d7b85a9dbf08c925a89802e2cd614e363
2013-02-09 11:51:20 -08:00
Devananda van der Veen
ed47731e12 Better error handling in baremetal spawn & destroy
Improve the exception handling in baremetal driver spawn and destroy.

Adds unit tests for destroy to check that bm_node is disassociated
from instance_uuid only when destroy succeeds, and is set to ERROR state
(but still associated) if any exception is not handled.

Change-Id: I75853210d4883b352da9876a682dbb7288689541
2013-02-09 10:35:16 -08:00
Devananda van der Veen
9f514596c2 Wait for baremetal deploy inside driver.spawn
Previously, baremetal driver.spawn returned as soon as the
machine power turned on, but before the user-image was deployed to the
hardware node, and long before the node was available on the network.
This meant the nova instance was marked as ACTIVE before provisioning
had actually finished. If the deploy failed and the baremetal node was
set to an ERROR state, the nova instance could still be left as ACTIVE
and the user was never informed of the error.

This patch introduces a LoopingCall to monitor the deployment status in
the baremetal database. As the deployment is performed by
nova-baremetal-deploy-helper, the database record is updated. Once the
deployment is complete, driver.spawn() sets the baremetal node status
and the nova instance status is also set properly. If an error occurs
during the deployment, an exception is raised within driver.spawn()
allowing nova to follow the normal cleanup and notify paths.

This also allows the baremetal PXE driver to delete cached image files
when a baremetal deployment fails.

Fixes bug 1088655.

Change-Id: I4feefd462fd956c9780995ec8b05b13e78278c8b
2013-02-08 20:36:19 -08:00
Devananda van der Veen
0214e43479 Add better status to baremetal deployments.
This patch introduces a few new baremetal states, which are used to
track the deploy process. Now, nova-baremetal-deploy-helper updates the
bm_nodes record directly when it begins and finishes deploying an image
to that node.

The next patch will add a LoopingCall inside driver.spawn() to wait for
the deploy to complete.

Also, since there can not be >1 active deployment per node, there
is no need to have a separate table for storing them. This patch drops
the table bm_deployments and adds the important information it contained
to bm_nodes. Since the previous behavior was to mark a deployment as
deleted once it completed, there is no need to copy any data from
bm_deployments prior to dropping the table -- assuming that no active
deployments are in process when the migration is run.

Since this is the first migration for the baremetal database, it also
adds a new test class, TestBaremetalMigrations, and refactors the
test_migrations.py file to allow for multiple test classes.

partially implements fix for bug 1096723

Change-Id: Iad30b462d49c88fc19babed43a2fb8540b1fad30
2013-02-19 12:39:55 -08:00
Mark McLoughlin
af276eb0b2 Use oslo-config-2013.1b4
The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.

Add the 2013.1b4 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.

Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.

Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.

Make tools/hacking.py include oslo in IMPORT_EXCEPTIONS like it already
does for paste. It turns out imp.find_module() doesn't correct handle
namespace packages.

Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).

Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
2013-02-15 22:30:16 +00:00
Arata Notsu
b2d77bcf1d Delete baremetal interfaces when their parent node is deleted
Change-Id: I504cbed0adea86cab840c40ea9b9f96faf299f73
2013-02-15 14:45:28 +09:00
Devananda van der Veen
ee76aba082 VirtualPowerDriver catches ProcessExecutionError.
Baremetal virtual_power_driver now catches ProcessExecutionError coming
from ssh_execute and logs an exception. Requested actions (eg, power_on)
return an error state instead of raising an exception.

Change-Id: I8f66ba843e12de13f9a14d736f7bef2eda4ef85c
2013-02-12 15:08:38 -08:00
Devananda van der Veen
380bca8ad5 Don't modify injected_files inside PXE driver
Make a local copy of injected_files so that we don't modify the
original.

Change-Id: I508cfc0baec15a091a1717eb2aebb9dd39278e5b
2013-02-12 18:16:30 -08:00
Devananda van der Veen
bfb4b8a5de Remove nova.db call from baremetal PXE driver
Baremetal PXE driver was fetching instance_type from the nova db.
Now that no-db-compute is done, that's not possible. Also, there's a
more efficient way -- just call extract_instance_type(instance)!

Baremetal unit tests should check for instance_types with swap = 0 and
with swap > 0. This required changing test/utils to include a
real copy of instance_types in instance['system_metadata'].

Change-Id: I2a91e5c026782946f6b01e4189ec85d30ba87583
2013-02-07 16:42:29 -08:00
Chris Krelle
09de325519 Add a virtual PowerDriver for Baremetal testing
This virtual_power_driver provides the ability for baremetal driver to
start/stop/reboot VMs within common dev/test environments. Its goal is to
approximate the same functionality as other PowerDrivers (eg. IPMI) within
a desktop development environments and within the devstack-gate environment.

Authored-by: Chris Krelle <nobodycam@gmail.com>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>

Change-Id: I9935a9a30a166a9c28a9abb139c49f219a32ddd7
2013-01-16 18:40:21 -08:00
Rafi Khardalian
d93b8109ae Recache or rebuild missing images on hard_reboot
The primary purpose of this change is to provide the ability to
re-cache missing backing files on hard_reboot. The old
pre_block_migration function was already performing a very similar
operation. That function has been refactored to be idempotent and
renamed to _create_images_and_backing. The pre_block_migration
function is a wrapper, with some additional checking, around the
renamed function.

Image backend was also adjusted to look for either a missing backing
file or disk image, recaching or creating accordingly. It should
also be idempotent, never clobbering existing images.

Change-Id: Icf4c488d6db59e732b463d08d0606b428ee1e7b9
2013-02-05 09:40:15 +00:00
Eric Windisch
3d07dfe4e4 Use oslo database code
Bring in the new database code from oslo.

Uses get_session() from oslo as well as changing NovaBase to derive from
a common class.

Remove test_sqlalchemy.py now that this code is test in oslo.

Implements blueprint db-common.

Change-Id: I090754981c871250dd981cbbe1a08e7181440120
2013-01-17 15:38:02 -05:00
Zhongyue Luo
c8386dbefe Fixes 'not in' operator usage
Change-Id: I1e26a8fcb9fa564308e63c11a72aaa55119e4eee
2013-01-31 13:56:42 +08:00
Joe Gordon
1017d78950 Make sure there are no unused import
Remove all currently unused imports
Prevent future unused imports

Change-Id: I6ac26d5c71b79952a7732db300355a00310c712e
2013-01-25 20:34:03 +00:00
Joe Gordon
e515a666f3 Enable N302: Import modules only
Fix all N302 issues, and re-enable.

Change-Id: Ic94d144c915b228b7ff2fd9c5951875e159ffcdd
2013-01-25 12:10:16 -05:00
Arata Notsu
c511d4dbb5 Correct a format string in virt/baremetal/ipmi.py
"%(err)%s" to "%(err)s"

Change-Id: I5ef1d3218c6121fed7c5d2b4d318da7c6d884776
2013-01-23 20:48:19 +09:00
Arata Notsu
1396d2ffbc Add REST api to manage bare-metal nodes
* create/delete/list/show bare-metal nodes
* add/remove interfaces to/from bare-metal nodes

blueprint general-bare-metal-provisioning-framework

Change-Id: I1e76f7e3f7f74087e844cfb23dc92154f4c3e127
2013-01-23 17:32:46 +09:00
Devananda van der Veen
fd6120c877 Baremetal/utils should not log certain exceptions
unlink_without_raise was logging exceptions when it failed to unlink a
file, which is confusing since it is often called on non-existing files.

create_link_without_raise was also logging exceptions when it failed
to create a symlink.

This patch corrects this behaviour; both functions now explicitly check
for the type of error they expect, and then suppress that. If another
type of OSError is encountered, a warning is logged. This patch also
adds unit tests for both functions.

fixes bug 1097931.

Change-Id: Ie36f59dce34a6c67765770c5f968c003003acc88
2013-01-17 13:30:58 -08:00
Devananda van der Veen
1c9111eaa2 PXE driver should rmtree directories it created
Baremetal PXE driver was failing to delete the per-instance tftpboot and
image directories which it created when the instance was deleted. This
happened partly because of dangling files within the directory, and
partly because 'unlink' does not remove directories and the error was
squelched.

Now, when destroy()ing an instance, PXE driver will call shutil.rmtree()
on the per-instance directories which it created.

Fixes bug 1101048.

Change-Id: I682d0264288add488ea23e5d5200357b7be52dd9
2013-01-18 09:40:37 -08:00
Davanum Srinivas
84d8b43626 Add support for Option Groups in LazyPluggable
Use @markmc's suggestion to enhance LazyPluggable with an
optional config group. Also fix the baremetal database
backend option to use the "baremetal" config group.

Fixes LP #1093043

Change-Id: I28cf51a2962f516fcef4ced19e30c985220e86dc
2013-01-21 21:18:34 -05:00
Robert Collins
5b04310d99 Remove obsolete baremetal override of MAC addresses.
Now that the hypervisor driver can specify what MAC addresses it
needs, overriding the MAC address during bare metal instance
provisioning is no longer appropriate.

Change-Id: I2b0790753297104ddd40a410c8acb5fdac97ad15
2013-01-16 13:20:47 +13:00
Devananda van der Veen
40177796d1 PXE driver should not accept empty kernel UUID.
Baremetal PXE driver was aborting only if the kernel and ramdisk
specified in image metadata were None, but not if they were another
non-true value, such as "". Now, exception is raised if they are any
non-true value.

Fix bug 1100589.

Change-Id: I9783a14cc242e1b9db04d797c61d54b06d52c680
2013-01-17 09:09:32 -08:00
Matt Joyce
8f09391466 Correcting improper use of the word 'an'.
Change-Id: I032509539c61f2f5fda2001e2db5a3298e30c561
2013-01-16 11:41:37 -08:00