11394 Commits

Author SHA1 Message Date
Robert Collins
7b7c28f939 Export the MAC addresses of nodes for bare-metal.
The bare-metal hypervisor needs to let the network layer know the MAC
address of each node it has, or TFTP boot will fail, as the MAC
addresses quantum / nova dynamically allocate will not match the
actual MAC of the node as it boots. This change exports the MAC
addresses to the manager, which passes them onto the network driver in
use.

With this change administrators should register all the MAC addresses
of a given node with Nova bare-metal as interfaces, even though that
may duplicate the MAC for PXE boot provisioning. Long term the
dedicated provisioning MAC address will be removed.

Change-Id: I55f6031294a2c5d31975462f868aa27441e11ad2
2013-01-15 14:16:57 +13:00
Robert Collins
2448914cfb Break out a helper function for working with bare metal nodes.
This makes it easier to get a bare metal internal node id when called
from the manager.

Change-Id: I1b39a7be5bd21d5b3060f11a74b9eaf79b107210
2013-01-15 13:44:21 +13:00
Sandy Walsh
47af92f3c0 Keep self and context out of error notification payload.
Back in the day, having self and context in the error notifications
was handy for debugging. Now, there is a lot of confidential stuff
stored in these objects (especially when self = ComputeManager) ...
like passwords, etc.

This patch strips it out.

Also removes dead wrap_exception calls (which did nothing since they
did not specify a notifier).

Change-Id: Ieab7bd79b64e01c7bca18dbce97455e50094871c
2013-01-08 16:20:12 -06:00
Mikyung Kang
fefe7d99ae Tests for PXE bare-metal provisioning helper server.
a part of blueprint general-bare-metal-provisioning-framework.

The deploy helper was landed without tests. This patch reinstates the
tests.

Change-Id: I690ec077e175dc37c2cc5f18170513b0e5bca103
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>
2012-11-10 08:07:13 +09:00
Robert Collins
7e683ab992 Change ComputerDriver.legacy_nwinfo to raise by default.
This makes non-updated hypervisors visible by grepping for 'def
legacy_nwinfo' rather than only the updated ones being visible, and
when new hypervisors are added, it will be clear whether they use the
legacy format or not.

Out of tree hypervisors will be broken by this, but the fix is
trivial::

def legacy_nwinfo(self):
    # XXX TODO Update to use the non-legacy format.
    return True

Change-Id: If5b461bc5d8e8dc21de3ca9cf521e7b341724900
2013-01-11 10:20:43 +13:00
Sean Dague
6d7fff01d9 fix new N402 errors
fix the N402 errors that have slipped in in the last 48 hrs since
starting this patch series.

fix an N401 error that our scanner current can't find because it
only looks for doc strings on classes and defs.

this is the xeno's paradox of patch series, but we're getting close.

Change-Id: I4a763bb4c812335d853eae05c72464f18ab93297
2013-01-09 16:00:36 -05:00
Mark McLoughlin
69f6994ecc Remove unused baremetal PXE options
These options were introduced by commit a2f260f and appear to never to
have been used.

Change-Id: Idf1387fd453b7b1c392db764934c148dcbfc5d66
2013-01-09 10:43:50 +00:00
Mark McLoughlin
8e21f48b01 Move global service networking opts to new module
The my_ip, host and use_ipv6 options are used all over the codebase
and they're pretty well related to each other. Create a new netconf
module for them to live in.

There are now no options registered globally in nova.config!

blueprint: scope-config-opts
Change-Id: Ifde37839ae6f38e6bf99dff1e80b8e25fd68ed25
2013-01-08 09:48:03 +00:00
Sean Dague
e0761c2f52 Fix N402 for nova/virt
Fix N402 errors (single line docstring should end in a period)
for nova/virt, part of a larger attempt to stop ignoring our own
hacking.py tests.

Change-Id: I523ce41bd2b38c73cf3fdb031101ccc0695f2488
2013-01-08 16:25:23 -05:00
Robert Collins
c09985122b Cope better with out of sync bm data.
The baremetal hypervisor tracks references to instances which are
deleted asynchronously from the commit to the nova bm. As such, its a
normal but rare condition for a bare metal node to refer to a deleted
(and even garbage collected) nova instance. We should treat such
instances are deleted rather than erroring on any call through
list_instances(), permitting starting new instances and listing
instances to work - fixing bug: 1096722.

Pathologically, the database may suffer permanent skew, which means we
need a way to fix it (and thats tracked as a separate bug).

Change-Id: Ic21ff66b1fc0ad64bb5feff26291873b96d20e4e
2013-01-07 15:17:42 +13:00
Devananda van der Veen
2032c91507 Fix baremetal VIFDriver
Commit Id287f7e661 removed the nova/virt/vif.py file and the VIFDriver
base class, and erroneously stated in its commit message that baremetal
driver did not use it. This patch repeats the work of that earlier patch
by removing the import of nova.virt.vif from baremetal/vif_driver.py.

blueprint general-bare-metal-provisioning-framework

Change-Id: Ifac9510a72a028dc5d3574148f3c4a6410b1d249
2013-01-08 09:39:37 -08:00
Mikyung Kang
af631b6a46 CLI for bare-metal database sync.
Part 3 of 6: blueprint general-bare-metal-provisioning-framework.

Change-Id: Ia19ce00edb84aa924c2ab2c9c2217f6b49073d69
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
2013-01-08 22:56:14 +09:00
Arata Notsu
2f772ae031 attach/detach_volume() take instance as a parameter
Previously the methods take instance['name'] as a parameter.
With this change, ComputeDriver can lookup informations about
the instance from hypervisor (in bare-metal driver, it is a local DB)
by any other attributes of the instance.

blueprint general-bare-metal-provisioning-framework

Change-Id: Ibd0567f34ed5053909ce1a408f9cbf87516ba597
2012-12-20 21:29:13 +09:00
Devananda van der Veen
ff7e294721 Convert short doc strings to be on one line
This patch just cleans up some one-line doc strings which were
split across 3 lines needlessly.

Change-Id: I3c2011d20fb7c3b8714d7b5a32151e427bf8a311
2013-01-03 11:27:22 -08:00
Arata Notsu
d96094b5a0 Check admin context in bm_interface_get_all()
This patch puts missing @require_admin_context before
the function.

blueprint general-bare-metal-provisioning-framework

Change-Id: I07d78f75b7f98a225398ca67b3bda251bc6c0440
2013-01-07 17:53:23 +09:00
Devananda van der Veen
5327259765 Provide a PXE NodeDriver for the Baremetal driver
This patch implements a PXE NodeDriver class within the Baremetal
provisioning framework, which provides a means for deploying machine
images using TFTP and PXE.

This patch relies on functionality provided by the nova-baremetal-deploy-helper
utility, implemented in review 15830.

blueprint general-bare-metal-provisioning-framework.

Change-Id: I8d849601186e3dc13f10382857ff2bbc1ff1026d
2012-12-27 17:23:29 -08:00
Michael Still
335f0f2eab Refactor periodic tasks.
This review allows periodic tasks to be enabled or disabled in the
decorator, as well as by specifying an interval which is negative.

The spacing between runs of a periodic task is now specified in
seconds, with zero meaning the default spacing which is currently 60
seconds.

There is also a new argument to the decorator which indicates if a
periodic task _needs_ to be run in the nova-compute process. There is
also a flag (run_external_periodic_tasks) which can be used to move
these periodic tasks out of the nova-compute process.

I also remove the periodic_interval flag to services, as the interval
between runs is now dynamic based on the number of seconds that a
periodic task wants to wait for its next run. For callers who want to
twiddle the sleep period (for example unit tests), there is a
create() argument periodic_interval_max which lets the period
periodic_tasks() specifies be overridden. This is not exposed as a
flag because I cannot see a use case for that. It is needed for unit
testing however.

DocImpact. Resolves bug 939087.

Change-Id: I7f245a88b8d229a481c1b65a4c0f1e2769bf3901
2012-12-24 15:00:52 +11:00
Mark McLoughlin
3cff674671 Add helper methods to nova.paths
We only ever do one of two things with the global path options:

  1) Reference the option in the default of another option so that the
     value can be interpolated it

  2) Use the value of the option to build a path

Add helper methods for both these cases - e.g. basedir_def() for the
former case and basedir_rel() for the latter case. This makes it much
more obvious how and where these options are used.

Change-Id: I7fd94a329fe911761d02d94e5381e950c6668d56
2013-01-04 17:36:29 +00:00
Mark McLoughlin
cf0363115a Move global path opts in nova.paths
Move the global path config options (i.e. state_path, pybasedir and
bindir) into a new nova.paths module. A new module may seem like
overkill but some utility methods associated with these options follow
in a later commit.

Moving them to nova.paths means they are no longer globally defined
and it's more obvious which modules require these options.

Change-Id: I381d23f1bbe36dc6967a38a65062b0983e1661aa
2013-01-04 17:32:36 +00:00
Zhongyue Luo
b144e6d1d7 Removes unused imports
Change-Id: I76d243329f810cdcd6f8a5459eb66fddd5871e33
2013-01-03 18:21:36 +08:00
Devananda van der Veen
b107b2d6b1 Improve baremetal driver error handling
Several improvements to baremetal driver are implemented in this patch.
There is now significantly more error handling during spawn().

It also includes an addition to nova/tests/utils.py to provide
additional sample information from get_test_network_info().

blueprint general-bare-metal-provisioning-framework

Change-Id: I65d93051d7fcfd79f4d24d4ddb62fb1a55bee646
2012-12-27 17:19:21 -08:00
Devananda van der Veen
45fa41ba92 baremetal power driver takes **kwargs
Some baremetal power drivers may require different information passed to
them. This patch prepares the way by having the base class take only
**kwargs, and having the driver pass both 'instance' and 'node', instead
of just passing 'node'.

blueprint general-bare-metal-provisioning-framework

Change-Id: Iff91024b1d019b0d07f2cbfe991748e618bfcb18
2012-12-27 15:13:58 -08:00
Devananda van der Veen
f9c10b39e4 Implement IPMI sub-driver for baremetal compute
This patch implements only the IPMI power manager for baremetal nova
compute. Documentation will come in a separate patch.

blueprint general-bare-metal-provisioning-framework

Change-Id: I60ccfbf963d7bbf6f840e627396601b7bba80e7f
2012-12-21 20:15:38 -08:00
Devananda van der Veen
755a945a17 Fix tests/baremetal/test_driver.py
Unit tests for baremetal/driver.py were not functioning and were
falsely reporting success due to inheriting from the wrong class.
This lead to drift between the tests and the code over time.
I decided it was easier to re-implement tests/baremetal/test_driver.py
instead of trying to fix the drift. This gave me the chance to make
the test class easier to extend. Additional tests will be added
in the future, when I refactor driver.py:spawn().

This patch also has some minor variable name changes for baremetal/driver.py.

blueprint general-bare-metal-provisioning-framework.

Change-Id: I36ba5ed340b722b9a6e8d02d6d57d3c8d53b1eef
2012-12-22 16:35:58 -08:00
Devananda van der Veen
b4d44338b9 Move baremetal options to [BAREMETAL] OptGroup
Move all the baremetal options into a new [BAREMETAL] OptGroup,
except for 'baremetal_db_backend', which breaks LazyPluggable if moved.

blueprint general-bare-metal-provisioning-framework

Change-Id: I06f53dcf10b9bbd3fd7cc0725cc6076af3d74d19
2012-12-21 11:09:48 -08:00
Boris Pavlovic
8001d748ea Remove session.flush() and session.query() monkey patching
Use custom Session class to avoid monkey patching of SqlAlchemy
Session instance.

blueprint db-session-cleanup

Change-Id: I7d40c20e585782403556f3587141805c197bfa64
2012-12-28 15:21:08 +04:00
Joe Gordon
022f405691 Remove unused imports
And one unused variable

Change-Id: I81edaf7ddedd7723c2601b30951ca82bd2cd5256
2012-12-26 23:42:14 +00:00
Chuck Short
e9cf0dce98 Removed unused imports.
Removed a bunch of unused imports based on pyflakes.

Change-Id: I00952371200cb4531ddf163447433da756582a21
Signed-off-by: Chuck Short <chuck.short@canonical.com>
2012-12-21 08:39:32 -06:00
Devananda van der Veen
c75e6b8f8e Parameterize database connection in test.py
Make more of the db connection options into parameters instead of CONF
values so that they behave better when testing multiple DB back-ends,
as with the baremetal driver.

Change-Id: Ib122b86ded4bce430773c36fdb5f85bb71100d3d
2012-12-20 09:40:01 -08:00
Mikyung Kang
35536a5ac7 Baremetal VIF and Volume sub-drivers.
This patch implements VIF and Volume subdrivers for baremetal driver.
It was separated from review 11354.

blueprint general-bare-metal-provisioning-framework.

Change-Id: Id187d04b6bd7b838159d46d61be72256414d0e72
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
2012-12-13 02:56:56 +09:00
Mikyung Kang
aaeb899f98 New Baremetal provisioning framework.
This is a minimal patch for the new baremetal driver.

With this driver, nova compute registers multiple entries of baremetal
nodes. It periodically updates the capabilities of the multiple
baremetal nodes and reports it as a list of capabilities.

It does not include Tilera or PXE back-ends, which will be provided
by subsequent patches. It also does not include VIF or volume components.

Part 4 of 7: blueprint general-bare-metal-provisioning-framework.

Change-Id: I55617a8da52d20d4df727b8bbde8e5f72d3bf130
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
Co-authored-by: Chris Krelle <NobodyCam@gmail.com>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>
2012-12-13 02:56:56 +09:00
Monty Taylor
63f55af401 Move baremetal database tests to fixtures.
The baremetal database tests didn't get migrated to fixtures when we migrated
base nova. Funny story: maybe we should re-use code rather than copying!

So I did that. It's reusable now.

Change-Id: I730aad4a6aedb3993e032a14c9b8d2cd58913b4f
2012-12-19 13:04:21 -08:00
Devananda van der Veen
63fff7e661 Add exceptions to baremetal/db/api
Raise exceptions from baremetal/db/sqlalchemy/api.py
  following the precedents from nova/db/sqlalchemy/api.py.
Fixes tests to expect the new exceptions.
Adds __init__ so that baremetal tests work in isolation.

blueprint general-bare-metal-provisioning-framework

Change-Id: Ic8ef66e8f3180460a7bf117fccbfe15078905d8b
2012-11-20 16:06:51 -08:00
Russell Bryant
a3dbc4c866 Add blank nova/virt/baremetal/__init__.py
Without this file in place I was seeing some test failures from the
baremetal unit tests.  The failure was:

    ImportError: No module named baremetal.db

Change-Id: I7596e7916796d5cb382d3856918b35c01814a455
2012-11-26 18:13:29 -05:00
Mark McLoughlin
0ad0d54a2b Move sql options to nova.db.sqlalchemy.session
With a few minor exceptions, the sql config options are used solely
within the nova.db.sqlalchemy.session module so it makes sense to move
their declaration into that module.

Change-Id: Iea9c2bb000cd713b01750ab3e796132ebeaa4ca8
2012-11-21 12:15:20 +00:00
Mark McLoughlin
afabe86de5 Use CONF.import_opt() for nova.config opts
The only reason for importing nova.config now is where one of the
options defined in that file is needed. Rather than importing
nova.config using an import statement, use CONF.import_opt() so
that it is clear which option we actually require.

In future, we will move many options out of nova.config so many
of these import_opt() calls will either go away or cause a module
other than nova.config to be imported.

Change-Id: I0646efddecdf2530903afd50c1f4364cb1d5dce1
2012-11-17 22:50:21 +00:00
Mark McLoughlin
8414a7c79f Remove nova.config.CONF
Modules import nova.config for two reasons right now - firstly, to
reference nova.config.CONF and, secondly, if they use one of the
options defined in nova.config.

Often modules import nova.openstack.common.cfg and nova.config
which is a bit pointless since they could just use cfg.CONF if
they just want to nova.config in order to reference CONF.

Let's just use cfg.CONF everywhere and we can explicitly state
where we actually require options defined in nova.config.

Change-Id: Ie4184a74e3e78c99658becb18dce1c2087e450bb
2012-11-17 22:50:17 +00:00
Devananda van der Veen
3deb8f1e13 remove old baremetal driver
Remove files from the old baremetal driver
to prepare the way for the new driver to be
added in the next patch.

Change-Id: I51f998444c805838d3cf20db59407f90f278399f
2012-11-16 10:00:48 -08:00
Mark McLoughlin
e22e6b38bd Remove nova.flags
Now that options have all moved from nova.flags to nova.config, we can
safely remove the nova.flags imports and replace them with nova.config
imports.

Change-Id: Ic077a72dd6419bbf1e1babe71acfa43c4e8b55c8
2012-11-15 19:41:58 -05:00
Chris Behrens
dafbf987cf Fix a couple uses of FLAGS
The services API extension was still using FLAGS.  And baremetal virt
driver define FLAGS but never used it.

Change-Id: I95c78d04f9103f13c09cbdbc23d6d4ed0d39dde8
2012-11-15 23:10:22 +00:00
Mikyung Kang
e0f10ea3be Added separate bare-metal MySQL DB.
Part 2 of 6: blueprint general-bare-metal-provisioning-framework

In baremetal provisioning, one nova-compute manages multiple bare-metal
machines. A bare-metal machine does not run openstack at all.
Previously, bare-metal provisioning used text files to store information
of bare-metal machines. In this patch, a MySQL database is used to store
the information. We target only MySQL database. The DB is designed to
support PXE/non-PXE booting methods, heterogeneous hypervisor types, and
architectures. Using a MySQL database makes maintenance and upgrades
easier than using text files. The DB for bare-metal machines is
implemented as a separate DB from the main Nova DB. The DB can be on any
machines/places. The location of the DB and its server needs to be
specified as a flag in the nova.conf file (as in the case of glance).
There are a couple of reasons for this approach. First, the information
needed for bare-metal machines is different from that for non-bare-metal
machines. With a separate database for bare-metal machines, the database
can be customized without affecting the main Nova DB. Second, fault
tolerance can be embedded in nova-compute. Since one nova-compute
manages multiple bare-metal machines, fault tolerance of a nova-compute
node is very important. With a separate DB for bare-metal machines,
fault-tolerance can be achieved independently from the main Nova DB.
Replication of the bare-metal DB and implementation of fault-tolerance
are not part of this patch. The implementation models nova and its DB as
much as possible. The bare-metal driver must be upgraded to use this DB.

Change-Id: I7b7ba1903a672a50c567f95fc6554d119463b0c5
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
2012-11-07 19:10:56 +09:00
Mark McLoughlin
0416bd96a7 Switch from FLAGS to CONF in tests
Use the global CONF variable instead of FLAGS. This is purely a cleanup
since FLAGS is already just another reference to CONF.

We leave the nova.flags imports until a later cleanup commit since
removing them may cause unpredictable problems due to config options not
being registered.

Change-Id: Ic0168188dfe214fc81af04c8a9644d2d9f31600d
2012-11-04 21:32:53 +00:00
Mikyung Kang
c14b4282ea Updated scheduler and compute for multiple capabilities.
Part 1 of 6: blueprint general-bare-metal-provisioning-framework.

This patch includes updates on scheduler and compute codes for
multiple capabilities. This feature is needed in bare-metal
provisioning which is implemented in later patches --- a bare-metal
nova-compute manages multiple bare-metal nodes where instances are
provisioned. Nova DB's compute_nodes entry needs to be created for
each bare-metal node, and a scheduler can choose an appropriate
bare-metal node to provision an instance.

With this patch, one service entry with multiple compute_node entries
can be registered by nova-compute. Distinct 'node name' is given for
each node and is stored at compute_node['hypervisor_hostname'].
And we added a new column "node" to "instances" table in Nova DB to
associate instances with compute_node. FilterScheduler puts <nodename>
to the column when it provisions the instance. And nova-computes
respect <nodename> when run/stop instances and when calculate
resources.

Also, 'capability’ is extended from a dictionary to a list of
dictionaries to describe the multiple capabilities of the multiple
nodes.

Change-Id: I527febe4dbd887b2e6596ce7226c1ae3386e2ae6
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
2012-11-10 10:20:45 +09:00
Mark McLoughlin
e98697fba0 Switch from FLAGS to CONF in nova.virt
Use the global CONF variable instead of FLAGS. This is purely a cleanup
since FLAGS is already just another reference to CONF.

We leave the nova.flags imports until a later cleanup commit since
removing them may cause unpredictable problems due to config options not
being registered.

Change-Id: Icb61d67965628dca65bda393d4373beab5c5c64a
2012-11-04 21:32:52 +00:00
Arata Notsu
a926c2ce2b Make ComputeDrivers send hypervisor_hostname
hypervisor_hostname is set in get_available_resource() and
in get_host_stats(). Except for vmwareapi driver.

Change-Id: I66348ae1e986162b8139b04ab03c42c424338175
2012-11-06 23:12:20 +09:00
Dan Smith
09c0ce4f05 Introduce VirtAPI to nova/virt
This patch introduces a VirtAPI class which will house
callbacks provided by the manager to the virt drivers, allowing
things such as direct database accesses to be pulled out of
the virt drivers and delegated to another service.

As a first step, this introduces an instance_update() method
and makes all the virt drivers use it instead of direct calls
to db.instance_update.*().

Change-Id: I2e40831f5cfb20a03b304097d84d592aab035ef1
2012-10-31 11:51:05 -07:00
Michael Still
39016b9672 Migrate to fileutils and lockutils.
Migrate nova to using openstack-common's file and lock utilities.
Resolves bug 1063230.

Change-Id: I1a4c87856bc08cd33b61d7098ed856baa4583654
2012-10-23 14:25:25 -07:00
Arata Notsu
d466c3ba4d Remove ComputeDriver.update_host_status()
Nobody calls the method.

Change-Id: Ib585f769788753c64e19fe49a1f186880e1c6f49
2012-10-18 22:40:01 +09:00
Michael Still
e4770a1792 Rename imagebackend arguments.
Rename function arguments to something which more clearly indicates
their purpose. No functional changes.

Change-Id: Iedbc5477ac54527b60c520fd7c774e608adb07c0
2012-09-03 20:49:45 +10:00
Michael Still
1d65383e08 Move ensure_tree to utils
Its useful to people other that virt drivers.

Change-Id: I721094a1785d7a275f4bfa8994b7b114a6ec07f6
2012-08-26 21:26:50 +10:00