Several improvements to baremetal driver are implemented in this patch.
There is now significantly more error handling during spawn().
It also includes an addition to nova/tests/utils.py to provide
additional sample information from get_test_network_info().
blueprint general-bare-metal-provisioning-framework
Change-Id: I65d93051d7fcfd79f4d24d4ddb62fb1a55bee646
Some baremetal power drivers may require different information passed to
them. This patch prepares the way by having the base class take only
**kwargs, and having the driver pass both 'instance' and 'node', instead
of just passing 'node'.
blueprint general-bare-metal-provisioning-framework
Change-Id: Iff91024b1d019b0d07f2cbfe991748e618bfcb18
This patch implements only the IPMI power manager for baremetal nova
compute. Documentation will come in a separate patch.
blueprint general-bare-metal-provisioning-framework
Change-Id: I60ccfbf963d7bbf6f840e627396601b7bba80e7f
Unit tests for baremetal/driver.py were not functioning and were
falsely reporting success due to inheriting from the wrong class.
This lead to drift between the tests and the code over time.
I decided it was easier to re-implement tests/baremetal/test_driver.py
instead of trying to fix the drift. This gave me the chance to make
the test class easier to extend. Additional tests will be added
in the future, when I refactor driver.py:spawn().
This patch also has some minor variable name changes for baremetal/driver.py.
blueprint general-bare-metal-provisioning-framework.
Change-Id: I36ba5ed340b722b9a6e8d02d6d57d3c8d53b1eef
Move all the baremetal options into a new [BAREMETAL] OptGroup,
except for 'baremetal_db_backend', which breaks LazyPluggable if moved.
blueprint general-bare-metal-provisioning-framework
Change-Id: I06f53dcf10b9bbd3fd7cc0725cc6076af3d74d19
Use custom Session class to avoid monkey patching of SqlAlchemy
Session instance.
blueprint db-session-cleanup
Change-Id: I7d40c20e585782403556f3587141805c197bfa64
Removed a bunch of unused imports based on pyflakes.
Change-Id: I00952371200cb4531ddf163447433da756582a21
Signed-off-by: Chuck Short <chuck.short@canonical.com>
Make more of the db connection options into parameters instead of CONF
values so that they behave better when testing multiple DB back-ends,
as with the baremetal driver.
Change-Id: Ib122b86ded4bce430773c36fdb5f85bb71100d3d
This patch implements VIF and Volume subdrivers for baremetal driver.
It was separated from review 11354.
blueprint general-bare-metal-provisioning-framework.
Change-Id: Id187d04b6bd7b838159d46d61be72256414d0e72
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
This is a minimal patch for the new baremetal driver.
With this driver, nova compute registers multiple entries of baremetal
nodes. It periodically updates the capabilities of the multiple
baremetal nodes and reports it as a list of capabilities.
It does not include Tilera or PXE back-ends, which will be provided
by subsequent patches. It also does not include VIF or volume components.
Part 4 of 7: blueprint general-bare-metal-provisioning-framework.
Change-Id: I55617a8da52d20d4df727b8bbde8e5f72d3bf130
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
Co-authored-by: Chris Krelle <NobodyCam@gmail.com>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>
The baremetal database tests didn't get migrated to fixtures when we migrated
base nova. Funny story: maybe we should re-use code rather than copying!
So I did that. It's reusable now.
Change-Id: I730aad4a6aedb3993e032a14c9b8d2cd58913b4f
Raise exceptions from baremetal/db/sqlalchemy/api.py
following the precedents from nova/db/sqlalchemy/api.py.
Fixes tests to expect the new exceptions.
Adds __init__ so that baremetal tests work in isolation.
blueprint general-bare-metal-provisioning-framework
Change-Id: Ic8ef66e8f3180460a7bf117fccbfe15078905d8b
Without this file in place I was seeing some test failures from the
baremetal unit tests. The failure was:
ImportError: No module named baremetal.db
Change-Id: I7596e7916796d5cb382d3856918b35c01814a455
With a few minor exceptions, the sql config options are used solely
within the nova.db.sqlalchemy.session module so it makes sense to move
their declaration into that module.
Change-Id: Iea9c2bb000cd713b01750ab3e796132ebeaa4ca8
The only reason for importing nova.config now is where one of the
options defined in that file is needed. Rather than importing
nova.config using an import statement, use CONF.import_opt() so
that it is clear which option we actually require.
In future, we will move many options out of nova.config so many
of these import_opt() calls will either go away or cause a module
other than nova.config to be imported.
Change-Id: I0646efddecdf2530903afd50c1f4364cb1d5dce1
Modules import nova.config for two reasons right now - firstly, to
reference nova.config.CONF and, secondly, if they use one of the
options defined in nova.config.
Often modules import nova.openstack.common.cfg and nova.config
which is a bit pointless since they could just use cfg.CONF if
they just want to nova.config in order to reference CONF.
Let's just use cfg.CONF everywhere and we can explicitly state
where we actually require options defined in nova.config.
Change-Id: Ie4184a74e3e78c99658becb18dce1c2087e450bb
Remove files from the old baremetal driver
to prepare the way for the new driver to be
added in the next patch.
Change-Id: I51f998444c805838d3cf20db59407f90f278399f
Now that options have all moved from nova.flags to nova.config, we can
safely remove the nova.flags imports and replace them with nova.config
imports.
Change-Id: Ic077a72dd6419bbf1e1babe71acfa43c4e8b55c8
The services API extension was still using FLAGS. And baremetal virt
driver define FLAGS but never used it.
Change-Id: I95c78d04f9103f13c09cbdbc23d6d4ed0d39dde8
Part 2 of 6: blueprint general-bare-metal-provisioning-framework
In baremetal provisioning, one nova-compute manages multiple bare-metal
machines. A bare-metal machine does not run openstack at all.
Previously, bare-metal provisioning used text files to store information
of bare-metal machines. In this patch, a MySQL database is used to store
the information. We target only MySQL database. The DB is designed to
support PXE/non-PXE booting methods, heterogeneous hypervisor types, and
architectures. Using a MySQL database makes maintenance and upgrades
easier than using text files. The DB for bare-metal machines is
implemented as a separate DB from the main Nova DB. The DB can be on any
machines/places. The location of the DB and its server needs to be
specified as a flag in the nova.conf file (as in the case of glance).
There are a couple of reasons for this approach. First, the information
needed for bare-metal machines is different from that for non-bare-metal
machines. With a separate database for bare-metal machines, the database
can be customized without affecting the main Nova DB. Second, fault
tolerance can be embedded in nova-compute. Since one nova-compute
manages multiple bare-metal machines, fault tolerance of a nova-compute
node is very important. With a separate DB for bare-metal machines,
fault-tolerance can be achieved independently from the main Nova DB.
Replication of the bare-metal DB and implementation of fault-tolerance
are not part of this patch. The implementation models nova and its DB as
much as possible. The bare-metal driver must be upgraded to use this DB.
Change-Id: I7b7ba1903a672a50c567f95fc6554d119463b0c5
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
Use the global CONF variable instead of FLAGS. This is purely a cleanup
since FLAGS is already just another reference to CONF.
We leave the nova.flags imports until a later cleanup commit since
removing them may cause unpredictable problems due to config options not
being registered.
Change-Id: Ic0168188dfe214fc81af04c8a9644d2d9f31600d
Part 1 of 6: blueprint general-bare-metal-provisioning-framework.
This patch includes updates on scheduler and compute codes for
multiple capabilities. This feature is needed in bare-metal
provisioning which is implemented in later patches --- a bare-metal
nova-compute manages multiple bare-metal nodes where instances are
provisioned. Nova DB's compute_nodes entry needs to be created for
each bare-metal node, and a scheduler can choose an appropriate
bare-metal node to provision an instance.
With this patch, one service entry with multiple compute_node entries
can be registered by nova-compute. Distinct 'node name' is given for
each node and is stored at compute_node['hypervisor_hostname'].
And we added a new column "node" to "instances" table in Nova DB to
associate instances with compute_node. FilterScheduler puts <nodename>
to the column when it provisions the instance. And nova-computes
respect <nodename> when run/stop instances and when calculate
resources.
Also, 'capability’ is extended from a dictionary to a list of
dictionaries to describe the multiple capabilities of the multiple
nodes.
Change-Id: I527febe4dbd887b2e6596ce7226c1ae3386e2ae6
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
Use the global CONF variable instead of FLAGS. This is purely a cleanup
since FLAGS is already just another reference to CONF.
We leave the nova.flags imports until a later cleanup commit since
removing them may cause unpredictable problems due to config options not
being registered.
Change-Id: Icb61d67965628dca65bda393d4373beab5c5c64a
hypervisor_hostname is set in get_available_resource() and
in get_host_stats(). Except for vmwareapi driver.
Change-Id: I66348ae1e986162b8139b04ab03c42c424338175
This patch introduces a VirtAPI class which will house
callbacks provided by the manager to the virt drivers, allowing
things such as direct database accesses to be pulled out of
the virt drivers and delegated to another service.
As a first step, this introduces an instance_update() method
and makes all the virt drivers use it instead of direct calls
to db.instance_update.*().
Change-Id: I2e40831f5cfb20a03b304097d84d592aab035ef1
Rename function arguments to something which more clearly indicates
their purpose. No functional changes.
Change-Id: Iedbc5477ac54527b60c520fd7c774e608adb07c0
Keep the compute host's ComputeNode model in sync with the
level of resource usage. This enables the ComputeNode
model to be used as a basis for scheduling decisions
rather than forcing scheduler to calculate free
resources from an instance_get_all on each request.
Resources like memory and disk are claimed as instances are built
or deleted. There is also support for configurable compute node
stats (a generic key/value store) for extensible advertising of other
usage stats that may be useful for a particular scheduler
implementation. Additionally, there is a periodic task on the
compute host that audits actual resource consumption at the virt
layer to ensure that the database stays in sync.
This change partially implements blueprint: scheduler-resource-race
This patch complements:
https://review.openstack.org/#/c/9540/ (build re-scheduling support)
Change-Id: Ibbe3839a054f8b80664b413d47f766ca8d68e3f2
The RPC API passes admin_pass and injected_files as arguments, but when
calling the driver API, those values get stuffed into the instance dict
which is otherwise all data that is stored in the database. This has
led to some bugs (such as 1034948) which could have been avoided if
the code was consistent.
Change-Id: I09a7de8eca3791a5ab27e93b22e5ccccee31cf09
Fixes bug 1029495
The trigger_members_refresh method in compute.api.py specifies
a group id in the call to refresh_security_group_members. This
is just the last group id seen and ignores the fact that a
refresh may impact members of multiple groups.
This is masked by the fact that on the host the group id is
ignored and all instances have their security rules refreshed
regardless of if they are part of the changed group or not.
This change modifies the logic surrounding refreshes so we send
a refresh request for each instance which is affected by a
security group change, this ensures we aren't spending time
refreshing unaffected instances and also removes the possibility
of refreshing an instance multiple times if it is a member of
more than one group.
Also changed to be instance-centric is the refresh carried out
when a rule is added/removed to a security group.
Change-Id: Iec98e9aed818fdc4ecc88c8dcdd4ee5fa9386e00
When reading image_ref from an instance, use dict style syntax. This is
to be compatible with no-db-messaging changes.
Part of blueprint no-db-messaging.
Change-Id: I19753da6301c5d78b8fb13b30fbc6996ba828096
We no longer use InstanceInfo.
Also removes some unused _map_to_instance_info functions in
various virt drivers.
Change-Id: I02ead10e43426b9ad051970e9e78e970de5b1ac1
Removes the list_instances_detail function from various compute drivers
which implement it. We no longer make use of this compute driver
call when syncing power states.
Change-Id: I4980bd8d4ec14c61e16d5be105659f0d5b6748ba
This import was recently reintroduced, due to
confusing pylint output. Therefore remove it again
and adjust things so pylint won't warn in future.
Change-Id: I25f293f171c651c25d27e8d55c9240fd808590ba
This refactoring of the libvirt live migration code is
required to enable live migration in the xenapi driver.
This change ensures libvirt specific checks are performed
only when the libvirt driver is enabled.
The complication is that some of these checks require
information to be passed between the source and destination
hosts. For example, when comparing CPU flags.
Change-Id: I7389f0b7f03313d7f04b907f481787dadf0716fd
Part of bp:virt-driver-cleanup
Make the baremetal driver consistent in naming as a driver
instead of a proxy / connection.
Change-Id: I75d7d90bd8139842b588f1fafb1267511f29a1fb
* Adds wait() to all the LoopingCall timers
* Fixes loopingCalls in baremetal/proxy.py as well
* Includes failing test to verify result from destroy
* Fixes tests depending on improper behavior
* Fixes bug 1015355
Change-Id: I9e1914c446170e49f0aab76c7745b55d12132425
Fixes bug #1013770
1. Reorder imports by full module path
2. Insert two blank lines after the last import
Change-Id: I294ac3ab528f17a72811392d1732158e9487f3bf
Progress on bp:virt-driver-cleanup
This series of patches converts the virt drivers to loading via
importutils making it possible to add a virt driver without changing
core code. It deprecates the use of connection_type in favor of a
full driver name in compute_driver.
Based on email thread with Vish and Jay Pipes on approaches to do
this.
Change-Id: I48366ec7efc7b095859988e5e6ac5a853b98b8a7
Removed duplicate and invalid state in power_state:
FAILED, SHUTOFF, BLOCKED
This is the first step in cleaning up nova state machine
and do better task management (bp/task-management)
http://wiki.openstack.org/VMState
Change-Id: I586b9058fada5efd468870fb187590fc0e37aa8f
address minor problems and errors found in the driver, such as old
docstrings, unused imports, and variables, references of non-existent
classes, and methods; plus remove the requirement that every sub-driver
needs to be implemented as singleton, as the tilera one.
Change-Id: I5943bcdd785c1b4b83e95f88bd7d69c1edc5f62f
Added a instance update notification (compute.instance.update) that
will report on changes to vm_state and task_state. The goal here is
to provide useful insight into instance state transitions. (e.g.
BUILDING->ACTIVE)
The new notification has minimial dependencies and is intended for
wide use across the different layers/packages within nova. Calls
in compute api/manager, scheduler, and the virt layer that modify
the instance state have been instrumented with this notification.
Change-Id: I223eb7eccc8aa079b782f6bb17727cd0b71d18ed
The latest release of pep8 adds stricter rules for extraneous
whitespace within lines. This patch also addresses these new
violations acoss the codebase.
Change-Id: Ib7e50281870473df1704ed50868c5c2e26bdb02e
