Currently it is not possible to override an existing image block device
by supplying the device with the same name at boot (see also
Ib1ba130042aabbbe7bb8d60fc212c66e446c1d73). Even though we want to
discourage usage of device names as much as possible in the Nova API (as
not all hypervisors can honour them), EC2 API requires that this is possible.
While we want to make sure we document that supplying device names at
boot is only really desirable if you want to override some of the ones
contained in the image, introducing a different labeling system just so
that we don't use the device names seems like an overkill for a feature
that does not seem to be very used.
This patch adds a method that will do this deterministically when
compiling all the block device information for the request.
It is also worth noting that The EC2 API allows only subset of block
device attributes to be overridden in this way (see [1]). This limitation
did not exist previously in Nova, and there seems to be no reason why we
would need that complexity, so it would be up to the EC2 compatibility
code to deal with this.
[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html#Using_OverridingAMIBDM
Doc-Impact
Closes-bug: #1370250
Change-Id: I60ecdcae81ff5dec34f0fa0a39e0739759a6fa59
MKS is the native protocol for VMware consoles.
DocImpact: two config properties are added in group 'mks':
'mksproxy_base_url' - specifies that base URL for the MKS proxy
'enabled' - enables MKS console
Implements: blueprint vmware-webmks-console
Change-Id: Ia494ce050bd4dc58e5947e7f07cc3c815a257004
This was meant to be fixed by I017bf749f426717dc76cf99a387102848fb1c541 ,
but it didn't take into account that BDM entry was destroyed, which
caused the rebuild to fail when spawning the instance.
Add a new parameter to detach_volume() to bypass destroying of BDM,
as we just want to detach a volume first and then re-attach it again.
A Tempest test is added in I50557c69b54003d3409c8e977966f5332f4fe690
to make sure this is actually tested in the gate.
Closes-Bug: #1440762
Co-Authored-By: melanie witt <melwitt@yahoo-inc.com>
Change-Id: I9134fbf5ce72c32cca91de90001c09e00b4e19e8
In commit 5bd81ab5fb, prep_resize was
updated to send a flavor object and some dot notation was added, but
the previous version of the RPC API will send a primitive, so we should
convert to an object on the manager side so downstream code can safely
assume it's accessing an object.
Related to blueprint liberty-objects
Change-Id: I22165c1d1ebf99677d75fcb896ba92856490e79c
We will introduce claiming for live-migrate and evacuate operations, and
the claiming logic is really the same for all of those (we will create
different Migration object though, but this is unrelated to claims).
Make sure that the Claim subclass name reflects this fact.
In addition rename a method on the resource tracker for dropping such
claims to 'drop_move_claim' to match.
Related-blueprint: migration-fix-resource-tracking
Change-Id: I9872f05174f9f79e755a8ac2585ce9a14760b26a
This adds some notifications to two parts of the evacuation path that are
currently missing them. When the request is made, we synchronously fire one
indicating that we're sending the request to the conductor task manager. Then,
when that actually schedules a host and asks that host to do the rebuild,
we fire another. When it lands on the compute host, we'll get some notifications
about the rebuild starting, along with any errors.
Related to blueprint robustify-evacuate
Change-Id: I0054809573f04f03a3e4deeb482d61f280576ad3
This makes the evacuate process leave a record in the database about the
evacuation, and from what host it was being performed. Now, the compute
manager will use that breadcrumb trail to decide whether it should delete
locally-running instances, instead of just the hostname mismatch.
Since this does a query by migration_type, I added a unit test to the
test_db_api module to prove this actually works.
DocImpact: Deprecates the workarounds/destroy_after_evacuate config option
Related to blueprint robustify-evacuation
Closes-Bug: #1471887
Change-Id: I972c224dfb954a9f06e221c9235ee6e4889c2619
Just as libvirt needs to decide on the device names when booting an
instance (as opposed to them being assigned by the user), the same needs
to be done when attaching additional volumes to a running instance.
This patch exposes it on the ComputeDriver base class and makes sure
that it will be called by the compute manager (and that an appropriate
fallback will happen if the drivers do not care about overriding this),
as a pre-req for doing the work in the libvirt driver.
Partial-bug: 1452224
Related-bug: 1231874
Change-Id: I66693a8a6a632b65f930a58905336397ac6c9f29
To be able to later make the onSharedStorage parameter of the evacuate
API optional first we have to make the on_shared_storage parameter of
the rebuild_instance optional in the compute manager.
After this modification if the on_shared_storage is not provided then
the rebuild_instance call will use the instance_on_disk function of
the virt dirver to decide if the instance files need to be recreated or
they can be reused.
Implements: bp optional-on-shared-storage-flag-in-rebuild-instance
Change-Id: I4217bd00d8c253db522241885dba2847a26af6df
Up until now we would really only care about the size of ephemeral and
swap BDM entries'volume_size' being set properly, while we would rely on
Cinder to be the canonical source of truth for the size information of
snapshots and volumes.
There is some overhead in querying Cinder every time this basic
information is needed, and we do have a volume_size field in all BDM
entries already, so let's keep it up to date.
Change-Id: I9ffbb6cf40c1012f774a382cf1dcceba64637e6d
Related-bug: #1370177
Currently - we would happily accept the request, but fail to create the
block device mapping record for such a volume. The result was that the
boot request for such an instance would be accepted, but would not
create a volume.
Also fix the _volume_size API method that was 0ing out blank volume's
size by mistake.
Change-Id: I1d243aed78da7c026e55ea145397b070d619a567
Closes-bug: #1461638
The object is terminally updated based on whether the driver calls
the success or failure callback, which should be when things have
completely finished and properly reflect the driver's notion of
success.
Related to blueprint robustify-live-migration
Change-Id: I515ab9d67eb205fe171078826e69d1016ab3ebda
The change to fake_instance here has a couple impacts in the
vmware driver, the test_server_password, and test_nova_manage
tests.
Change-Id: I48c4fbfcac66e9a50b29349cce6824e8985e7205
Also renamed 'vnc_enabled' to 'enabled', and 'vnc_keymap' to 'keymap' now
that those options are prefixed by the 'vnc' group.
Change-Id: I2ff668d4ebd8e6ebf9bf4e8ca5b077958f051144
Closes-Bug: #1447528
This code change implements the conversion of flavor to use object. Rpcapi of
compute manager sends objects for new version and sents primitivized data for
older versions.
Related to blueprint liberty-objects
Change-Id: I11a28cd3b2eb98b42d78f9327ed6b38dc2bc8f30
Sometimes we only want to resize the cpu and ram of the instance,
but nova still resizes and convert disk type even if the disk size
remains the same with the original, which will cost lots of time.
So we need to avoid resizing disk if the disk size doesn't change.
Change-Id: Iea1dd794ae8f9314a16bdf7143e0a639921a4b3c
Closes-Bug: #1454165
Currently, a primitivized object is sent to sync to the API cell
in Instance.save because instance_update_at_top has not yet been
converted to handle objects. This change does the conversion and
makes Instance.save send an object for the sync.
This change should also address a race where deleting an instance
can result in an "undeleted" instance if an update from a child
occurs after the instance has been destroyed at the top, because
in instance_update_at_top() it uses read_deleted='yes' and
db.instance_update() will update all fields provided, unlike
objects which only update fields that have changed.
Closes-Bug: #1460350
Change-Id: I4e8c1a82a3c9c86038faa7f528b9dfb835f82ee6
The get_image_metadata method has some unhelpful semantics
where it takes the image metadata from the instance's
system metadata record, and then overwrites it with the
current metadata associated with the original image.
The result is that, if the image metadata in glance was
changed after the instance was first booted, Nova will
end up making decisions based on image metadata that does
not correspond to that which the instance was booted with.
Since the image metadata controls many aspects of hardware
configuration, this could lead to incorrect behaviour when
modifying hardware later, eg disk/vif/pci hotplug.
What is worse, is that some of the nova operations will
update the instance system metadata when completed, thus
permanently overwriting the original image metadata with
the new data.
Almost all code which operates against an existing instance
is updated to use nova.utils.get_image_from_system_metadata.
The exception is the rescue codepath, which must use the
metadata associated with the new rescue image.
The nova.compute.utils.get_image_metadata method can thus
be removed from use, avoiding the problematic logic.
Closes-bug: #1460079
Change-Id: I35c9d26e3967e93a2c368c3f9fdc807a69816dd2
The block_device_info structure is needed in several places, so it's
useful to move it into a method and remove the code repetition that is
all over the place.
Also make sure there is a single way to get the legacy dict format for
the drivers that need this (however at this point we should probably
remove that soon, and convert all the in-tree drivers to expect the new
format).
Related-bug: 1231874
Partial-bug: 1452224
Change-Id: I608042c58c04f333f6fbb8d9824eb8f3913c6310
Currently, when boot instance from block device, user can choose three
kinds of sources: (blank, snapshot or image), and compute manager will
create a volume from one of them. But the volume creation may fail or
timeout at this time, then the process of instance boot will fail.
During cleanup, the failed volume cannot be deleted, and becomes unusable.
This patch removes the corresponding volume when block device allocation
fails or times out.
Closes-Bug: #1447884
Change-Id: Ib3c915b5b5920c8289d5fc69cde4a3c485ed8600
The oslo_config library supports the choices keyword argument. This commit
leverages the use of choices for StrOpts in Nova's configuration.
This change also adds 'ploop' as a valid value for the images_type option.
This was previously a valid value, but not documented.
Closes-Bug: #1423973
Change-Id: I4c0eb1366cc8a1b001fd403b67c3eec11f831541
After preparing the block device mapping for creating new instance(s),
check if any ephemeral and/or a swap disks have been already defined. If
that is not the case and the instance type requires them, create the
needed ephemeral and/or swap BDMs.
Closes-Bug: #1297325
Related-Bug: #1457527
Change-Id: I44b30625cf1023d20ebec5e38f46b7b8dab325f6
This patch was generated by the sixer tool version 0.2 using the
"iteritems" operation:
https://pypi.python.org/pypi/sixer
Manual changes:
- Don't change get_instance_metadata() in nova/compute/api.py:
fixed by the change Ifd455e70002eb9636b87f83788384127ba6edeeb.
- Don't change sqlalchemy code and
nova/tests/unit/db/test_db_api.py. sqlalchemy objects cannot be
converted to a dictionary using dict(obj) directly yet. It will be
possible with the change I702be362a58155a28482e733e60539d36c039509.
- Revert change in a comment in nova/objects/instance.py; the sixer tool
is limited and don't understand comments
- Reformat nova/virt/vmwareapi/driver.py to respect the 80 columns
contraint
Blueprint nova-python3
Change-Id: I81465661cb8a74778d70ba9b6641073f1effa49b
The function xrange() was renamed to range() in Python 3.
Use "from six.moves import range" to get xrange() on Python 2 and range() on
Python 3 as the name "range", and replace "xrange()" with "range()".
The import is omitted for small ranges (1024 items or less).
This patch was generated by the following tool (revision 0c1d096b3903)
with the "xrange" operation:
https://bitbucket.org/haypo/misc/src/tip/python/sixer.py
Manual change:
* Replace range(n) with list(range(n)) in a loop of
nova/virt/libvirt/driver.py which uses list.pop()
Blueprint nova-python3
Change-Id: Iceda35cace04cc8ddc6adbd59df4613b22b39793
This reverts commit d1baa9fe7e
The change introduced a race on delete where there isn't a host while
prepping block devices and we hit a NoneType error in nova-compute.
There was also a recheck grind on the original change showing it wasn't
safe and needs to be reworked.
Change-Id: Id4e405e7579530ed1c1f22ccc972d45b6d185f41
Closes-Bug: #1456771
Related-Bug: #1448316
Nikola Dipanov