The 'prep_resize()' method of compute now accepts a 'host_list'
parameter that supplies alternate hosts for retrying. This required
bumping the RPC version to 4.21 for compute. The
MigrationTask._execute() method is also changed to request alternates
from the scheduler.
Blueprint: return-alternate-hosts
Change-Id: If6a0bb766e70ab6f1c313da38bb2b0756a2e8772
The earlier patches in the series generated alternates and Selection
objects, and modified the RPC calls to send them to the conductor. This
patch has the conductor pass these host_lists to the compute for the
build process, and, if the build fails, has the compute pass the
host_list back to the conductor.
Also fixes a bug in the scheduler manager exposed by this change
when using the CachingScheduler in a reschedule functional test.
Blueprint: return-alternate-hosts
Change-Id: Iae904afb6cb4fcea8bb27741d774ffbe986a5fb4
There are two things that call the conductor task API's
build_instances method:
1. nova-api when using cellsv1
2. nova-compute when rescheduling during a build failure
Since we want to eventually replace the filter_properties
usage in the compute service with the request_spec, this
allows passing the request_spec from the compute service
during a reschedule.
For now we have to continue munging that RequestSpec a
bit using the filter_properties for retries and the
next chosen host from the scheduler, but that can eventually
go away. With this, the computes can start to at least
pull scheduler_hints out of the request_spec rather than
the filter_properties, which is one more step in removing
the compute's dependency on filter_properties.
Note that we don't pass the request_spec from nova-api
for cells v1, simply because cells v1 is deprecated.
Part of blueprint request-spec-use-by-compute
Change-Id: Ie5233bd481013413f12e55201588d37a9688ae78
This is a preliminary change for the counting quotas series to make
any legacy quota "reservations" arguments default to None so that in
the next change, we can simply not pass any reservations instead of
passing empty reservations. Reservations are being deprecated as part
of the counting quotas series.
Part of blueprint cells-count-resources-to-check-quota-in-api
Change-Id: I86e56c1abd979ac9968a3ce7daad07530080bde4
This is the third patch of the series,
this patch adds a new rpc version to
ComputeTaskManager to accept tags
as a parameter in schedule_and_build_instances()
Change-Id: Iac54b9627cb4398e45f1f15a2f4e7d6f86242093
Implemetes: blueprint support-tag-instance-when-boot
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who want to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds the first wsgi trace point with info about the HTTP request
* Add initialization of osprofiler on start of a service
Currently that includes oslo.messaging notifier instance creation
to send Ceilometer backend notifications.
oslo-spec: https://review.openstack.org/#/c/103825/
python-novaclient change: https://review.openstack.org/#/c/254699/
based on: https://review.openstack.org/#/c/105096/
Co-Authored-By: Boris Pavlovic <boris@pavlovic.me>
Co-Authored-By: Munoz, Obed N <obed.n.munoz@intel.com>
Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com>
Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com>
Implements: blueprint osprofiler-support-in-nova
Change-Id: I82d2badc8c1fcec27c3fce7c3c20e0f3b76414f1
This patch is part of refactoring to move instance saving to the
conductor. It only creates new conductor manager method to schedule and
build instance.
The new method does quite the same thing as build_instances but has some
differences:
- build_requests and request_spec are used instead of
instances param
- it's cell aware i.e. it creates instance and related
objects in the appropriate cell
- in the case of failure method creates instance in cell0
Co-Authored-By: Dan Smith <dansmith@redhat.com>
Change-Id: I19d6d765894785b214458b32fe612b4ef78a270d
Implements: bp cells-scheduling-interaction
This adds upgrade level aliases for newton for all of our services,
including the ones we expect to remove soon.
Change-Id: If97f8f87f3d8b4db8f6b6b826c5c1f5d0b3f10a8
This change makes the checks that happen before a live migration
non-blocking so that the user can receive an HTTP accepted and
later check the results of the live-migration via instance-actions.
bp: async-live-migration-rest-check
Change-Id: I4f788d48ec0e166898184842ac55560163c1b343
There are some misspell words in nova,
availabilty (availability)
Synchronise (Synchronize)
evacaute (evacuate)
leagacy (legacy)
doen't (doesn't)
Assocate (Associate)
Change-Id: I9a5e85278c94ba830d94ac15e0b7b8e3efc8e4d8
Closes-Bug: #1579997
Move config options of nova.conf section "upgrade_levels" to new
centralized location nova/conf/upgrade_levels.
Also, previously some config options for "upgrade_levels" were scattered
in different conf files. Moved these options to the central
location: nova/conf/upgrade_levels.
Change-Id: I45651e8527dd98635f25e9586f68c288203d8cf5
Implements: blueprint centralize-config-options-newton
Per our usual procedure, this sets version aliases for the current RPC
versions of all our internal RPC APIs for mitaka.
Change-Id: I455f3b80a9b64dfea42ee1c74685d11ae0178513
Now that we have done this for live-migrate and evacuate, we can modify
the unshelve conductor method to get the RequestSpec.
Partially-Implements: blueprint check-destination-on-migrations
Change-Id: I8d17c7eb908c6aa614744058ed7e483ce6afe27d
Since the RequestSpec object is now persisted in the API layer every time that
we start an instance, we can now fetch it from the API DB and pass it thru the
ComputeTask API to the conductor so it can directly call the scheduler with it.
Partially-Implements: blueprint check-destination-on-migrations
Change-Id: I9d6472220b941072eb3f998c949755c967fb71f6
This change moves all of the configuration options previously defined
in nova/conductor/api to the new centralized nova/conf directory.
Blueprint centralize-config-options
Change-Id: I8c420ac1de16d18613062112b2d86a5f66149dbf
Since the RequestSpec object is now persisted in the API layer every time that
we start an instance, we can now fetch it from the API DB and pass it thru the
ComputeTask API to the conductor so it can directly call the scheduler with it.
NOTE(sbauza): Yeah, it's ugly to see the ReqsSpec object being dehydrated to
legacy dicts and then rehydrated by the next call, but I'll modify that in my
next change.
Partially-Implements: blueprint check-destination-on-migrations
Change-Id: I7fe694175bb47f53d281bd62ac200f1c8416682b
This is unused and can be cleaned out. The manager method is left for
backwards compatibility but changed to return an empty list since that
yields the same result as before. The db query always resulted in no
rows being returned since the table contained nothing.
Change-Id: Id5f0406c64f9a6ef9e4220cd6e6d2facbf40a550
Update all of the rpc client API classes to include a version alias
for the latest version implemented in Liberty. This alias is needed when
doing rolling upgrades from Liberty to Mitaka. With this in place, you can
ensure all services only send messages that both Liberty and Mitaka will
understand.
Change-Id: I3eabae03483806602d7dea0e4ab737db4917d0fd
This commit adds a new major revision to the rpc interface used to
talk to the conductor service. The old version will be retained in
Liberty for Kilo rolling upgrade capability. 2.x can be dropped as soon
as Mitaka development opens.
When 2.x gets dropped, we will be able to remove the following methods
from the conductor manager, as well as all other supporting code for
these methods throughout the conductor:
- instance_update()
- instance_get_by_uuid()
- instance_get_all_by_host()
- migration_get_in_progress_by_host_and_node()
- aggregate_host_add()
- aggregate_host_delete()
- aggregate_metadata_get_by_host()
- bw_usage_update()
- agent_build_get_by_triple()
- block_device_mapping_update_or_create()
- block_device_mapping_get_all_by_instance()
- instance_get_all_by_filters()
- instance_get_active_by_window()
- instance_destroy()
- instance_fault_create()
- vol_usage_update()
- service_get_all_by()
- action_event_start()
- action_event_finish()
- service_create()
- service_destroy()
- compute_node_create()
- compute_node_update()
- compute_node_delete()
- service_update()
- task_log_get()
- task_log_begin_task()
- task_log_end_task()
- notify_usage_exists()
- security_groups_trigger_handler()
- security_groups_trigger_members_refresh()
- network_migrate_instance_start()
- network_migrate_instance_finish()
- quota_commit()
- quota_rollback()
- get_ec2_ids()
- compute_unrescue()
- object_class_action()
- object_backport()
Note that the client side will only send 3.0 messages, but that's
fine. We expect the nova-conductor service to get updated at the same
time as, or before anything that uses this client code.
For more information on this type of change, see:
https://wiki.openstack.org/wiki/RpcMajorVersionUpdates
Related to blueprint liberty-bump-object-and-rpcapi-versions
Change-Id: Ia976c7f3362df9d91b2e7d23f0465b8d57149714
Two recent changes[1][2] related to objects support for manifests-based
versions replaced calls to existing methods object_backport() and
object_class_action(), leaving them unused. This will clean out the old
client-side api:s and prepare for the removal of server-side api:s in an
upcoming conductor major version bump.
Since oslo.versionedobjects still need some work before the client-side
of object_class_action() can be removed, add a TODO about removal once
the needed changes are in place.
[1] commit d5f59c1a32
[2] commit 452053e8c7
Related to blueprint liberty-bump-object-and-rpcapi-versions
Change-Id: I9158920152b69d80e8d452d7d1a3187cba3f81f1
Now that the one and only call to create a compute node record in
resource tracker has been converted to objects[1], the old method
in conductor can be cleaned out.
[1] commit 2ef014eb31
Related to blueprint liberty-bump-object-and-rpcapi-versions
Change-Id: I6f8043efcc5928d7e42bc32b638bbd4e782a1e2a
This adds a new object_class_action_versions() handler in conductor, much
like the manifest-aware backport method recently added. Since we back-convert
object results from remotable_classmethod operations, we need to receive
the version manifest there as well. Since oslo.versionedobjects doesn't
currently pass that, we can do a little indirection in our rpcapi to start
calling the new method, gathering the manifest first.
Related to blueprint liberty-bump-object-and-rpcapi-versions
Change-Id: I25751edee551304ec849d7b88e42624970fef45f
This adds the conductor implementation required to support version-manifest
backport operations. Ideally we will drop all of the relationship mapping
stuff after we released this in Liberty and roll to Mitaka. We need to
support both operations here to bridge that gap.
Related to bp/use-oslo-objects
Change-Id: I3702c02c53876a5b593ba69c6ebd607e9b6b7152
Calls that used to proxy through conductor method instance_update()
has been converted to objects, so remove from conductor.
Change-Id: I169998c0171480c366d7cb5b66f3c5a828b4180d
Now that all calls that previously used conductor vol_usage_update()
have been converted to objects, lets clean out the unused code in
conductor.
Change-Id: I7d7930fbcc748587d3278c128802dcebedbba892
This change reverts commit b1838c258d.
This makes network manager go back to issuing security group refresh
calls directly instead of going through conductor now that the
compute api call it is making is converted to objects.
This also removes and deprecates the conductor proxying code that
becomes unused after this change.
Related to blueprint liberty-objects
Change-Id: I74e002384b11063e30450e3854a95b6b848ddfb6
Now that all calls previously using conductor task_log_*() calls have
been converted to objects, these methods can be cleaned out.
Change-Id: I070e2d4dceed714e9ac99d26f5fdcef7886c5a1f
There are some conductor api methods left unused that can be removed
together with two very old manager methods that were deprecated and
scheduled for removal with the rpc api version bump to 2.0.
This change removes the following unused api client methods:
- instance_get_all_by_host()
- instance_get_all_by_host_and_node()
- migration_get_in_progress_by_host_and_node()
- aggregate_metadata_get_by_host()
- block_device_mapping_create()
- block_device_mapping_update()
- block_device_mapping_update_or_create()
- block_device_mapping_get_all_by_instance()
- compute_node_update()
- compute_node_delete()
- security_groups_trigger_handler()
Furthermore, it removes these previously deprecated manager methods:
- ping()
- instance_get_active_by_window()
Manager methods that are deprecated and scheduled for removal on the
next rpc version bump are annotated with a todo comment if not there
already.
Change-Id: Ia825cee94e6ba9431a85c1b6f64cb26e7c83c95f
Update all of the rpc client API classes to include a version alias
for the latest version implemented in Kilo. This alias is needed when
doing rolling upgrades from Kilo to Liberty. With this in place, you can
ensure all services only send messages that both Kilo and Liberty will
understand.
Change-Id: I2952aec9aae747639aa519af55fb5fa25b8f3ab4
Closes-Bug: #1444745
A previous change converted service and servicegroup to objects.
This change removes the now unused service methods from conductor.
Change-Id: I3e35598ec89b18d1121547caa37ec5dbc506cb06
A previous change converted the one use of this method to objects so
remove and deprecate the now unused method in conductor.
Change-Id: I22681b6cac638d471519eecc0b1ebec84664a72f
In Havana, commit 3cf4cb253 moved the volume usage notify code to
conductor.
In Juno, commit 47f6ffad8 dropped vol_get_usage_by_time from conductor
manager but not from the conductor api/rpcapi modules. Without
vol_get_usage_by_time being in conductor manager it's an API pointing to
nothing and won't work so we should remove it.
The DB APIs are left in since the vol_usage_update DB API is tested
using vol_get_usage_by_time.
If we were to leave the conductor APIs in tree, we should add the
conductor manager method back in and convert the volume_usages table to
nova objects, and possibly also expose it via a v2.1 API extension.
Closes-Bug: #1434687
Change-Id: Ie2980d681c62cd3457b359fbe4f5c32a2bcfc5db
Conductor methods bw_usage_update() and notify_usage_exists() are
no longer used and can be removed/deprecated.
Change-Id: I55a957867c80666d9f8622db84e27e33466c68ab
Once we have consolidated all the block device information for an
instance coming from different sources (request, image metadata) we turn
them into objects and use objects update_or_create() for saving them into
the database.
Blueprint: kilo-objects
Change-Id: I9a70761d6a0a448cfbaf7102c45d6dc180a4cdea
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
Change Iec3dfd17725440958aac395ebc471e51afd6522e added
clean_shutdown support to the compute manager in Juno.
This change is part of a set to extend that to the compute
API layer, which will in turn allow us to extend the REST API
to provide a forced-shutdown option to those operations.
Change-Id: Ie93c3c4aa30b16e277628e1a741e46a37b794a75
Partially-Implements: blueprint user-defined-shutdown
This change completes the conversion of flavors to use objects instead
of direct database access. The rpcapi sends objects for the new version
and primitivized data is sent for older versions.
Related to blueprint kilo-objects
Change-Id: Ib876f276cff99a21c5bca1c9f062a6c221a783e8
This makes the notify_usage_exists() method for conductor take an
Instance object instead of a dict primitive.
Related to blueprint kilo-objects
Change-Id: I932c46e20f1d203f9587a39381563d58a10a3485
This change converts flavors to use objects instead of direct database
access. The rpcapi primitivizes objects before they go over the wire.
The flavors.add_flavor_access() and flavors.remove_flavor_access()
functions were also removed as flavor object add_access() and
remove_access() should be used instead.
Related to blueprint kilo-objects
Change-Id: I86432720b600e479d8d8b7f7531a2c85ab364319
Very simple import change in each file, just touches a
whole lot of files. jsonutils has graduated into a stand alone
library and has been removed from the oslo-incubator repository.
We should be using the library for all projects.
Change-Id: I09d780a47d377871f357654603ee4b0fa0169d66
nova.servicegroup.drivers.db.DbDriver._report_state() is called every
service.report_interval seconds from a timer in order to periodically
report the service state. It calls self.conductor_api.service_update().
If this ends up calling nova.conductor.rpcapi.ConductorAPI.service_update(),
it will do an RPC call() to nova-conductor.
If anything happens which causes the RPC reply to be lost or
never sent in the first place, by default the RPC code will
wait 60 seconds for a response (blocking the timer-based
calling of _report_state() in the meantime).
This is long enough to cause the status in the database to get old
enough that other services consider this service to be "down".
if rpc_reponse_timeout is smaller than report_interval then we could
use the existing RPC timeout, but wait longer won't hurt. So the patch
didn't check it and only use report_interval.
Change-Id: I88743183bce1a534812cfe6110c3fc2892058c53
Closes-Bug: #1368989
Update all of the rpc client API classes to include a version alias
for the latest version implemented in Juno. This alias is needed when
doing rolling upgrades from Juno to Kilo. With this in place, you can
ensure all services only send messages that both Juno and Kilo will
understand.
Change-Id: Ia81538130bf8530b70b5f55c7a3d565903ff54b4
This makes the conductor and compute RPC paths for building instances
support and use NetworkRequest objects, which gets this all the way
down to the compute managers.
Change-Id: Ia45e15896b80ae33533d77719dd23bbdf60a1945
This was not used in v2.0 of the RPC API, so we can remove it
outright without deprecating it in conductor manager.
Related to blueprint compute-manager-objects-juno
Change-Id: I68068c1d0ef8f346406fe3ccb341a8d0a2474bdd
Ed Leafe