The pick_context_manager method will use a connection to a cell
database if one is present in the RequestContext, else it falls
back on the global main_context_manager in the DB API.
Currently, there are several places in our DB API code where
pick_context_manager isn't used because in a real scenario, each
cell is in a separate process where main_context_manager points
to its local database. This causes problems for testing though,
because we are unable to patch the DB API to simulate switching
between multiple 'main' databases in our functional tests because
of the global nature of main_context_manager.
This replaces all uses of main_context_manager with
pick_context_manager to:
1. Make switching between multiple databases able to work in
functional tests
2. Fix any possible cases where pick_context_manager is not
used for a DB API method that could be called from the
API using target_cell
Change-Id: I31e3170e0953cefbf49bfc84b29edab514c90cb5
Add data migration methods to migrate Aggregates to the API db.
blueprint cells-aggregate-api-db
Co-Authored-By: Dan Smith <dansmith@redhat.com>
Change-Id: Ia2b5ff8046195635b68eddfa7c83d60750ec9932
Make aggregate.create() and destroy() use the API rather than cell database.
Also block aggregate creation until main database empty. This makes
Aggregate.create() fail until the main database has had all of its aggreagtes
migrated. Since we want to avoid any overlap or clashes in integer ids we
need to enforce this.
Note that this includes a change to a notification sample, which encodes
the function and module of a sample exception (which happens to be during
an aggregate operation). Since the notifications are encoding internal
function names, which can and will change over time, this is an expected
change.
blueprint cells-aggregate-api-db
Co-Authored-By: Dan Smith <dansmith@redhat.com>
Change-Id: Ida70e3c05f93d6044ddef4fcbc1af999ac1b1944
Aggregate.save now updates aggregates if they
are located in the api db.
blueprint cells-aggregate-api-db
Co-Authored-By: Dan Smith <dansmith@redhat.com>
Change-Id: I6066857a421bfe489ae70f0e22f338c434195bf9
Adds the ability to update aggregate metadata where the aggregate is
contained in the API db. Modifications will be made to the 'aggregate_metadata'
table in the API db if that is where the aggregate is located or the cell
database otherwise.
blueprint cells-aggregate-api-db
Change-Id: I456beb753298f4aabbed6d20bea7cbc537842e50
Adds the ability to add and delete hosts to Aggregates that are
contained in the API db. Modifications will be made to the
'aggregate_hosts' table in the API db if that is where the aggregate is
located, or the cell database otherwise.
blueprint cells-aggregate-api-db
Change-Id: Icb11e5cbcc743d1ae788c792cffba9de00fddc3f
Make Aggregate.get_by_uuid access the api db first only accessing the
cell db if not found.
blueprint cells-aggregate-api-db
Change-Id: Iaf89c041b6b6990f65e90c5086b13edea1af26f1
db.api.aggregate_get_by_uuid and objects.Aggregate.get_by_uuid
methods are added to make it possible to work with Aggregates by
their newly added uuid attribute.
Partially-Implements: blueprint generic-resource-pools
Change-Id: I7c46fd1ffebb6907c949cfa302131bfbfcd433de
Make the AggregateList get_all, get_by_hosts and get_by_metadata
functions return items from both the API and cell databases.
blueprint cells-aggregate-api-db
Change-Id: I0e1e11283e6ab2f7d2b976d126e914f9d7b4d8fb
This makes the Aggregate object use the API database first only falling
back to the cell database if it is not available.
blueprint cells-aggregate-api-db
Change-Id: Id3616cef1937744bf600763bae5de50f1d27b783
Note that this filters uuid from the aggregate view in the API because we would
need a microversion. That may be a thing in the future (for the 2.x api) but
not something we can do here.
Related to blueprint generic-resource-pools
Change-Id: I45006e546867d348563831986b91a317029a1173
This change removes the NovaObjectDictCompat mixin from the Aggregate
versioned object. All occurrences of key access or get have been changed
to use getattr/setattr. Because none of the objects externally-facing
functionality has changed, the object version does not need to be
updated.
There was a fake aggregate in the xen api tests that contained two
empty dictionaries for a couple of keys in the metadata. These are actually strings in the
code, so they have been changed to empty strings.
Within the tests for the aggregate API, the fake dictionaries that were
returned by the AggregateAPI stubs were structured incorrectly. Within
the object, the availability zone is stored in the metadata dictionary,
not the top-level dictionary. Also, those dictionaries were changed to
Aggregate objects, so the stubs now return true objects (similar to how
the actual code works). The result of the calls are then compared by the
object primitives.
There was usage of .items() in _marshall_aggregate() within the api
controller for aggregates, so a unit test was added to run through that
function to make sure it is transforming the aggregate object correctly.
(props to Tempest for finding that one for me).
Partially-Implements: bp rm-object-dict-compat
Change-Id: Ibcb896b2eef673bce23491161ff394bcc87aa541
In Liberty, we converted to using the manifest-based class
action and backport methods, which no longer rely on these
object relationship maps. They provide a full manifest of
client-side versions with each call, which is far more
robust than us trying to hand-edit these mappings (which
we get incorrect a lot).
Since we're now in Mitaka, we can drop compatibility with
clients that don't make the version-manifest calls. Those
have also been deprecated in oslo.versionedobjects.
Change-Id: Iea8b6348e3b01aaa335ec5408f07826f1e758d20
List objects used to use child_versions, which was tested in a more
shallow way than test_relationships() tests obj_relationships. This
change moves over all list classes to use obj_relationships (and
therefore are now tested in test_relationships() with all other
objects).
Change-Id: I334dba243de45bf11ccb5ab55e2aa8390fe9d8c2
Closes-Bug: #1470154
In oslo.versionedobjects, the registry is opt-in instead of implicit.
This patch sets the stage for moving to that by defining a no-op
registry with a compatible register method so that we can go ahead and
decorate the object classes and then move the registry underneath them.
Related to blueprint use-oslo-objects
Change-Id: Iddc68a2d8a9b103621bc60a7426fe495edf80aad
This is the final step in removing the context parameter from the
signature of a remotable method.
This includes a change of most of the object hashes, without version
bumps. That's because the hashing algorithm is just looking for changes
in things like a call signature, in order to signal that a version bump is
required. Since context is in the signature, removing it triggers the
alert. However, context is not _actually_ part of the on-the-wire API,
as it is stripped out on the caller side, and re-added on the callee side
(hence why we're making this change in the first place). If the test had
been uuber-pedantic to exclude context from consideration, then the hashes
would not be changing here, but alas.
In short, the hash changes are false alarms and do not mean we need
version bumps for all the things.
Related to blueprint kilo-objects
Change-Id: I89464c0ab7e6e0d84e677b9a69a86468727b6438
This removes the use of the context parameter in remotable methods
in these objects. This is a precursor to actually removing the
context from the call, but we need to remove dependence on the
argument first. Remotable methods should be using the bundled object
context.
Related to blueprint kilo-objects
Change-Id: Iad5093622337077418a8ef94c9867dfd7a7fdf64
The existing logic was overly complicated and missed out on those hosts
whose services were disabled.
This is a complete rewrite that makes use of a single aggregate query,
thereby bypassing a lot of the extra logic needed by the old code.
Fixes an issue in objects.AggregateList.get_by_metadata_key() where
filtering by an empty list of hosts will return metadata for all hosts.
Also removes a call to db.aggregate_metadata_get_by_metadata_key() to
avoid the need for special handling due to that method returning
metadata as sets of values instead of strings and also because the same
metadata is already fetched in another method.
Change-Id: I514e63ce863f2c77dcd47af3e3674019033a77de
Closes-Bug: #1419115
The XXXList() classes only ever have a single 'objects' attribute
and this is never directly accessed as a dict key, instead all
callers use it in list context. Thus the dict compat support can
be removed from all these objects
Change-Id: I3f8d49d25b99dd0a498dfaad705c6332908cb72f
The dict compat support in NovaObject is useful for incrementally
converting existing code over to use objects. Any brand new objects
though have no reason to support dictionary access, so there should
be a way to disable this compat mode.
This moves all the dict compat support methods out of NovaObject,
to a new class NovaObjectDictCompat. All existing objects are
updated to add this new class as a mix-in parent.
In future any completely new objects (ie ones which aren't being
used to converted existing code from dict instances) should avoid
inheriting from NovaObjectDictCompat. Existing objects should
also have this parent class removed once all callers are audited
and/or updated to ensure they do not require dict compat.
Change-Id: I03f93f0c40df6f5f7df9cefe28dff900c22294c9
Unlike all object object save methods, Aggregate.save() returns an
aggregate object. This change removes the return to make it consistent
with other objects.
Change-Id: I2179bc90ae92dd3a4f739aae2802a417bd7e2d9d
This change adds the get_by_metadata_key function to the AggregateList object
and the aggregate_get_by_metadata_key function to the DB API.
Previously, the only DB API function available for finding aggregates matching
a metadata key returned a dictionary of aggregated metadata. The
aggregate_get_by_metadata_key function returns all rows matching the specified
metadata key, and is called by the existing aggregate_host_get_by_metadata_key
DB API function.
The get_by_metadata_key function returns a list of Aggregate objects matching
the specified metadata key.
Related to blueprint compute-manager-objects-juno
Change-Id: If50c9f613dd192ab44577f26a81dd5b40e3a7af7
The object code itself should not explicitly reference its own objects,
as objects could be subclassed.
Also: There were a few objects that had their own __init__()s that
didn't support accepting 'context' or kwargs. At some point here, we
should also require that context is passed when instantiating the
objects. So, I added the (currently optional) context argument on some
object instantiations near the places I was touching.
Partial-Blueprint: object-subclassing
Change-Id: Icf631c770e347b4bb32999c4ba2431d1db49e11c
Remove duplicate code updates.pop('id', None) in create() function.
Because if self.obj_attr_is_set('id'):
raise exception.ObjectActionError(action='create',
reason='already created')
already make sure object doesn't have 'id' values, which means that
'id' won't appear in updates.
Change-Id: I582ef8857b32c07ea8ba1eb5fc1e64516110f3ce
When update metadata for a host aggregation, all of the values will
be in the variable of "updates", if remove metadata, the format would
be {"foo": None, "bar": None}; If adding metadata, the format would
be {"foo": "foo1", "bar": "bar1"}.
The current logic of update_metadata() is as following:
1) Get all metadata from variable "updates".
2) Traverse updates to see which values are new added and which needs
to be removed. For the new added values, put the metadata to
variable "to_add"; for the values which need to be removed, just
remove them from metadata.
3) Set the values of variable "to_add" as payload metadata.
The above logic will make the values which was removed from metadata
cannot be in payload, this caused the third party receiver cannot know
which metadata was removed.
The fix was always using the values from variable "updates" as the
metadata for payload as it include values removed and new added.
Change-Id: I7b78795e325f52cfb9c24eabd9ef42990666c02d
Closes-Bug: #1269684
The patch is to remove id when it was set before creating an aggregate
in objects, and check if the aggregate exists before it is created.
Related to blueprint icehouse-objects.
Change-Id: I0d7c5c5da9a93e52c7fee9eee7df80d2f460593e
Right now, a client declares its supported version of a given object
automatically in the remoted calls it makes to conductor. However,
in the case of things like InstanceList.get_by_foo(), they are
reporting the version of their InstanceList object, not their
Instance object. Conductor fills a version-matching InstanceList
object with brand new Instance objects, which the client, of course,
barfs on.
There may be a better way to handle this going forward, but for now,
stop the bleeding by requiring a version bump to the corresponding
List object whenever the object type it contains takes a version
bump. This adds a test to validate that all the objects registered
have a suitable mapping for the current version in the tree.
Since this actually caused a breakage in the Instance object
recently, this also bumps the InstanceList version so that
conductors running this commit (or later) will properly send
version 1.9 Instance objects to Havana clients and version 1.10+
to newer ones.
Change-Id: I2668dead4784fbd0411d1b6372a9a8006eeb2e84
Related-Bug: #1258256
Closes-Bug: #1254902
The object infrastructure has a concept of an object name, which
is what we use to determine if something implements a particular
thing, and it is how we look up object classes when we
deserialize them. Since many objects will eventually reference
each other, we should be using their names to determine
compatibility, not their classes.
Related to blueprint structured-object-fields
Change-Id: I2edb8b8475662ad25e9f3319ce5e44317a2d0547
This simply adds the ability to pass the key attribute to the
by-host lookup for aggregates.
Related to blueprint compute-manager-objects
Change-Id: I7651bc1470845b1e3586aa3616809bdf2ef2b4a2
Originally, all of the objects used str() to specify string type
attributes. This prevents the use of unicode values. A previous
patch errantly updated the utility function without bumping all
of the affected objects' versions. This does that, and also
fixes the issue for bare str() attributes.
Change-Id: Iad83589d399309b95c32813a1a660a12e571066b
Closes-bug: 1221346
Almost every object thus far has done this:
changes = {}
for key in self.obj_what_changed():
changes[key] = self[key]
to get a dict of updates to apply to the database. This patch adds
that as part of the base object and makes every place that does
the above to just use that.
Change-Id: I847f5d35181b0305668b107f86faa164e71c3375
Each object with a list duplicated the _make_list() method in its
own module. This removes that duplication and adds a generalized
helper in objects/base.py. The instance object still uses its own
because it has to do a bunch of other stuff in the loop for
efficiency.
Change-Id: Ic910f39087ebc167b2b930979f7951116caf8598
Creates a new DBObjectMixin class for DB-backed objects to use. This
allows us to create non-DB-backed objects without having the common DB
fields automagically added to them.
Related to blueprint unified-object-model
Change-Id: Iaf2d8500505e9acdbffffb1d4bd3db0870ae82a3