The code to generate a support matrix has been pulled into a common
library. Using this instead of duplicating code in various projects that
need it.
Change-Id: If5c0bf2b0dcd7dbb7d316139ecb62a936fd15439
Co-Authored-By: Stephen Finucane <stephenfin@redhat.com>
When an allocation is created with PUT /allocations/{consumer_uuid} the
consumer_uuid is not validated as UUID in the API and the DB schema only
checks that is fits into String(36). However the Consumer object stores
this value in a UUIDField oslo only raises a warning today.
This patch adds a UUID validation for that URL path.
Change-Id: Idedd85ca9266f10ce09231c34cab7ca43029a56c
Closes-Bug: #1780238
This is the first change that implements basic virt.driver methods
to allow nova-compute process start successfully.
A set of subsequent changes will implement spawn, snapshot, destroy
and instance power actions.
Change-Id: Ica6117c2c64f7518b78b7fb02487622250638e88
blueprint: add-zvm-driver-rocky
The aggregate.update_prop.start and aggregate.update_prop.end
notifications has been transformed to the versioned notification
framework.
Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: I37b19573b6d0e1131c446fcec361f01fa2560f82
Implements: bp versioned-notification-transformation-rocky
Ever since we introduced support for setting multiple consumers in a
single POST /allocations, the AllocationList.delete_all() method has
been housing a latent bad assumption and bug.
The AllocationList.delete_all() method used to assume that the
AllocationList's Allocation objects were only ever for a single
consumer, and took a shortcut in deleting the allocation by deleting all
allocations with the "first" Allocation's consumer UUID:
```python
def delete_all(self):
# Allocations can only have a single consumer, so take advantage of
# that fact and do an efficient batch delete
consumer_uuid = self.objects[0].consumer.uuid
_delete_allocations_for_consumer(self._context, consumer_uuid)
consumer_obj.delete_consumers_if_no_allocations(
self._context, [consumer_uuid])
```
The problem with the above is that if you get all the allocations for a
single resource provider, using
AllocationList.get_all_by_resource_provider() and there are more than
one consumer allocating resources against that provider, then calling
AllocationList.delete_all() will only delete *some* of the resource
provider's allocations, not all of them.
Luckily, the handler code has never used AllocationList.delete_all()
after calling AllocationList.get_all_by_resource_provider(), and so
we've not hit this latent bug in production.
However, in the next patch in this series (the reshaper DB work), we
*do* call AllocationList.delete_all() for allocation lists for each
provider involved in the reshape operation, which is why this fix is
important to get done correctly.
Note that this patch renames AllocationList.create_all() to
AllocationList.replace_all() to make it absolutely clear that all of
the allocations for all consumers in the list are first *deleted* by the
codebase and then re-created. We also remove the check in
AllocationList.create_all() that the Allocation objects in the list must
not have an 'id' field set. The reason for that is because in order to
properly implement AllocationList.delete_all() to call DELETE FROM
allocations WHERE id IN (<...>) we need the list of allocation record
internal IDs. These id field values are now properly set on the
Allocation objects when AllocationList.get_all_by_resource_provider()
and AllocationList.get_all_by_consumer_id() are called. This allows that
returned object to have delete_all() called on it and the DELETE
statement to work properly.
Change-Id: I12393b033054683bcc3e6f20da14e6243b4d5577
Closes-bug: #1781430
The change items in this patch as below:
1. Add policy and rules to InstanceGroup, deprecate
the policies field.
2. Also, make _from_db_object and _create_in_db method to
support new "policy" field.
The internal usage of InstanceGroup.policies is converted to
use the new model in the REST API and RequestSpec compat code.
Related to blueprint complex-anti-affinity-policies
Change-Id: Ib33719a4b9599d86848c618a6e142c71ece79ca5
We have two kinds of generation conflicts that happen (for both
resource providers and consumers). In
I5a164a0ee298846231113ad7b62b82a3e8f1f409 error codes are added for
conflicts that happen when incrementing the generation in the database.s
In this change the codes are added for those situations where in code we
are checking an incoming generation against the stored generation.
Since there are incoming genearations in this case, we can add tests
so this does, via gabbi.
Change-Id: I5ed005068c1c19194ed67556376604ae90727412
Add a gabbi test case demonstrating that the consumer with UUID
{8}-{4}-{4}-{4}-{12} is not the same as the consumer with UUID
{8}{4}{4}{4}{12}.
Change-Id: I1852ac6004fedc5dfa9dd3de187c937ab385d1b5
Related-Bug: #1758057
Bump the minimum version of oslo.config to 6.1.0, which adds proper
support for parsing Opt.help as rST [1]. This in turn allows us to
revert commit 75fc30090133c31316e6ae790568f1b622807d0c, which is a
temporary fix relying on deprecated features of Sphinx.
[1] https://review.openstack.org/#/c/553860/
Change-Id: I8f56bdce37cfc538348490052a24e463164c86a3
A pending api-sig guideline
I473918ce9c6b49c0b904e93b7140421525f4e7c0 reflects discussion amongst
various parties that adding error codes to structured error responses
should not be considered a breaking API change.
Given that, it is simply "okay" to add them. This change adds
errors.CONCURRENT_UPDATE wherever the ConcurrentUpdateDetected results
in an HTTPConflict (409) response.
In practice this means that if microversion 1.23 or beyond is being
used, there will be a code in any error response, and for some
409 responses that code will be `placement.concurrent_update`.
Note that because we haven't instrumented a way to cause a truly
concurrent update, we've never had real tests for this kind of
thing.
Change-Id: I5a164a0ee298846231113ad7b62b82a3e8f1f409
This change provides a small number of updates to the placement user and
contributor documentation to reflect some of the things that have
changed recently. This is by no means complete, but is an improvement
over what was there.
Partial-Bug: #1778227
Change-Id: Ia348cd3c99b1a5104e15595fdebc83e1ca582a98
The removed variable is probably a leftover from experimenting with
different strategies for getting the migration to work.
Change-Id: I500aa048ec701c69d1af40784d79bc8b79f559c3
Address various minor issues from
https://review.openstack.org/#/c/565604/
Change-Id: I69df4c8d8c4b8813f78aeeb46f7b788d36238d35
Blueprint: add-consumer-generation
Adds code to the util.ensure_consumer() method that updates the
consumer's project and user foreign key identifiers when a user supplies
a different external project or user identifier in the paylod of PUT
/allocations/{consumer_uuid} or POST /allocations.
Change-Id: I9368ad2280551ffa9a011d3a30c4b45861305455
Closes-bug: #1779717
Related-Bug: #1781008
nova/db/__init__.py was importing * from nova.db.api. This meant that
any time any code anywhere within the nova.db package was imported
then nova.db.api was too, leading to a cascade of imports that may
not have been desired. Also, in general, code in __init__.py is a pain.
Therefore, this change adjusts code that so that either:
* nova.db.api is used directly
* nova.db.api is imported as 'db'
In either case, the functionality remains the same.
The primary goal of this change was to make it possible to import the
model files without having to import the db api. Moving the model files
to a different place in the directory hierarchy was considered, but
given that "code in __init__.py is a pain" this mode was chosen.
This looks like a very large change, but it is essentially adjusting
package names, many in mocks.
Change-Id: Ic1fd7c87ceda05eeb96735da2a415ef37060bb1a
When updating a parent provider of a resource provider, placement
didn't update a root provider of another resource provider in the
same tree.
This patch fixes it to update the root provider field of all the
resource providers in the same tree.
Change-Id: Icdedc10cdd5ebfda672ca2d65a75bf0143aa769c
Closes-Bug: #1779818
This column is being added to have information regarding an instance
being active/deleted in the nova-api database as a part of the handling
a down cell spec.
Related to blueprint handling-down-cell
Change-Id: Ic6253acab2ae08025d65add9a5be9ed463457290
When we fail to create allocations for new consumers (either when
issuing a PUT /allocations/{new_consumer_uuid} or a POST /allocations
where the payload includes a new consumer UUID), we need to ensure that
we delete the Consumer object and underlying record in the consumers
table that gets auto-created before calling AllocationList.create_all().
This auto-created consumer record is what is used to compare things like
consumer generation in later calls to PUT|POST /allocations, and this
phantom consumer record was causing confusion when normal retries (for
things like 409 Conflict due to concurrent provider or inventory
updates) would be rejected stating that the expected consumer generation
was 0 and not null (null being the sentinel that indicates the caller is
expecting the consumer is a new consumer).
Change-Id: If37ef318bd5482a2d19928002c6f1fa24932946f
Closes-bug: #1779725
Closes-bug: #1778576
We made the decision [1] to delete consumer records when those consumers
no longer had any allocations referring to them (as opposed to keeping
those consumer records around and incrementing the consumer generation
for them).
This patch adds a small check within the larger
AllocationList.create_all() and AllocationList.delete_all() DB
transactions that deletes consumer records when no allocation records
remain that reference that consumer. This patch does not, however,
attempt to clean up any "orphaned" consumer records that may have been
created in previous calls to PUT|POST /allocations that removed the last
remaining allocations for a consumer.
[1] https://goo.gl/DpAGbW
Change-Id: Ic2b82146d28be64b363b0b8e2e8d180b515bc0a0
Closes-bug: #1780799
Unfortunately, when I added the CONF.incomplete_consumer_project_id and
CONF.incomplete_consumer_user_id options, the default value for those
options was an invalid UUID (it had 13 zeroes for the last part of the
UUID instead of 12). This is preventing some work that tries to disable
warnings about invalid UUIDs being stored in ovo UUIDField objects.
The work to "heal allocations" will be able to fix up any deployments
that deployed the invalid UUID default CONF value because it looks for
any instances in Nova where the placement allocation records don't have
a matching user/project and replaces the mismatched user/project IDs on
the consumer appropriately.
Change-Id: I67b3b771f20e0b83225fce4839d378696fe1ab24
Closes-bug: #1780107
A group can only have 1 policy associated it, but now we have
ONE to MANY model in group model, this patch change it to ONE
to ONE Model.
We also adapt create method to new model, and remove policy
update support in group.save because now we don't support to
update a policy of the group.
Related to blueprint complex-anti-affinity-policies
Change-Id: I95eb1c8470a567408696132ae656305be53dc5bb
This adds the rules column to instance_group_policy table.
The ''Text'' column "rules" which is a dict, it can be
applied to the policy, and represents the rules for
specific policy.
Related to blueprint complex-anti-affinity-policies
Change-Id: I61ffb5ddb2d808bfef4e60088f4524bd98e0474e
We can't use assertJsonEqual here even if that function deserializes
the nested connection_info_json. When deserializing the "bdms" entry
in the migrate_data object, it creates a random order on the list
which can still cause mismatch errors.
So rather than rely on assertJsonEqual handling this, that change
is reverted and we simply deserialize the connection_info_json
in the test in question before comparing to our expected data.
This reverts commit ac9acce283a28326c6f9af4a158e6f3c1e09db45.
Change-Id: I7bba91e490a9ac5d0bbffa90e9cbefe60bb31f27
Closes-Bug: #1779711
There are a few places in the resource_provider.py placement
API handler module that need to be updated to return custom
error codes.
Change-Id: I7efdf7f82d581cf5dc9a3decc063b69c8cef22ff
Closes-Bug: #1778929
It seems gabbi does not resolve yaml refs in the URL part of a REST
call. This causes that the tests in ensure-consumer.yaml uses invalid
consumer id '*consumer_id'
This patches replace the *consumer_id with a fixture ref that works.
Change-Id: I0576f282b1801340db6827fb4c9520a0d48ad7f0
Related-Bug: #1780238
