Add support for checking min_unit, max_unit and step_size when
making allocations to the placement API. When the constraints are
violated a new exception InvalidAllocationConstraintsViolated is
raised.
Change-Id: I18596a3c0f2b0049aaccd0f3e73aef90b684c4a8
Closes-Bug: #1623545
This is the initial set of docs for the placement API
service introduced in the Newton release.
We still have to flesh out the API reference in detail
but this gets us started.
The deployment steps are taken from how devstack does
this today.
Change-Id: Ic2436d92a7cefaeb1ae67ed878da968444f2f18d
The following warning appears in the unit test logs a number of times.
"UserWarning: This test uses methods that set internal oslo_db
state, but it does not claim to use the database. This will conflict
with the setup of tests that do use the database and cause failures
later."
Note that this warning is only emitted once per unit test worker, so new
offenders will show up in the logs each time you fix a test until they
are all gone.
This patch is intended to be merged after all the existing warnings are
addressed by other patches in this series:
https://review.openstack.org/#/q/topic:bug/1568414
Change-Id: I91909a356187bac70fa2fa920cebf3888410ae01
Closes-Bug: #1568414
Add informative messages to the 404 error response from
nova/objects/resource_provider.py
For the method _update_inventory_in_db() from resource_provider.py,
gabbi test validation is not added since it is not being used and
from the placement API you create an inventory but then call
set_inventory or add_inventory on a resource_provider.
Similar change, i.e. adding informative messages to 404 error can be
done for api tests from inventory.yaml and usage.yaml
It includes adding informative logging in code for 404 exceptions and
adding gabbi tests for those.
This can be done in follow up patch.
Change-Id: If53f84ac5f7521e9926b97bdcce3cf77ec5b4ffd
Closes-Bug: #1634115
In order to facilitate a future extraction of the placement service
we want to record the association between a resource provider and an
arbitrary aggregate uuid in its own table.
A PlacementAggregate model is joined from ResourceProvider via
ResourceProviderAggregate. Note that this structure is used so we can
join on ids instead of strings (the uuids). A direct mapping between
ResourceProvider uuid and Aggregate uuid was mooted earlier in the year
but was determined to be suboptimal.
The name 'placement_aggregates' is used as the least problematic of
several choices after discussion amongst several parties.
The data will be used by the forthcoming get_ and set_aggregates
methods on the ResourceProvider object.
Change-Id: Id0355cb022f68e962af306ff04cf724d22b68d19
Partially-Implements: blueprint generic-resource-pools-ocata
A number of tests in test_compute_mgr.py patch the event reporter to
prevent database poison warnings but otherwise don't assert on the event
reporter behaviour. Replace those individual mocks with a fixture that
stubs out the event reporter in the test setUp.
Change-Id: I57233259065d887b38a79850a05177fcbbdfb8c3
Related-Bug: #1568414
The ResourceProvider.set_inventory() method was not raising NotFound
when an Inventory object in the supplied InventoryList object was using
a resource class that did not exist in the resource class cache. This
patch fixes that by having the string_from_id() and id_from_string()
methods of the resource class cache raise ResourceClassNotFound instead
of returning None. We raise specific ResourceClassNotFound and
InventoryWithResourceClassNotFound exception subclasses to differentiate
in the inventory API handler between an unknown resource class and a
known resource class that doesn't exist in a given resource provider's
inventory.
We return a 400 Not Found when InventoryWithResourceClassNotFound is
raised on updating a single inventory, but return a 409 Conflict when
InventoryWithResourceClassNotFound is raised on updating a set of
inventory records since in the latter scenario, it represents a race
condition that frankly should not occur.
Change-Id: I350b02dcdbaa9d30d885cd085f60daa6b53a7745
Prior to Ib563b0ea the passphrase used by CryptsetupEncryptor and
LuksEncryptor had any leading zeros per hexadecimal digit removed, for
example 0x04 or 04 would turn into 0x4 or 4. As a result any volume
encrypted prior to the release of Newton used a modified passphrase that
was different to that stored by the key manager being used in the
environment.
To correct this for LuksEncryptor volumes permission denied errors are
now caught when attempting to open a volume. A second attempt to open
the volume is then made using a mangled passphrase. If successful the
correct passphrase is then added to the volume before the mangled
passphrase is finally removed. This workaround can be removed in a
future release once it is safe to assume that all LuksEncryptor volumes
have had any mangled passphrases replaced in this way.
This isn't possible for CryptsetupEncryptor volumes as the plain mode
used by cryptsetup does not provide a way for adding and removing keys.
As such on a permission denied error a second attempt is made to open
the volume using a mangled passphrase. Unlike the above workaround this
cannot be removed in a future release.
Change-Id: I7096463c5eba951dd6322ee6965435e877ca0371
Partial-bug: #1633518
This sets up a basic framework for policy checking in the placement API
and replaces the current admin check.
There are some limitations here based on oslo.policy limitations. It
turns out that policy rule defaults can only be registered if a policy
file is also used. Since we don't have immediate plans for a placement
policy file the issue is avoided by loading the necessary rule when
policy.Enforcer is initialized. There is also no support for generating
policy files using the oslopolicy CLI scripts. This can all be addressed
later when we determine a path for separating placement policy from Nova
or merging it together.
Although there is an existing nova.policy the changes for placement
policy are implemented in nova.api.openstack.placement.policy to
ease any later extraction that may happen. This is straightforward
because the placement policy code doesn't use any code from
nova.policy.
Co-Authored-By: Chris Dent <cdent@anticdent.org>
Change-Id: Ia62158e0eed50a34114718ee724b038f577c7e87
Partially-Implements: bp placement-api-policy-authz
The build_requests.instance column is a serialized
instance object, and the instances.user_data column
is MediumText, so the build_requests.instance column
itself needs to be at least MediumText in size for MySQL.
Change-Id: I7d65df37c02750593037744543ad15e5bc64e913
Closes-Bug: #1635446
Add the following notification when swapping volumes.
* 'instance.volume_swap.error'
Change-Id: I90d4ffcb2ffc318de2365a655b5fde8bb6c05ff2
Implements: blueprint add-swap-volume-notifications
Provide a decorator that allows Placement API handlers to
express a microversion window within which that method should be
used. This makes it possible to define N different methods (with the
same name) to present different behaviors dependent on what
microversion is requested by the user-agent.
If there is no match between a valid requested microversion and
available versioned handlers a 404 response is returned.
The mechanism for managing the versioned handlers is borrowed
from the compute api but adjusted to allow for the fact that
placement handlers are module level functions. Some code is
borrowed from twisted to get fully qualified names of methods for
the VERSION_METHODS dictionary.
Tests are added which validate that versioned handlers do not
intersect. In the rest of nova this is done at runtime, but
in the placement context this isn't necessary. Doing it at testime
is sufficient. Since there are currently no versioned handlers
in addition to the faked tests of the intersection handling I
also manually forced some tests with real handlers, just to be
sure.
Change-Id: I713c139ee12bb7f5301edd85951f8960fda84ac3
Fixes volume related unit tests.
Fixes non-sortable None items.
Fixes __getattr__ infinite recursion.
Fixes is_dict_like method. Dicts in python 3.4 do not
have the 'has_key' method.
Partially Implements: blueprint goal-python35
Change-Id: I97efc09f7657436f706b08e0b2795f0e59ac1dcd
Found a small bug in the handling of a 406 error by the placement API's
microversion handling code that only appeared when more than one
microversion was used.
Change-Id: I8562e9ce962352033e43a3b993f8b8ba6f211fd4
Remove gabbi tests which are reporting success or failure depending
on the environment they are ran in as this was misleading.
Change-Id: Ifb06b353e4264f4276ebb294a089c4fede74067f
This patch modifies the fields.ResourceClass field type from an
EnumField to a StringField. This keeps the over-the-wire format of the
field backwards-compatible while allowing us to add non-standardized,
custom resource classes to the new resource_classes database table.
We change all locations of fields.ResourceClass.index() and
fields.ResourceClass.from_index() to use the ResourceClassCache object
added in the previous patch in this series and add some check logic in
Inventory and Allocation object's obj_make_compatible() methods to
ensure backversioned objects requesting or sending with a resource class
string different than the set of strings in the ResourceClass field at
the time of this patch raises a ValueError.
Change-Id: I2c1d4ae277ba25791c426e1c638dca1b1cb207a4
blueprint: custom-resource-classes
Use asserts with more specific assert methods, e.g.: from
assertTrue(sth == None) to assertIsNone(*),
assertTrue(isinstance(inst, type)) to assertIsInstace(inst, type) or
assertTrue(not sth) to assertFalse(sth).
This change ensures a better description will be shown on fail. This is
the recommended way to use assertions provided by testtools.
Change-Id: Ia4a34e0145176e077bbf7e9ed6f69d815c6ed840
Co-authored-by: Stephen Finucane <sfinucan@redhat.com>
Quotas are required to exist in the API database as we need to enforce
quotas across cells.
blueprint cells-quota-api-db
Change-Id: I52fd680eaa4880b06f7f8d4bd1bb74920e73195d
We will store custom resource classes in the new resource_classes table.
These custom resource classes represent non-standardized resource
classes. Followup patches add the plumbing to handle existing
standardized classes currently using the fields.ResourceClass field type
and to perform CRUD operations against a /resource-classes REST API
endpoint.
Change-Id: I60ea0dcb392c1b82fead4b859fc7ed6b32d4bda0
blueprint: custom-resource-classes
Add the following notifications when swapping volumes.
* 'instance.volume_swap.start'
* 'instance.volume_swap.end'
A subsequent patch will add 'instance.volume_swap.error'.
Change-Id: Ic4d9d25bdc611221157e4663817e918b8d667ce9
Implements: blueprint add-swap-volume-notifications
String interpolation should be delayed to be handled
by the logging code, rather than being done
at the point of the logging call.
So add the following hacking rule for it.
- [N354] String interpolation should be delayed at logging calls.
See the oslo i18n guideline.
* http://docs.openstack.org/developer/oslo.i18n/guidelines.html
Change-Id: Ief6d3ee3539c0857098fffdb7acfeec3e0fed6eb
Closes-Bug: #1596829
Move all scheduler options into their one of two groups. Many of the
options are simply renamed to remove their 'scheduler_' prefix, with
some exceptions:
* scheduler_default_filters -> enabled_filters
* scheduler_baremetal_default_filters -> baremetal_enabled_filters
* scheduler_driver_task_period -> periodic_task_interval
* scheduler_tracks_instance_changes -> track_instance_changes
Change-Id: I3f48e52815e80c99612bcd10cb53331a8c995fc3
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
Implements: blueprint centralize-config-options-ocata
