9738 Commits

Author SHA1 Message Date
Zuul
97196f849b Merge "[placement] Separate API schemas (inventory)" 2017-12-17 10:33:19 +00:00
Takashi NATSUME
571f96b046 [placement] Separate API schemas (resource_class)
In compute APIs, they have their schemas in the
independent directory (nova/api/openstack/compute/schemas).
Placement APIs should be like that as well.

This patch separates API schemas to an independent directory
(nova/api/openstack/placement/schemas)
from nova/api/openstack/placement/handlers/resource_class.py.

Subsequent patches will move schemas of other handlers.

Change-Id: I61bfc96953d1158d3204d3f578a5255c18af2d6e
2017-12-17 01:01:58 +00:00
Zuul
79b575cb96 Merge "Updated from global requirements" 2017-12-16 00:17:16 +00:00
OpenStack Proposal Bot
22d2551d48 Updated from global requirements
Change-Id: Ibc2285102f5c23e38badfcf1ed6dba073704a213
2017-12-15 21:49:49 +00:00
Zuul
2884980f07 Merge "[placement] Separate API schemas (trait)" 2017-12-15 20:17:33 +00:00
Zuul
b5b46d35a6 Merge "[placement] Separate API schemas (aggregate)" 2017-12-15 18:41:21 +00:00
Zuul
1b1bfaadad Merge "[placement] Separate API schemas (usage)" 2017-12-15 18:41:13 +00:00
Zuul
2203ffd44f Merge "Make request_spec.spec MediumText" 2017-12-15 18:41:08 +00:00
Zuul
a841b4f036 Merge "SchedulerReportClient._get_providers_in_aggregates" 2017-12-15 17:15:16 +00:00
Artom Lifshitz
6a899c5f89 Make request_spec.spec MediumText
request_spec.instance_group.members is a list of instance UUIDs. It
can get so long that it overflows its TEXT column. This patch changes
request_spec.spec to MEDIUMTEXT.

Change-Id: I6bf0fa19b72887803e77b66698587c2108c9372a
Closes-bug: 1738094
2017-12-14 21:28:18 -05:00
Zuul
4904cd915a Merge "qemu-img do not use cache=none if no O_DIRECT support" 2017-12-15 02:11:20 +00:00
Eric Fried
5dcca7f5f2 SchedulerReportClient._get_providers_in_aggregates
Introducing SchedulerReportClient._get_providers_in_aggregates, a
private method to retrieve the list of resource providers associated
with some number of aggregate UUIDs.  Returns the empty list (not None)
if no aggregate UUIDs are specified, or if there are no providers
associated with that aggregate.  Logs an error and raises on any
HTTP response other than 200.

This is in anticipation of ComputeDriver.update_provider_tree, which
will need to be supplied a ProviderTree that's fully populated with all
the providers associated with that of the compute node.  This includes
all providers in the compute RP's tree *and* those directly associated
via aggregates (but not *their* trees or aggregate associations).

Change-Id: Ia7f95a17fd00bea414459b90adc04a8bf309c396
blueprint: nested-resource-providers
2017-12-14 15:21:22 +00:00
Takashi NATSUME
b5c2099cfe [placement] Separate API schemas (inventory)
In compute APIs, they have their schemas in the
independent directory (nova/api/openstack/compute/schemas).
Placement APIs should be like that as well.

This patch separates API schemas to an independent directory
(nova/api/openstack/placement/schemas)
from nova/api/openstack/placement/handlers/inventory.py.

Subsequent patches will move schemas of other handlers.

Change-Id: Iab542cfb4d09b26f1d9a3db8a8678a8dba173eb9
2017-12-14 15:17:14 +00:00
Takashi NATSUME
337d402df2 [placement] Separate API schemas (aggregate)
In compute APIs, they have their schemas in the
independent directory (nova/api/openstack/compute/schemas).
Placement APIs should be like that as well.

This patch separates API schemas to an independent directory
(nova/api/openstack/placement/schemas)
from nova/api/openstack/placement/handlers/aggregate.py.

Subsequent patches will move schemas of other handlers.

Change-Id: I1b526b1df28c281bd456f35bc0f5c5d9c2464c7e
2017-12-14 15:16:27 +00:00
Takashi NATSUME
88f7ec3dd9 [placement] Separate API schemas (trait)
In compute APIs, they have their schemas in the
independent directory (nova/api/openstack/compute/schemas).
Placement APIs should be like that as well.

This patch separates API schemas to an independent directory
(nova/api/openstack/placement/schemas)
from nova/api/openstack/placement/handlers/trait.py.

Subsequent patches will move schemas of other handlers.

Change-Id: Ieba7ca1e58d931c7cc0b11cb9313d37c5fb7cd3a
2017-12-14 15:16:01 +00:00
Takashi NATSUME
4253e676fa [placement] Separate API schemas (usage)
In compute APIs, they have their schemas in the
independent directory (nova/api/openstack/compute/schemas).
Placement APIs should be like that as well.

This patch separates API schemas to an independent directory
(nova/api/openstack/placement/schemas)
from nova/api/openstack/placement/handlers/usage.py.

Subsequent patches will move schemas of other handlers.

Change-Id: I38fc7e848fb3f95d79482e5903679e585e3675cc
2017-12-14 15:15:33 +00:00
Zuul
9ecf7d32e0 Merge "[placement] Add 'Location' parameters in API ref" 2017-12-14 14:26:12 +00:00
Zuul
63b4c1f03e Merge "[placement] add name to resource provider create error" 2017-12-14 14:26:07 +00:00
Matt Riedemann
89eb6a9274 Pass mountpoint to volume attachment_create with connector
Similar to I11ba269c3f7a2e7707b2b7e27d26eb7a2c948a82, when
we create a volume attachment with a connector, we need to
also provide the mountpoint via the connector to Cinder,
because internally within Cinder the attachment_create code
is calling attachment_update if a connector is provided.

Change-Id: If3afe8d8bd6b8c327ccc7d1140053bccaf7e1ad7
Closes-Bug: #1737779
2017-12-13 20:15:44 -05:00
Matt Riedemann
0a00a675ea Pass mountpoint to volume attachment_update
The old volume attach flow would pass the mountpoint,
which is the BlockDeviceMapping.device_name, to the
os-attach volume action API. With the new flow, apparently
the mountpoint is expected to be passed to the update
attachment API via the connector dict, which is not obvious
and if not provided, causes Cinder to default the mountpoint
to "/dev/na" which is wrong.

We work around this in Nova by providing the mountpoint in a
copy of the connector dict and pass that to attachment_update,
and update the two places in the code that are updating an
attachment (the normal driver block device attach code and
swap volume). Long-term this should be fixed in the Cinder
attachments update API, but that requires a microversion so
we need to handle it client-side for now.

Change-Id: I11ba269c3f7a2e7707b2b7e27d26eb7a2c948a82
Partial-Bug: #1737779
2017-12-13 20:10:14 -05:00
Zuul
84ec5178aa Merge "Add instance action db and obj pagination support." 2017-12-14 00:59:01 +00:00
Zuul
72c7aca396 Merge "Update Instance action's updated_at when action event updated." 2017-12-14 00:29:48 +00:00
Matt Riedemann
22b282e95b Update nova-status and docs for nova-compute requiring placement 1.14
With change I2f367b06e683ed7c815dd9e0536a46e5f0a27e6c, nova-compute
now unconditionally requires Placement 1.14 to be available (the
client side code doesn't check to see if 1.14 is available before
trying to use it).

This change updates the nova-status check for the minimum required
version of Placement and also starts the "Queens" section of the
Placement upgrade notes docs.

Change-Id: I37415e384d375bc9b548a0223f787a9236286bb0
2017-12-13 15:18:27 -05:00
Zuul
dfc1202fcb Merge "Refactor placement version check" 2017-12-13 07:11:24 +00:00
Zuul
bccb14e1c8 Merge "[placement] Add info about last-modified to contrib docs" 2017-12-13 05:35:18 +00:00
Zuul
87ccac69fc Merge "[placement] Add cache headers to placement api requests" 2017-12-13 05:30:32 +00:00
Zuul
fcc4c8a262 Merge "SchedulerReportClient._get_providers_in_tree" 2017-12-13 03:41:18 +00:00
Chris Dent
8605ec994a [placement] Add info about last-modified to contrib docs
Add some instructions on how and when to add last-modified headers
when creating a new handler in the placement API.

Change-Id: I8f0c3006d1bb97d228f73256c58a79235cd12670
2017-12-12 15:53:26 +00:00
Chris Dent
4ee7c0e0e2 [placement] Add cache headers to placement api requests
In relevant requests to the placement API add last-modified
and cache-control headers.

According the HTTP 1.1 RFC headers last-modified headers SHOULD always
be sent and should have a tie to the real last modified time. If we do
send them, we need Cache-Control headers to prevent inadvertent caching
of resources.

This change adds a microversion 1.15 which adds the headers to GET
requests and some PUT or POST requests.

Despite what it says 'no-cache' means "check to see if the version you
have is still valid as far as the server is concerned". Since our server
doesn't currently validate conditional requests and will always return an
entity, it ends up meaning "don't cache" (which is what we want).

The main steps in the patch are:

* To both the get single entity and get collection handlers add
  response.cache_control = 'no-cache'
* For single entity add response.last_modified = obj.updated_at or
  obj.created_at
* For collections, discover the max modified time when traversing the
  list of objects to create the serialized JSON output. In most of
  those loops an optimization is done where we only check for
  last-modified information if we have a high enough microversion such
  that the information will be used. This is not done when listing
  inventories because the expectation is that no single resource
  provider will ever have a huge number of inventory records.
* Both of the prior steps are assisted by a new util method:
  pick_last_modfied.

Where a time cannot be determined the current time is used.

In typical placement framework fashion this has been done in a very
explicit way, as it makes what the handler is doing very visible, even
though it results in a bit of boilerplate.

For those requests that are created from multiple objects or by doing
calculations, such as usages and aggregate associations, the current time
is used.

The handler for PUT /traits is modified a bit more extensively than some
of the others: This is because the method can either create or validate
the existence of the trait. In the case where the trait already exists,
we need to get it from the DB to get its created_at time. We only do
this if the microversion is high enough (at least 1.15) to warrant
needing the info.

Because these changes add new headers (even though they don't do
anything) a new microversion, 1.15, is added.

Partial-Bug: #1632852
Partially-Implements: bp placement-cache-headers

Change-Id: I727d4c77aaa31f0ef31c8af22c2d46cad8ab8b8e
2017-12-12 15:51:58 +00:00
Zuul
e3b29c5358 Merge "[placement] Fix API reference for microversion 1.14" 2017-12-11 21:24:15 +00:00
Hongbin Lu
17ffe8096b placement: skip authentication on root URI
The placement API GET / is for version discovery. Skip authentication
on this URL.

The change to test_deploy is required to make it request a URL that
requires authentication. It had been using /.

Change-Id: I5ed1f7a607620c43671e90eaf8a1f6285f1d15e4
Closes-Bug: #1733630
2017-12-11 17:28:36 +00:00
Yikun Jiang
520b121d20 Add instance action db and obj pagination support.
This will be used by instance action pagination API.

Add limit/marker/filters support to get_by_instance_uuid of
InstanceActionList object, also add limit/marker/filters support to
actions_get method of db.

Part of blueprint pagination-add-changes-since-for-instance-action-list

Change-Id: Ic7dd6480a4b250ae6529d94ee0386b5e95b0ca04
2017-12-11 11:01:42 -05:00
Yikun Jiang
612db97f4b Update Instance action's updated_at when action event updated.
When we do some operation on instances, will record some
instance action(such as 'create') in 'instance_actions' table,
and some sub-event will record(such as
compute__do_build_and_run_instance) in 'instance_actions_events'
table.

we need update the instance action's updated_at when instance
action events are created and instance action created or finished.

Change-Id: I75a827b759b59773c08ffc6b1e3e54d6189b5853
Closes-Bug: #1719561
2017-12-11 11:01:23 -05:00
Takashi NATSUME
ce525a805f [placement] Fix API reference for microversion 1.14
It is a follow-up for I4db74e4dc682bc03df6ec94cd1c3a5f5dc927a7b.
Fix description of placement microversion 1.14.

Change-Id: I7a7ffc395d444fe7cf0434ea6745dde0dae11ad5
blueprint nested-resource-providers
2017-12-11 11:37:31 +09:00
Zuul
f88d40be10 Merge "Follow up on removing old-style quotas code" 2017-12-09 07:24:15 +00:00
Zuul
892d5d06c0 Merge "[placement] Fix an error message in API validation" 2017-12-09 07:24:02 +00:00
Zuul
977e20e5a9 Merge "Add API and nova-manage tests that use the NoopQuotaDriver" 2017-12-09 07:23:55 +00:00
Zuul
96998afc97 Merge "Implement new attach Cinder flow" 2017-12-09 03:06:14 +00:00
melanie witt
22cdd1c753 Follow up on removing old-style quotas code
This is a follow up to another patch [1] that removed the old-style
quotas code which is no longer in use. Here, we remove the 'reserved'
key from quotas internally and set it in the APIs where it's expected.
We also remove code that accesses the quota_usages and reservations
tables as they're no longer used.

[1] https://review.openstack.org/#/c/511689

Change-Id: I75291571468ddb79b7561810de0953bb462548e3
2017-12-08 22:11:24 +00:00
melanie witt
9a89362364 Add API and nova-manage tests that use the NoopQuotaDriver
These are written in preparation for follow up old-style quotas code
removal that moves the logic for the 'reserved' field from the quota
internals (where it's no longer used) to the API (where it's expected
to be provided until we have a new microversion). These tests will
help catch any unintended change in behavior.

Change-Id: I1f743a8db7d032e09372587622ab64250a05e0f0
2017-12-08 22:11:24 +00:00
Zuul
51917f46ac Merge "Updated from global requirements" 2017-12-08 19:16:33 +00:00
Chris Dent
a3e310adf6 [placement] add name to resource provider create error
When a resource provider is created an ObjectActionError can
happen. The API handler for POST /resource_provider catches
this and sends out an error response. Prior to this change the
response only identified the resource provider by uuid. Since
it is possible to create a resource provider by only providing
a name, this error response may not be super useful.

This change adds the name of the proposed resource provider to
the message, to make sure things are clear.

However, it's worth noting that at the current time most
situations which would cause an ObjectActionError are not
possible from the API code path: they are caught by the
JSONSchema validation. The one clear path that is not caught,
and thus is the place where a validation of the response has
been done in a gabbit, is when a resource provider tries to
set its parent uuid to its own uuid. In order for the caller
to do this, the caller has to know its own uuid, so the
ambiguity problem is moot.

As we might make additional changes in the future, adding
the name seems the most flexible thing to do, so here it
is.

Change-Id: I7fee5344fc7cb2e4ba3f46e192ba0122edeb0605
Closes-Bug: #1735405
2017-12-08 15:28:15 +00:00
Takashi NATSUME
3d194baca3 [placement] Add 'Location' parameters in API ref
The 'Location' parameters are missing in the follwoing APIs of
Placement API reference. So add them.

* POST /resource_providers
* POST /resource_classes
* PUT /resource_classes/{name} (microversion 1.7-)
* PUT /traits/{name}

Change-Id: Ieed5cb7d4697472ab46b2e80d6d2df68098c5631
Closes-Bug: #1733329
2017-12-07 21:00:43 +00:00
Zuul
60a3ff4bf0 Merge "placement: adds REST API for nested providers" 2017-12-07 16:38:40 +00:00
John Griffith
d4ac674750 Implement new attach Cinder flow
This change integrates support of the Cinder 3.44
volume attachment API.

The patch bumps the compute service version to
check whether all the compute nodes are upgraded to the
version that can handle attach and detach with the new
flow.

To enable the new flow we also need the 3.44 or higher
microversion from Cinder. We check that in the API and
if it's not available we fall back to the old
attach/detach flow.

Co-Authored-By: Ildiko Vancsa <ildiko.vancsa@gmail.com>

Partially Implements: blueprint cinder-new-attach-apis
Change-Id: Ifc01dbf98545104c998ab96f65ff8623a6db0f28
2017-12-07 10:29:15 -05:00
Eric Fried
113c4a4d04 SchedulerReportClient._get_providers_in_tree
Introducing SchedulerReportClient._get_providers_in_tree, a private
method to retrieve the list of resource providers associated with one
provider UUID.  Returns the empty list (not None) if there's no provider
with the specified UUID.  Logs an error and raises an exception on any
HTTP response other than 200.

Change-Id: Ifcaa1c9c0432af17ccc91a8bf0a85e37c9b00de5
blueprint: nested-resource-providers
2017-12-07 09:12:39 -06:00
OpenStack Proposal Bot
7deb9902c1 Updated from global requirements
Change-Id: I05c26b75c2e4fe37dd6832b695851befa2f48e4b
2017-12-07 13:37:18 +00:00
Zuul
ee4ba084a6 Merge "[placement] re-use existing conf with auth token middleware" 2017-12-07 08:57:06 +00:00
ghanshyam
fd0c8fd17d Deprecate configurable Hide Server Address Feature
This commit deprecates the config option and policy
for hide server address. They are marked for removal.

Implement blueprint remove-configurable-hide-server-address-feature

Depends-On: I6aed4909b0e7efe9c95d1f7398db613eca05e5ce
Change-Id: I6040e8c2b3e132b0dfd09f82ae041b4786a63483
2017-12-07 08:45:18 +00:00
Stephen Finucane
5734d078fc placement: adds REST API for nested providers
Adds a new microversion (1.14) to the placement REST API for supporting
nested resource providers.

For POST /resource_providers and PUT /resource_providers/{uuid}, a new
optional 'parent_provider_uuid' field is added to the request payload.

For GET /resource_providers/{uuid} responses, the
'parent_provider_uuid' field and a convenience field called
'root_provider_uuid' are provided.

For GET /resource_providers, a new '?in_tree=<rp_uuid>' parameter is
supported. This parameter accepts a UUID of a resource provider. This
will cause the resulting list of resource providers to be only the
providers within the same "provider tree" as the provider identified by
<rp_uuid>

Clients for the placement REST API can specify either
'OpenStack-API-Version: placement 1.14' or 'placement latest' to handle
the new 'parent_provider_uuid' attribute and to query for resource
providers in a provider tree.

Change-Id: I4db74e4dc682bc03df6ec94cd1c3a5f5dc927a7b
blueprint: nested-resource-providers
APIImpact
2017-12-06 10:48:09 -06:00