80 Commits

Author SHA1 Message Date
Chris Dent
4e07d81260 [placement] Support forbidden traits in API
In a new microversion (1.22) expose support for processing
forbidden traits in GET /resource_providers and GET
/allocation_candidates. A forbidden trait is expressed as
part of the required parameter with a "!" prefix:

    required=CUSTOM_FAST,!CUSTOM_SLOW

This change uses db and query processing code adjustments
already present in the code but guarded by a flag. If the
currently requested microversion matches 1.22 or beyond
that flag is True, otherwise False.

Reno, api-ref update and api history update are included.
Because this microversion changes the value of an existing
parameter it was unclear how to best express that in the
api-ref. In this case existing parameter references were
annotated.

Partially implements blueprint placement-forbidden-traits

Change-Id: I43e92bc5f97db7a2b09e64c6cb953c07d0561e63
2018-04-13 19:24:08 +01:00
Zuul
f577ecbdbe Merge "[placement] api-ref: Fix parameters" 2018-04-04 19:12:03 +00:00
Takashi NATSUME
9dd1923672 [placement] api-ref: Fix parameters
This patch fixes the following items.

* GET /resource_providers

  Wrong order of query parameters.
  They should be sorted by microversion order.

* POST /resource_providers

  A required parameter is defined as optional.
  Set it as required.

Change-Id: I66c9608cd6b4a938ca7f8450d4130e942360a9f0
Closes-Bug: #1760520
2018-04-03 17:05:42 +09:00
Takashi NATSUME
9fccb60be3 Remove unnecessary code encoding specification
TrivialFix
Change-Id: Ifce837e794fb01aca2e09b3f7872757939a8b09a
2018-03-30 19:26:56 +00:00
Ed Leafe
a69e05d29a Add 'member_of' param to GET /allocation_candidates
The call to GET /allocation_candidates now accepts a 'member_of'
parameter, representing one or more aggregate UUIDs. If this parameter
is supplied, the allocation_candidates returned will be limited to those
with resource_providers that belong to at least one of the supplied
aggregates.

Blueprint: alloc-candidates-member-of

Change-Id: I5857e927a830914c96e040936804e322baccc24c
2018-03-16 16:32:02 +00:00
Eric Fried
388db7e6e2 placement: Return new provider from POST /rps
To facilitate opaqueness of resource provider generation internals, we
need to return the (initial) generation when a provider is created. For
consistency with other APIs, we will do this by returning the entire
resource provider record (which includes the generation) from POST
/resource_providers.

Change-Id: I8624e194fe0173531c5aa2119c903e3c68b8c6cd
blueprint: generation-from-create-provider
2018-03-14 17:08:55 -05:00
Eric Fried
3216f078d4 placement: generation in provider aggregate APIs
Placement API microversion 1.19 enhances the payloads for the `GET
/resource_providers/{uuid}/aggregates` response and the `PUT
/resource_providers/{uuid}/aggregates` request and response to be
identical, and to include the ``resource_provider_generation``. As with
other generation-aware APIs, if the ``resource_provider_generation``
specified in the `PUT` request does not match the generation known by
the server, a 409 Conflict error is returned.

Change-Id: I86416e35da1798cdf039b42c9ed7629f0f9c75fc
blueprint: placement-aggregate-generation
2018-03-14 17:08:52 -05:00
Zuul
04eb8f2aea Merge "[placement] api-ref: Fix a missing response code" 2018-03-01 12:56:21 +00:00
Zuul
e3d5a083e3 Merge "rp: GET /resource_providers?required=<traits>" 2018-02-26 13:24:13 +00:00
Eric Fried
558540a27c rp: GET /resource_providers?required=<traits>
Introduce placement microversion 1.18 with a new ?required=<trait list>
query parameter accepted on the GET /resource_providers API.  Results
are filtered by providers possessing *all* of the specified traits.
Empty/invalid traits result in 400 errors.

Change-Id: I8191c9a390cb02b2a38a3f1c6e29457435994981
blueprint: traits-on-list-resource-providers
2018-02-23 12:08:32 -06:00
Eric Fried
9abccf8f77 Clarify resources query param for /r_p and /a_c
It was pointed out [1] that the semantic difference between how the
`resources` query parameter is handled by GET /resource_providers versus
GET /allocation_candidates is not clear based on the API reference
descriptions, which are identical.  This change set clarifies the
wording for both, making it clear that a) allocations are taking into
account; and b) GET /allocation_candidates is working with groups of
providers that must *collectively* satisfy the resource specification.

[1] https://review.openstack.org/#/c/547056/2/specs/rocky/approved/traits-on-list-resource-providers.rst@33
Change-Id: Ia2a4f8f80ec68dd9c95e084c292a2fe2d6ec1b2d
2018-02-22 16:47:06 +00:00
Takashi NATSUME
b2cf359491 [placement] api-ref: Fix a missing response code
In "List resource providers" API,
a 400 badRequest is missing in API reference.
So add it.

Change-Id: I92fcbd6bc1477dd0fe6554c0ba631b9cb7d6fa53
Closes-Bug: #1750728
2018-02-21 14:47:59 +09:00
Eric Fried
e8406e6437 api-ref: Further clarify placement aggregates
Following up on the comments at [1], provide further disambiguation and
clarity on the difference between placement aggregates and nova host
aggregates, with some illustrative examples that include where the
former may be used to represent the latter (as is being proposed via
[2]).

[1] https://review.openstack.org/#/c/539033/1/placement-api-ref/source/aggregates.inc@14
[2] https://review.openstack.org/#/q/bp/placement-mirror-host-aggregates

Change-Id: I53588fca311992f885c4157591232a864b6a86b1
2018-02-16 08:52:30 -06:00
Zuul
0cf7a8a648 Merge "api-ref: provide more detail on what a provider aggregate is" 2018-02-16 11:15:12 +00:00
Takashi NATSUME
de6e2b9948 [placement] Move body examples to an isolated directory
JSON body examples of Placement API reference
are in the same directroy where *.inc files exist.

This patch moves JSON body examples to an isolated directory
in order to make it easier to maintain the API reference.

Change-Id: I8fd587100d22bd10dd4b2b2f1b25c5ecc7fc8537
2018-02-08 21:59:19 +00:00
Zuul
3bb7c8dd61 Merge "placement doc: Conflict caveat for DELETE APIs" 2018-02-07 13:33:29 +00:00
Eric Fried
f3b0cf3975 placement doc: Conflict caveat for DELETE APIs
Since the DELETE /resource_providers/{u}/inventories and .../traits APIs
don't have a way to accept generation, they're not "threadsafe" in the
sense of multiple client threads managing traits/inventories for the
same provider.  This change adds a note to the documentation for these
APIs to this effect, suggesting the use of PUT with empty
traits/inventories instead.

Change-Id: Icfd79cc1f5a912131845a22b4fe900147b19f934
Closes-Bug: #1746373
2018-02-05 17:34:35 +00:00
Eric Fried
5d0b00a0ef Doc: Nix os-traits link from POST resource_classes
The placement API reference documentation for POST /resource_classes [1]
said:

The new class must be a custom resource class, prefixed with CUSTOM_ and
distinct from the _standard_ resource classes.

...where _standard_ was a link to os-traits documentation [2]. Since we
don't have os-resource-classes (yet), this change set removes that
linkitude.

[1] https://developer.openstack.org/api-ref/placement/#create-resource-class
[2] https://docs.openstack.org/os-traits/latest/

Change-Id: I15fbed460598b7d603eca48cbaecf1036906f6c7
Closes-Bug: #1747095
2018-02-02 18:00:21 -06:00
He Jie Xu
284ba35c33 Fix nits in support traits changes
Addresses the comments from earlier patches:

https://review.openstack.org/535642

https://review.openstack.org/536085

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Change-Id: I366b97ef3c141834f48949700edb968a7c7c4167
2018-01-31 11:07:07 -05:00
Matt Riedemann
03be4beb4c api-ref: provide more detail on what a provider aggregate is
The description of placement resource provider aggregates is mostly
discussing how they aren't nova's concept of host aggregates, but
doesn't give a very simple description of what they are.

This provides a simple description and the canonical example of a
shared storage pool.

Change-Id: I3b7c6404d364c045f19390e86fd1b747baf11f8c
2018-01-29 17:39:49 -05:00
He Jie Xu
8a307bbdf4 placement: support traits in allocation candidates API
This patch add new query parameter `required` to the
`GET /allocation_candidates` API, which is used to filter candidates
with required traits.  The candidate attached traits return in the
provider summary also. Those API changes are added by new microversion.

Also using specific exception TraitNotFound instead of the generic
exception ValueError when invalid traits in the request.

Change-Id: Id821b5b2768dcc698695ba6570c6201e1e9a8233
Implement blueprint add-trait-support-in-allocation-candidates
2018-01-22 22:10:10 +08:00
Stephen Finucane
bcfd1eebe8 trivial: Remove crud from 'conf.py'
No need to have any of this here.

Change-Id: I87c291126bdb9bd3c45232e1041fd9355e953955
2018-01-18 10:39:24 +00:00
Stephen Finucane
62ea2ff04d Fix openstackdocstheme options for api-ref
Initialize the parameter for current openstackdocstheme so that report a
bug feature works and displays all parameter including git URL and SHA.

We need to add openstackdocstheme as extension so that the parameter are
properly initialized. html_context is not needed anymore. The display of
time of last commit is done by openstackdocstheme.

Change-Id: Ic46f5ff6bc42b48ce9de5f5bf3a2193ed75fb063
Closes-Bug: #1743728
2018-01-18 10:38:51 +00:00
Andreas Jaeger
33a96b2282 Fix OpenStack capitalization
Reviewing the Placement API, I noticed wrong OpenStack capitalization.
Fix the docs and some strings as well.

Change-Id: I14a2443687a0d517ece80e794e7ef0d4e165af6f
2018-01-07 12:38:39 +01:00
Chris Dent
4a97bbd824 [placement] Enable limiting GET /allocation_candidates
This adds a limit query parameter to GET
/allocation_candidates?limit=5&resource=VCPU:1

A 'limit' filter is added to the AllocationCandidates. If set, after
the database query has been run to create the allocation requests and
provider summaries, a slice or sample of the allocation requests is
taken to limit the results. The summaries are then filtered to only
include those in the allocation requests.

This method avoids needing to make changes to the generated SQL, the
creation of which is fairly complex, or the database tables. The amount
of data queried is still high in the extreme case, but the amount of
data sent over the wire (as JSON) is shrunk. This is a trade-off that
was discussed in the spec and the discussion surrounding its review.
If it turns out that memory use server-side is an issue we can
investigate changing the SQL.

A configuration setting, [placement]/randomize_allocation_candidates,
is added to allow deployers to declare whether they want the results
to be returned in whatever order the database chooses or a random
order. The default is "False" which is expected to preserve existing
behavior and impose a packing placement strategy.

When the config setting is combined with the limit parameter, if
"True" the limited results are a random sampling from the full
results. If "False", it is a slice from the front.

This is done as a new microversion, 1.16, with updates to docs, a reno
and adjustments to the api history doc.

Change-Id: I5f3d4f49c34fd3cd6b9d2e12b3c3c4cdcb409bec
Implements: bp allocation-candidates-limit
2017-12-20 20:08:39 +00:00
Takashi NATSUME
1190c34183 [placement] Add x-openstack-request-id in API ref
Add the description about 'x-openstack-request-id'
in the request and the response headers.

Change-Id: I6ffdfbacb81660b89d7bf8ba83dbab1aa25a80bd
Closes-Bug: #1733496
2017-12-18 06:02:44 +00:00
Zuul
b11c9a7e89 Merge "[placement] Add 'Location' parameters in API ref" 2017-12-14 14:26:12 +00:00
Takashi NATSUME
0dee4cd6c4 [placement] Fix API reference for microversion 1.14
It is a follow-up for I4db74e4dc682bc03df6ec94cd1c3a5f5dc927a7b.
Fix description of placement microversion 1.14.

Change-Id: I7a7ffc395d444fe7cf0434ea6745dde0dae11ad5
blueprint nested-resource-providers
2017-12-11 11:37:31 +09:00
Takashi NATSUME
0027ee4018 [placement] Add 'Location' parameters in API ref
The 'Location' parameters are missing in the follwoing APIs of
Placement API reference. So add them.

* POST /resource_providers
* POST /resource_classes
* PUT /resource_classes/{name} (microversion 1.7-)
* PUT /traits/{name}

Change-Id: Ieed5cb7d4697472ab46b2e80d6d2df68098c5631
Closes-Bug: #1733329
2017-12-07 21:00:43 +00:00
Stephen Finucane
109f21f3c8 placement: adds REST API for nested providers
Adds a new microversion (1.14) to the placement REST API for supporting
nested resource providers.

For POST /resource_providers and PUT /resource_providers/{uuid}, a new
optional 'parent_provider_uuid' field is added to the request payload.

For GET /resource_providers/{uuid} responses, the
'parent_provider_uuid' field and a convenience field called
'root_provider_uuid' are provided.

For GET /resource_providers, a new '?in_tree=<rp_uuid>' parameter is
supported. This parameter accepts a UUID of a resource provider. This
will cause the resulting list of resource providers to be only the
providers within the same "provider tree" as the provider identified by
<rp_uuid>

Clients for the placement REST API can specify either
'OpenStack-API-Version: placement 1.14' or 'placement latest' to handle
the new 'parent_provider_uuid' attribute and to query for resource
providers in a provider tree.

Change-Id: I4db74e4dc682bc03df6ec94cd1c3a5f5dc927a7b
blueprint: nested-resource-providers
APIImpact
2017-12-06 10:48:09 -06:00
Zuul
785dd41fb1 Merge "[placement]Enhance doc for placement allocation list" 2017-12-05 13:32:38 +00:00
Zuul
df569ffa9a Merge "Add description for resource class creation" 2017-12-05 13:16:59 +00:00
Takashi NATSUME
adac748c9e [placement] Add aggregate link note in API ref
The aggregate link has been added in resource provider APIs
since microversion 1.1.
But the note is missing in Placement API reference.
Add the note.

The allocations link is missing in the response example
of "Update resource provider" API.
Add it as well.

Change-Id: I325ff34c8b436429c2a2623cf1fb16b368807d29
Closes-Bug: #1733317
2017-12-04 22:07:33 +00:00
jichenjc
5f5c94d38d Add description for resource class creation
POST /placement/resource_classes with incorrect format body
of the resource class name lead to something like:

JSON does not validate: u'_FPGA' does not match '^CUSTOM\\_[A-Z0-9_]+$'
with return code 400

we should add this into doc

Change-Id: Iab264f6d731a5f757a992d9bd148e7eb30922b9d
2017-12-02 01:24:37 +00:00
Zuul
b1eca3d4df Merge "[placement] Add 'CUSTOM_' prefix description in API ref" 2017-12-01 01:44:01 +00:00
Zuul
e1499c6ecc Merge "[placement] Fix parameter order in placement API ref" 2017-12-01 01:42:27 +00:00
jichenjc
1753195c75 [placement]Enhance doc for placement allocation list
add a note about when the non-existing consumer id input result

Change-Id: I2a19d8fe8dda5754112361be5ceac8a0dc747ae9
2017-11-30 20:41:08 +00:00
Takashi NATSUME
49a8495e12 [placement] Fix format in placement API ref
In "Delete traits" API of Placement API reference,
the format of response codes description is wrong.
"409 Conflict if the name to delete has associations with any"
is bold, but "ResourceProvider." is not.
So fix it.

TrivialFix
Change-Id: I963c350bfc339d4db769b0bfd6bba1087fa065e0
2017-11-30 08:36:40 +00:00
Chris Dent
453fd67da1 [placement] Fix GET PUT /allocations nits
In the review of I49f5680c15413bce27f2abba68b699f3ea95dcdc, a few
non-blocking nits were identified. This change addresses some of
those nits, fixing some typos, clarifying method names and what
microversion is in use at particular times.

Change-Id: Iff15340502ce43eba3b98db26aa0652b1da24504
2017-11-28 12:25:13 +00:00
Chris Dent
8caf4f5148 [placement] POST /allocations to set allocations for >1 consumers
This provides microversion 1.13 of the placement API, giving the
ability to POST to /allocations to set (or clear) allocations for
more than one consumer uuid.

It builds on the recent work to support a dict-based JSON format
when doing a PUT to /allocations/{consumer_uuid}.

Being able to set allocations for multiple consumers in one request
helps to address race conditions when cleaning up allocations during
move operations in nova.

Clearing allocations is done by setting the 'allocations' key for a
specific consumer to an empty dict.

Updates to placement-api-ref, rest version history and a reno are
included.

Change-Id: I239f33841bb9fcd92b406f979674ae8c5f8d57e3
Implements: bp post-allocations
2017-11-28 12:15:53 +00:00
Zuul
b60a599b5f Merge "[placement] Symmetric GET and PUT /allocations/{consumer_uuid}" 2017-11-28 11:09:03 +00:00
Takashi NATSUME
20b5b00722 [placement] Add 'CUSTOM_' prefix description in API ref
In the following resource class APIs,
the name of a resource class must start with 'CUSTOM_'.
If not, the request returns a 'Bad Request (400)' response code.
It should be described in the API reference. So add it.

* POST /resource_classes
* PUT /resource_classes/{name}

Change-Id: I132c532678bb74a460515067187fbf1e30885335
Closes-Bug: #1733308
2017-11-23 21:01:10 +00:00
Takashi NATSUME
0965e7ae6e [placement] Fix parameter order in placement API ref
Optional parameters should be after required paramenters.
Parameters added in microversions should be after
original parameters.
So fix them.

Change-Id: I5e3f5df5601d3747debed273e09a8b9b18046a96
Closes-Bug: #1728557
2017-11-23 21:01:07 +00:00
Chris Dent
808323e0c5 [placement] Symmetric GET and PUT /allocations/{consumer_uuid}
In a new microversion, 1.12, include project_id and user_id in the
output of GET /allocations/{consumer_uuid} and add JSON schema
to enable PUT to /allocations/{consumer_uuid} using the same dict-based
format for request body that is used in the GET response. In later
commits a similar format will be used in POST /allocations. This
symmetry is general good form and also will make client code a little
easier.

Since GET /allocation_candiates includes objects which are capable
of being PUT to /allocations/{consumer_uuid}, its response body has
been updated as well, to change the 'allocation_requests' object
to use the dict-based format.

Internally to handlers/allocation.py the same method (_set_allocations)
is used for every microversion. Any previous data structure is
transformed into the dict-ish form. This means that pre-existing tests
(like allocation-bad-class.yaml) continue to exercise the problems it
was made for, but needs to be pinned to an older microversion, rather than
being latest.

Info about these changes is added to placement-api-ref,
rest_api_version_history and a reno.

Change-Id: I49f5680c15413bce27f2abba68b699f3ea95dcdc
Implements: bp symmetric-allocations
Closes-Bug: #1708204
2017-11-21 19:39:59 +00:00
Zuul
bb4faf40df Merge "placement: Document in: prefix for ?member_of=" 2017-11-21 15:25:58 +00:00
Eric Fried
44b77d887a placement: Document request headers in api-ref
Briefly document in the Placement API reference:

a) the use of Content-Type: application/json
b) how to use microversion headers, including a link to the main
Microversions reference.

Change-Id: I60d7b9dfe4f1c50fc50eaf07a04848c220ffcf62
2017-11-20 10:47:12 -06:00
Eric Fried
ed51eee6c3 placement: Document in: prefix for ?member_of=
The documentation for the member_of query parameter for GET
/resource_providers [1] claims that it accepts a "comma-separated list
of strings representing aggregate uuids".  But it turns out the code [2]
is expecting a single UUID unless the prefix 'in:' is specified.

This change set updates the documentation accordingly.

[1] https://developer.openstack.org/api-ref/placement/#resource-providers
[2] 57728836f2/nova/api/openstack/placement/handlers/resource_provider.py (L229-L233)

Change-Id: I2ec46d767ded5be02cfc73136f2d217f1347662a
Closes-Bug: #1733030
2017-11-17 20:02:40 -06:00
Eric Fried
ad04ef197b Update placement api-ref: allocations link in 1.11
Following on [1], this change updates the placement api-ref to include
the allocations link in the sample output for GET
/resource_providers[/uuid].

[1] https://review.openstack.org/499826

Depends-On: I6a1d320ce914926791d5f45e89bf4c601a6b10a0

Change-Id: I161da67258afaccd90df12364153f068ef55a3b1
2017-10-27 15:47:16 +00:00
Jenkins
0e4088e3c3 Merge "api-ref: note the microversions for GET /resource_providers query params" 2017-10-11 11:19:03 +00:00
jichenjc
9f2ddaba30 Add 400 as error code for resource class delete
DELETE /placement/resource_classes/VCPU
leads to something like:
Cannot delete standard resource class VCPU:

with return code 400, we should add this into doc

actually it returns 2 lines of
Cannot delete standard resource class VCPU:
now, another patch will fix it.

Partial implement blueprint placement-doc-enhancement-queens

Change-Id: I957325bc421033743472cdd4de17d260da942174
2017-09-25 17:54:16 +08:00