diff --git a/doc/source/specs/index.rst b/doc/source/specs/index.rst index a5a6376b6..42fb64900 100644 --- a/doc/source/specs/index.rst +++ b/doc/source/specs/index.rst @@ -67,7 +67,26 @@ In Progress train/approved/* + +Xena +---- + +Implemented +~~~~~~~~~~~ + + +In Progress +~~~~~~~~~~~ + +.. toctree:: + :maxdepth: 1 + :glob: + + xena/approved/* + + .. toctree:: :hidden: template.rst + diff --git a/doc/source/specs/xena/approved/allow-provider-re-parenting.rst b/doc/source/specs/xena/approved/allow-provider-re-parenting.rst new file mode 100644 index 000000000..3da9a215a --- /dev/null +++ b/doc/source/specs/xena/approved/allow-provider-re-parenting.rst @@ -0,0 +1,242 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +======================================== +Allow provider re-parenting in placement +======================================== + +https://storyboard.openstack.org/#!/story/2008764 + +This spec proposes to allow re-parenting and un-parenting (or orphaning) RPs +via ``PUT /resource_providers/{uuid}`` API in Placement. + +Problem description +=================== + +Today placement API only allows change the parent of an RP from None to a valid +RP UUID. However there are use case when moving an RP between parents make +sense. + +Use Cases +--------- + +* An existing PGPU RP needs to be moved under the NUMA RP when NUMA is modeled. + +* We have a `neutron bug`_ that introduced an unwanted change causing that + SRIOV PF RPs was created under the root RP instead of under the neutron agent + RP. We can fix the broken logic in neutron but we cannot fix the already + wrongly parented RP in the DB via the placement API. + +.. _`neutron bug`: https://bugs.launchpad.net/neutron/+bug/1921150 + +Proposed change +=============== + +Re-parenting is rejected today and the code has the following `comment`_ : + + TODO(jaypipes): For now, "re-parenting" and "un-parenting" are + not possible. If the provider already had a parent, we don't + allow changing that parent due to various issues, including: + + * if the new parent is a descendant of this resource provider, we + introduce the possibility of a loop in the graph, which would + be very bad + * potentially orphaning heretofore-descendants + + So, for now, let's just prevent re-parenting... + +.. _`comment`: https://github.com/openstack/placement/blob/6f00ba5f685183539d0ebf62a4741f2f6930e051/placement/objects/resource_provider.py#L777 + + +The first reason is moot as the loop check is already needed and implemented +for the case when the parent is updated from None to an RP. + +The second reason does not make sense to me. By moving an RP under another RP +all the descendants should be moved as well. Similarly how the None -> UUID +case works today. So I don't see how can we orphan any RP by re-parenting. + +I see the following possible cases of move: + +* RP moved upwards, downwards, side-wards in the same RP tree +* RP moved to a different tree +* RP moved to top level, becoming a new root RP + +From placement perspective every case results in one or more valid RP trees. + +Based on the data model if there was allocations against the moved RP those +allocations will still refer to the RP after the move. This means that if a +consumer has allocation against a single RP tree before the move might have +allocation against multiple trees after the RP move. Such consumer is already +supported today. + +An RP move might invalidate the original intention of the consumer. If the +consumer used an allocation candidate query to select and allocate resources +then by such query the consumer defined a set of rules (e.g. in_tree, +same_subtree) the allocation needs to fulfill. The rules might not be valid +after an RP is moved. However placement never promised to keep such invariant +as that would require the storage of the rules and correlating allocation +candidate queries and allocations. Moreover such issue can already +be created with the POST /reshape API as well. Therefore keeping any such +invariant is the responsibility of the client. So I propose to start supporting +all form of RP re-parenting in a new placement API microversion. + +Alternatives +------------ +See the API alternatives below. + +Data model impact +----------------- + +None + +REST API impact +--------------- + +In a new microversion allow changing the parent_uuid of a resource provider to +None or to any valid RP uuid that does not cause a loop in any of the trees via +the ``PUT /resource_providers/{uuid}`` API. + +Protecting against unwanted changes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As noted above re-parenting can significantly change the RP model in the +Placement database. So such action needs to be done carefully. While the +Placement API is already admin only by default, the request is raised on the +Xena PTG for extra safety measures against unintentional parent changes. +During the spec discussion every the reviewer expressed the view that such +safety measure is not really needed. So this spec only propose to use the new +microversion and extensive documentation to signal the new behavior. + +Still there is the list of alternatives discussed during the review: + +* `Do nothing`: While it is considered not safe enough during the PTG, during + the spec review we ended up choosing this as the main solution. +* `A new query parameter`: A new query parameter is proposed for the + ``PUT /resource_providers/{uuid}`` API called ``allow_reparenting`` the + default value of the query parameter is ``False`` and the re-parenting cases + defined in this spec is only accepted by Placement if the request contains + the new query parameter with the ``True``. It is considered hacky to add a + query parameter for a PUT request. +* `A new field in the request body`: This new field would have the same meaning + as the proposed query parameter but it would be put into the request body. It + is considered non-RESTful as such field is not persisted or returned as the + result of the PUT request as it does not belong to the representation of the + ResourceProvider entity the PUT request updates. +* `A new Header`: Instead of a new query paramtere use a new HTTP header + ``x-openstack-placement-allow-provider-reparenting:True``. As the name shows + this needs a lot more context encoded in it to be specific for the API it + modifies while the query parameter already totally API specific. +* `Use a PATCH request for updating the parent`: While this would make the + parent change more explicit it would also cause great confusion for the + client for multiple reasons: + + 1) Other fields of the same resource provider entity can be updated via the + PUT request, but not the ``parent_uuid`` field. + 2) Changing the ``parent_uuid`` field from None to a valid RP uuid is + supported by the PUT request but to change it from one RP uuid to another + would require a totally different ``PATCH`` request. +* `Use a sub resource`: Signal the explicit re-parenting either in a form of + ``PUT /resource-providers/{uuid}/force`` or + ``PUT /resource-providers/{uuid}/parent_uuid/{parent}``. While the second + option seems to be acceptable to multiple reviewers, I think it will be + confusing similarly to ``PATCH``. It would create another way to update a + field of an entity while other fields still updated directly on the parent + resource. + + +Security impact +--------------- + +None + +Notifications impact +-------------------- + +N/A + +Other end user impact +--------------------- + +None + +Performance Impact +------------------ + +The loop detection and the possible update of all the RPs in the changed +subtree with a new ``root_provider_id`` needs extra processing. However the +re-parenting operation is considered very infrequent. So the overall Placement +performance is not affected. + +Other deployer impact +--------------------- + +None + +Developer impact +---------------- + +None + +Upgrade impact +-------------- + +None + +Implementation +============== + +Assignee(s) +----------- + + +Primary assignee: + balazs-gibizer + +Feature Liaison +--------------- + +Feature liaison: + None + +Work Items +---------- + +* Add a new microversion to the Placement API. Implement an extended loop + detection and update ``root_provider_id`` of the subtree if needed. +* Mark the new microversion osc-placement as supported. + +Dependencies +============ + +None + +Testing +======= + +* Unit testing +* Gabbit API testing + +Documentation Impact +==================== + +* API doc needs to be updated. Warn the user that this is a potentially + dangerous operation. + +References +========== + +None + +History +======= + +.. list-table:: Revisions + :header-rows: 1 + + * - Release Name + - Description + * - Xena + - Introduced diff --git a/doc/source/specs/xena/approved/support-consumer-types.rst b/doc/source/specs/xena/approved/support-consumer-types.rst new file mode 100644 index 000000000..8e0412d24 --- /dev/null +++ b/doc/source/specs/xena/approved/support-consumer-types.rst @@ -0,0 +1,358 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +====================== +Support Consumer Types +====================== + +https://storyboard.openstack.org/#!/story/2005473 + +This spec aims at providing support for services to model ``consumer types`` +in placement. While placement defines a consumer to be an entity consuming +resources from a provider it does not provide a way to identify similar +"types" of consumers and henceforth allow services to group/query them based +on their types. This spec proposes to associate each consumer to a particular +type defined by the service owning the consumer. + +Problem description +=================== + +In today's placement world each allocation posted by a service is against a +provider for a consumer (ex: for an instance or a migration). However a +service may want to distinguish amongst the allocations made against its +various types of consumers (ex: nova may want to fetch allocations against +instances alone). This is currently not possible in placement and hence the +goal is to make placement aware of "types of consumers" for the services. + +Use Cases +--------- + +* Nova using placement as its `quota calculation system`_: Currently this + approach uses the nova_api database to calculate the quota on the "number of + instances". In order for nova to be able to use placement to count the number + of "instance-consumers", there needs to be a way by which we can + differentiate "instance-consumers" from "migration-consumers". + +* Ironic wanting to differentiate between "standalone-consumer" versus + "nova-consumer". + +Note that it is not within the scope of placement to model the coordination of +the consumer type collisions that may arise between multiple services during +their definition. Placement will also not be able to identify or verify correct +consumer types (eg, INTANCE versus INSTANCE) from the external service's +perspective. + +Proposed change +=============== + +In order to model consumer types in placement, we will add a new +``consumer_types`` table to the placement database which will have two columns: + +#. an ``id`` which will be of type integer. +#. a ``name`` which will be of type varchar (maximum of 255 characters) and + this will have a unique constraint on it. The pattern restrictions for the + name will be similar to placement traits and resource class names, i.e + restricted to only ``^[A-Z0-9_]+$`` with length restrictions being {1, 255}. + +A sample look of such a table would be: + ++--------+----------+ +| id | name | ++========+==========+ +| 1 | INSTANCE | ++--------+----------+ +| 2 | MIGRATION| ++--------+----------+ + +A new column called ``consumer_type_id`` would be added to the ``consumers`` +table to map the consumer to its type. + +The ``POST /allocations`` and ``PUT /allocations/{consumer_uuid}`` REST API's +will gain a new (required) key called ``consumer_type`` which is of type string +in their request body's through which the caller can specify what type of +consumer it is creating or updating the allocations for. If the specified +``consumer_type`` key is not present in the ``consumer_types`` table, a new +entry will be created. Also note that once a consumer type is created, it +lives on forever. If this becomes a problem in the future for the operators +a tool can be provided to clean them up. + +In order to maintain parity between the request format of +``PUT /allocations/{consumer_uuid}`` and response format of +``GET /allocations/{consumer_uuid}``, the ``consumer_type`` key will also be +exposed through the response of ``GET /allocations/{consumer_uuid}`` request. + +The external services will be able to leverage this ``consumer_type`` key +through the ``GET /usages`` REST API which will have a change in the format +of its request and response. The request will gain a new optional key called +``consumer_type`` which will enable users to query usages based on the consumer +type. The response will group the resource usages by the specified +consumer_type (if consumer_type key is not specified it will return the usages +for all the consumer_types) meaning it will gain a new ``consumer_type`` key. +Per consumer type we will also return a ``consumer_count`` of consumers of that +type. + +See the `REST API impact`_ section for more details on how this would be done. + +The above REST API changes and the corresponding changes to the ``/reshaper`` +REST API will be available from a new microversion. + +The existing consumers in placement will have a ``NULL`` value in their +consumer_type_id field, which means we do not know what type these consumers +are and the service to which the consumers belong to needs to update this +information if it wants to avail the ``consumer_types`` feature. + +Alternatives +------------ + +We could create a new REST API to allow users to create consumer types +explicitly but it does not make sense to add a new API for a non-user facing +feature. + +Data model impact +----------------- + +The placement database will get a new ``consumer_types`` table and the +``consumers`` table will get a new ``consumer_type_id`` column that by default +will be ``NULL``. + +REST API impact +--------------- + +The new ``POST /allocations`` request will look like this:: + + { + "30328d13-e299-4a93-a102-61e4ccabe474": { + "consumer_generation": 1, + "project_id": "131d4efb-abc0-4872-9b92-8c8b9dc4320f", + "user_id": "131d4efb-abc0-4872-9b92-8c8b9dc4320f", + "consumer_type": "INSTANCE", # This is new + "allocations": { + "e10927c4-8bc9-465d-ac60-d2f79f7e4a00": { + "resources": { + "VCPU": 2, + "MEMORY_MB": 3 + }, + "generation": 4 + } + } + }, + "71921e4e-1629-4c5b-bf8d-338d915d2ef3": { + "consumer_generation": 1, + "project_id": "131d4efb-abc0-4872-9b92-8c8b9dc4320f", + "user_id": "131d4efb-abc0-4872-9b92-8c8b9dc4320f", + "consumer_type": "MIGRATION", # This is new + "allocations": {} + } + } + +The new ``PUT /allocations/{consumer_uuid}`` request will look like this:: + + { + "allocations": { + "4e061c03-611e-4caa-bf26-999dcff4284e": { + "resources": { + "DISK_GB": 20 + } + }, + "89873422-1373-46e5-b467-f0c5e6acf08f": { + "resources": { + "MEMORY_MB": 1024, + "VCPU": 1 + } + } + }, + "consumer_generation": 1, + "user_id": "66cb2f29-c86d-47c3-8af5-69ae7b778c70", + "project_id": "42a32c07-3eeb-4401-9373-68a8cdca6784", + "consumer_type": "INSTANCE" # This is new + } + +Note that ``consumer_type`` is a required key for both these requests at +this microversion. + +The new ``GET /usages`` response will look like this for a request of type +``GET /usages?project_id=&user_id=`` or +``GET /usages?project_id=`` where the consumer_type key is not +specified:: + + { + "usages": { + "INSTANCE": { + "consumer_count": 5, + "DISK_GB": 5, + "MEMORY_MB": 512, + "VCPU": 2 + } + "MIGRATION": { + "consumer_count": 2, + "DISK_GB": 5, + "MEMORY_MB": 512, + "VCPU": 2 + } + "unknown": { + "consumer_count": 1, + "DISK_GB": 5, + "MEMORY_MB": 512, + "VCPU": 2 + } + } + } + +The new ``GET /usages`` response will look like this for a request of type +``GET /usages?project_id=&user_id=&consumer_type="INSTANCE"`` +or ``GET /usages?project_id=&consumer_type="INSTANCE"`` where the +consumer_type key is specified:: + + { + "usages": { + "INSTANCE": { + "consumer_count": 5, + "DISK_GB": 5, + "MEMORY_MB": 512, + "VCPU": 2 + } + } + } + +A special request of the form +``GET /usages?project_id=&consumer_type=all`` will be allowed to +enable users to be able to query for the total count of all the consumers. The +response for such a request will look like this:: + + { + "usages": { + "all": { + "consumer_count": 3, + "DISK_GB": 5, + "MEMORY_MB": 512, + "VCPU": 2 + } + } + } + +A special request of the form +``GET /usages?project_id=&consumer_type=unknown`` will be allowed +to enable users to be able to query for the total count of the consumers that +have no consumer type assigned. The response for such a request will look like +this:: + + { + "usages": { + "unknown": { + "consumer_count": 3, + "DISK_GB": 5, + "MEMORY_MB": 512, + "VCPU": 2 + } + } + } + +Note that ``consumer_type`` is an optional key for the ``GET /usages`` request. + +The above REST API changes and the corresponding changes to the ``/reshaper`` +REST API will be available from a new microversion. + +Security impact +--------------- + +None. + +Notifications impact +-------------------- + +N/A + +Other end user impact +--------------------- + +The external services using this feature like nova should take the +responsibility of updating the consumer type of existing consumers +from ``NULL`` to the actual type through the +``PUT /allocations/{consumer_uuid}`` REST API. + +Performance Impact +------------------ + +None. + +Other deployer impact +--------------------- + +None. + +Developer impact +---------------- + +None. + +Upgrade impact +-------------- + +The ``placement-manage db sync`` command has to be run by the operators in +order to upgrade the database schema to accommodate the new changes. + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + + +Other contributors: + + + +Work Items +---------- + +* Add the new ``consumer_types`` table and create a new ``consumer_type_id`` + column in the ``consumers`` table with a foreign key constraint to the ``id`` + column of the ``consumer_types`` table. +* Make the REST API changes in a new microversion for: + + * ``POST /allocations``, + * ``PUT /allocations/{consumer_uuid}``, + * ``GET /allocations/{consumer_uuid}``, + * ``GET /usages`` and + * ``/reshaper`` + +Dependencies +============ + +None. + + +Testing +======= + +Unit and functional tests to validate the feature will be added. + + +Documentation Impact +==================== + +The placement API reference will be updated to reflect the new changes. + +References +========== + +.. _quota calculation system: https://review.opendev.org/#/q/topic:bp/count-quota-usage-from-placement + + +History +======= + +.. list-table:: Revisions + :header-rows: 1 + + * - Release Name + - Description + * - Train + - Introduced + * - Xena + - Reproposed