Negative member_of query with microversion 1.32

This patch adds microversion 1.32 supporting the forbidden aggregate
expression within existing ``member_of`` queryparam both in
``GET /resource_providers`` and in ``GET /allocation_candidates``.
Forbidden aggregates are prefixed with a ``!``.

We do NOT support ``!`` within the ``in:`` list:

  ?member_of=in:<agg1>,<agg2>,!<agg3>

but we support ``!in:`` prefix:

  ?member_of=!in:<agg1>,<agg2>,<agg3>

which is equivalent to:

  ?member_of=!<agg1>&member_of=!<agg2>&member_of=!<agg3>

where candidate resource providers must not be in agg1, agg2, or agg3.

Change-Id: Ibba7981744c71ab5d4d0ee5d5a40709c6a5c6b5e
Story: 2005297
Task: 30183

api-ref/source/parameters.yaml

@@ -102,6 +102,36 @@ member_of: &member_of
     any of aggregates B or C, the user could issue the following query::
 
         member_of=AGGA_UUID&member_of=in:AGGB_UUID,AGGC_UUID
+
+    **Starting from microversion 1.32** specifying forbidden aggregates is
+    supported in the ``member_of`` query string parameter. Forbidden aggregates
+    are prefixed with a ``!``. This negative expression can also be used in
+    multiple ``member_of`` parameters::
+
+        member_of=AGGA_UUID&member_of=!AGGB_UUID
+
+    would translate logically to "Candidate resource providers must be in aggA
+    and *not* in AGGB.
+
+    We do NOT support ``!`` on the values within ``in:``, but we support
+    ``!in:``. Both of the following two example queries return candidate
+    resource providers that are NOT in AGGA, AGGB, or AGGC::
+
+        member_of=!in:AGGA_UUID,AGGB_UUID,AGGC_UUID
+        member_of=!AGGA_UUID&member_of=!AGGB_UUID&member_of=!AGGC_UUID
+
+    We do not check if the same aggregate uuid is in both positive and negative
+    expression to raise 400 BadRequest. We still return 200 for such cases.
+    For example::
+
+        member_of=AGGA_UUID&member_of=!AGGA_UUID
+
+    would return empty ``allocation_requests`` and ``provider_summaries``,
+    while::
+
+        member_of=in:AGGA_UUID,AGGB_UUID&member_of=!AGGA_UUID
+
+    would return resource providers that are NOT in AGGA but in AGGB.
   min_version: 1.3
 member_of_1_21:
   <<: *member_of
@@ -124,6 +154,35 @@ member_of_1_21:
     could issue the following query::
 
         member_of=AGGA_UUID&member_of=in:AGGB_UUID,AGGC_UUID
+
+    **Starting from microversion 1.32** specifying forbidden aggregates is
+    supported in the ``member_of`` query string parameter. Forbidden aggregates
+    are prefixed with a ``!``. This negative expression can also be used in
+    multiple ``member_of`` parameters::
+
+        member_of=AGGA_UUID&member_of=!AGGB_UUID
+
+    would translate logically to "Candidate resource providers must be in AGGA
+    and *not* in AGGB.
+
+    We do NOT support ``!`` on the values within ``in:``, but we support
+    ``!in:``. Both of the following two example queries return candidate
+    resource providers that are NOT in AGGA, AGGB, or AGGC::
+
+        member_of=!in:AGGA_UUID,AGGB_UUID,AGGC_UUID
+        member_of=!AGGA_UUID&member_of=!AGGB_UUID&member_of=!AGGC_UUID
+
+    We do not check if the same aggregate uuid is in both positive and negative
+    expression to return 400 BadRequest. We still return 200 for such cases.
+    For example::
+        member_of=AGGA_UUID&member_of=!AGGA_UUID
+
+    would return empty ``allocation_requests`` and ``provider_summaries``,
+    while::
+
+        member_of=in:AGGA_UUID,AGGB_UUID&member_of=!AGGA_UUID
+
+    would return resource providers that are NOT in AGGA but in AGGB.
   min_version: 1.21
 member_of_granular:
   type: string
@@ -133,14 +192,22 @@ member_of_granular:
     A string representing an aggregate uuid; or the prefix ``in:`` followed by
     a comma-separated list of strings representing aggregate uuids. The
     returned resource providers must directly be associated with at least one
-    of the aggregates identified by uuid. The parameter key is ``member_ofN``,
+    of the aggregates identified by uuid.
-    where ``N`` represents a positive integer suffix corresponding with a
+
-    ``resourcesN`` parameter.  The value format is the same as for the
+    **Starting from microversion 1.32** specifying forbidden aggregates is
-    (unnumbered) ``member_of`` parameter; but all of the resources and traits
+    supported. Forbidden aggregates are expressed with a ``!`` prefix; or the
-    specified in a numbered grouping will always be satisfied by the same
+    prefix ``!in:`` followed by a comma-separated list of strings representing
-    resource provider. Separate groupings - numbered or unnumbered - may or may
+    aggregate uuids. The returned resource providers must not directly be
-    not be satisfied by the same provider, depending on the value of the
+    associated with any of the aggregates identified by uuid.
-    ``group_policy`` parameter.
+
+    The parameter key is ``member_ofN``, where ``N`` represents a positive
+    integer suffix corresponding with a ``resourcesN`` parameter.  The value
+    format is the same as for the (unnumbered) ``member_of`` parameter; but
+    all of the resources and traits specified in a numbered grouping will
+    always be satisfied by the same resource provider.
+    Separate groupings - numbered or unnumbered - may or may not be satisfied
+    by the same provider, depending on the value of the ``group_policy``
+    parameter.
 
     It is an error to specify a ``member_ofN`` parameter without a
     corresponding ``resourcesN`` parameter with the same suffix.

placement/handlers/resource_provider.py

@@ -222,7 +222,8 @@ def list_resource_providers(req):
     # special handling of member_of qparam since we allow multiple member_of
     # params at microversion 1.24.
     if 'member_of' in req.GET:
-        filters['member_of'] = util.normalize_member_of_qs_params(req)
+        filters['member_of'], filters['forbidden_aggs'] = (
+            util.normalize_member_of_qs_params(req))
 
     qpkeys = ('uuid', 'name', 'in_tree', 'resources', 'required')
     for attr in qpkeys:

placement/lib.py

@@ -35,7 +35,7 @@ _QS_KEY_PATTERN = re.compile(
 class RequestGroup(object):
     def __init__(self, use_same_provider=True, resources=None,
                  required_traits=None, forbidden_traits=None, member_of=None,
-                 in_tree=None):
+                 in_tree=None, forbidden_aggs=None):
         """Create a grouping of resource and trait requests.
 
         :param use_same_provider:
@@ -57,6 +57,7 @@ class RequestGroup(object):
         self.forbidden_traits = forbidden_traits or set()
         self.member_of = member_of or []
         self.in_tree = in_tree
+        self.forbidden_aggs = forbidden_aggs or set()
 
     def __str__(self):
         ret = 'RequestGroup(use_same_provider=%s' % str(self.use_same_provider)
@@ -101,8 +102,8 @@ class RequestGroup(object):
                 # request group.
                 # TODO(jaypipes): Do validation of query parameters using
                 # JSONSchema
-                request_group.member_of = util.normalize_member_of_qs_params(
+                request_group.member_of, request_group.forbidden_aggs = (
-                    req, suffix)
+                    util.normalize_member_of_qs_params(req, suffix))
             elif prefix == _QS_IN_TREE:
                 request_group.in_tree = util.normalize_in_tree_qs_params(
                     val)
@@ -119,7 +120,8 @@ class RequestGroup(object):
                 'Found the following orphaned traits keys: %s')
             raise webob.exc.HTTPBadRequest(msg % ', '.join(orphans))
         orphans = [('member_of%s' % suff) for suff, group in by_suffix.items()
-                   if group.member_of and not group.resources]
+                   if not group.resources and (
+                       group.member_of or group.forbidden_aggs)]
         if orphans:
             msg = ('All member_of parameters must be associated with '
                    'resources. Found the following orphaned member_of '

placement/microversion.py

@@ -79,6 +79,8 @@ VERSIONS = [
              # inventories and allocations.
     '1.31',  # Add in_tree and in_tree<N> queryparam on
              # `GET /allocation_candidates` API
+    '1.32',  # Support negative member_of queryparams on
+             # `GET /resource_providers` and `GET /allocation_candidates`
 ]
 
 

placement/objects/allocation_candidate.py

@@ -121,6 +121,8 @@ class AllocationCandidates(object):
                 trait_map.update(trait_obj.ids_from_names(context, traits))
 
         member_of = request.member_of
+        forbidden_aggs = request.forbidden_aggs
+
         tree_root_id = None
         if request.in_tree:
             tree_ids = rp_obj.provider_ids_from_uuid(context, request.in_tree)
@@ -132,9 +134,6 @@ class AllocationCandidates(object):
             LOG.debug("getting allocation candidates in the same tree "
                       "with the root provider %s", tree_ids.root_uuid)
 
-        # TODO(tetsuro): get the forbidden aggregates from the request
-        forbidden_aggs = []
-
         any_sharing = any(sharing_providers.values())
         if not request.use_same_provider and (has_trees or any_sharing):
             # TODO(jaypipes): The check/callout to handle trees goes here.

placement/rest_api_version_history.rst

@@ -562,3 +562,40 @@ and ``DISK_GB`` resources from ``sharing1`` might look like::
   ?resources=VCPU:1&in_tree=<myhost_uuid>
   &resources1=VGPU:1&in_tree1=<myhost_uuid>
   &resources2=DISK_GB:100&in_tree2=<sharing1_uuid>
+
+
+Train
+-----
+
+1.32 - Support forbidden aggregates
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: Train
+
+Add support for forbidden aggregates in ``member_of`` queryparam
+in ``GET /resource_providers`` and ``GET /allocation_candidates``.
+Forbidden aggregates are prefixed with a ``!``.
+
+This negative expression can also be used in multiple ``member_of``
+parameters::
+
+    ?member_of=in:<agg1>,<agg2>&member_of=<agg3>&member_of=!<agg4>
+
+would translate logically to
+
+"Candidate resource providers must be at least one of agg1 or agg2,
+definitely in agg3 and definitely *not* in agg4."
+
+We do NOT support ``!`` within the ``in:`` list::
+
+    ?member_of=in:<agg1>,<agg2>,!<agg3>
+
+but we support ``!in:`` prefix::
+
+    ?member_of=!in:<agg1>,<agg2>,<agg3>
+
+which is equivalent to::
+
+    ?member_of=!<agg1>&member_of=!<agg2>&member_of=!<agg3>``
+
+where candidate resource providers must not be in agg1, agg2, or agg3.

placement/tests/functional/fixtures/gabbits.py

@@ -361,8 +361,8 @@ class NUMAAggregateFixture(APIFixture):
 
 
 class NonSharedStorageFixture(APIFixture):
-    """An APIFixture that has two compute nodes with local storage that do not
+    """An APIFixture that has three compute nodes with local storage that do
-    use shared storage.
+    not use shared storage.
     """
     def start_fixture(self):
         super(NonSharedStorageFixture, self).start_fixture()
@@ -376,12 +376,14 @@ class NonSharedStorageFixture(APIFixture):
 
         cn1 = tb.create_provider(self.context, 'cn1')
         cn2 = tb.create_provider(self.context, 'cn2')
+        cn3 = tb.create_provider(self.context, 'cn3')
 
         os.environ['CN1_UUID'] = cn1.uuid
         os.environ['CN2_UUID'] = cn2.uuid
+        os.environ['CN3_UUID'] = cn3.uuid
 
         # Populate compute node inventory for VCPU, RAM and DISK
-        for cn in (cn1, cn2):
+        for cn in (cn1, cn2, cn3):
             tb.add_inventory(cn, 'VCPU', 24)
             tb.add_inventory(cn, 'MEMORY_MB', 128 * 1024)
             tb.add_inventory(cn, 'DISK_GB', 2000)

placement/tests/functional/gabbits/allocation-candidates-bug-1792503.yaml

@@ -7,7 +7,7 @@ defaults:
     request_headers:
         x-auth-token: admin
         accept: application/json
-        openstack-api-version: placement 1.29
+        openstack-api-version: placement 1.32
 
 tests:
 
@@ -104,3 +104,71 @@ tests:
       $.allocation_requests.`len`: 1
       $.allocation_requests..allocations["$ENVIRON['NUMA1_1_UUID']"].resources.VCPU: 1
       $.allocation_requests..allocations["$ENVIRON['SS2_UUID']"].resources.DISK_GB: 1000
+
+# Tests for negative aggregate membership from microversion 1.32.
+# The negative aggregate feature had not yet been implemented when bug1792503
+# was reported, but we include the tests here to make sure that it is
+# consistent with the positive aggregate strategy with nested providers above.
+- name: get allocation candidates with shared storage without aggregate A
+  GET: /allocation_candidates?resources=VCPU:1,DISK_GB:1000&member_of=!$ENVIRON['AGGA_UUID']
+  response_json_paths:
+      # Aggregate A is on the root rps (both cn1 and cn2) so it spans on the
+      # whole tree. We have no allocation requests here.
+      $.allocation_requests.`len`: 0
+
+- name: get allocation candidates with shared storage without aggregate B
+  GET: /allocation_candidates?resources=VCPU:1,DISK_GB:1000&member_of=!$ENVIRON['AGGB_UUID']
+  response_json_paths:
+      # Aggregate B is on the root of cn2 and it spans on the whole tree
+      # including rps of NUMA2_1 and NUMA2_2 so we exclude them.
+      # As a result, there should be 4 allocation candidates:
+      #   [
+      #     (numa1-1, ss1), (numa1-2, ss1),
+      #     (numa1-1, ss2), (numa1-2, ss2),
+      #   ]
+      $.allocation_requests.`len`: 4
+      $.allocation_requests..allocations["$ENVIRON['NUMA1_1_UUID']"].resources.VCPU: [1, 1]
+      $.allocation_requests..allocations["$ENVIRON['NUMA1_2_UUID']"].resources.VCPU: [1, 1]
+      $.allocation_requests..allocations["$ENVIRON['SS1_UUID']"].resources.DISK_GB: [1000, 1000]
+      $.allocation_requests..allocations["$ENVIRON['SS2_UUID']"].resources.DISK_GB: [1000, 1000]
+
+- name: get allocation candidates with shared storage without aggregate C
+  GET: /allocation_candidates?resources=VCPU:1,DISK_GB:1000&member_of=!$ENVIRON['AGGC_UUID']
+  response_json_paths:
+      # Aggregate C is *NOT* on the root. We should exclude NUMA1_1 and SS2,
+      # but we should get NUMA1_2
+      #    [
+      #      (numa1-2, ss1), (numa2-1, ss1), (numa2-2, ss1)
+      #    ]
+      $.allocation_requests.`len`: 3
+      $.allocation_requests..allocations["$ENVIRON['NUMA1_2_UUID']"].resources.VCPU: 1
+      $.allocation_requests..allocations["$ENVIRON['NUMA2_1_UUID']"].resources.VCPU: 1
+      $.allocation_requests..allocations["$ENVIRON['NUMA2_2_UUID']"].resources.VCPU: 1
+      $.allocation_requests..allocations["$ENVIRON['SS1_UUID']"].resources.DISK_GB: [1000, 1000, 1000]
+
+- name: get allocation candidates with shared storage in (aggA or aggB) and (not aggC)
+  GET: /allocation_candidates?resources=VCPU:1,DISK_GB:1000&member_of=in:$ENVIRON['AGGA_UUID'],$ENVIRON['AGGB_UUID']&member_of=!$ENVIRON['AGGC_UUID']
+  response_json_paths:
+      # Aggregate C is *NOT* on the root. We should exclude NUMA1_1 and SS2,
+      # but we should get NUMA1_2
+      #    [
+      #      (numa1-2, ss1), (numa2-1, ss1), (numa2-2, ss1)
+      #    ]
+      $.allocation_requests.`len`: 3
+      $.allocation_requests..allocations["$ENVIRON['NUMA1_2_UUID']"].resources.VCPU: 1
+      $.allocation_requests..allocations["$ENVIRON['NUMA2_1_UUID']"].resources.VCPU: 1
+      $.allocation_requests..allocations["$ENVIRON['NUMA2_2_UUID']"].resources.VCPU: 1
+      $.allocation_requests..allocations["$ENVIRON['SS1_UUID']"].resources.DISK_GB: [1000, 1000, 1000]
+
+- name: get allocation candidates with shared storage neither in aggB nor in aggC but in aggA
+  GET: /allocation_candidates?resources=VCPU:1,DISK_GB:1000&member_of=$ENVIRON['AGGA_UUID']&member_of=!in:$ENVIRON['AGGB_UUID'],$ENVIRON['AGGC_UUID']
+  response_json_paths:
+      # Aggregate B is on the root. We should exclude all the rps on CN2
+      # Aggregate C is *NOT* on the root. We should exclude NUMA1_1 and SS2,
+      # but we should get NUMA1_1
+      #    [
+      #      (numa1-1, ss1)
+      #    ]
+      $.allocation_requests.`len`: 1
+      $.allocation_requests..allocations["$ENVIRON['NUMA1_2_UUID']"].resources.VCPU: 1
+      $.allocation_requests..allocations["$ENVIRON['SS1_UUID']"].resources.DISK_GB: 1000

placement/tests/functional/gabbits/allocation-candidates-member-of.yaml

@@ -31,7 +31,7 @@ tests:
   GET: /allocation_candidates?resources=VCPU:1,MEMORY_MB:1024,DISK_GB:100&member_of=$ENVIRON['AGGA_UUID'],$ENVIRON['AGGB_UUID']
   status: 400
   response_strings:
-      - Multiple values for 'member_of' must be prefixed with the 'in:' keyword
+      - Multiple values for 'member_of' must be prefixed with the 'in:' or '!in:' keyword using the valid microversion.
 
 - name: get allocation candidates multiple member_of with 'in:' but invalid values
   GET: /allocation_candidates?resources=VCPU:1,MEMORY_MB:1024,DISK_GB:100&member_of=in:$ENVIRON['AGGA_UUID'],INVALID_UUID
@@ -139,3 +139,115 @@ tests:
   status: 200
   response_json_paths:
       $.allocation_requests.`len`: 1
+
+# Tests for negative aggregate membership from microversion 1.32
+# Now the aggregation map is as below
+# {
+#  CN1: [AGGA, AGGC],
+#  CN2: [AGGA, AGGB],
+#  CN3: []
+# }
+- name: negative agg error on old microversion with ! prefix
+  GET: /allocation_candidates?resources=VCPU:1&member_of=!$ENVIRON['AGGA_UUID']
+  status: 400
+  request_headers:
+    openstack-api-version: placement 1.31
+  response_strings:
+      - "Forbidden member_of parameters are not supported in the specified microversion"
+
+- name: negative agg error on old microversion with !in prefix
+  GET: /allocation_candidates?resources=VCPU:1&member_of=!in:$ENVIRON['AGGA_UUID']
+  status: 400
+  request_headers:
+    openstack-api-version: placement 1.31
+  response_strings:
+      - "Forbidden member_of parameters are not supported in the specified microversion"
+
+- name: negative agg error on orphaned queryparam
+  GET: /allocation_candidates?member_of=!$ENVIRON['AGGA_UUID']
+  status: 400
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_strings:
+      - "All member_of parameters must be associated with resources"
+
+- name: negative agg error on invalid agg
+  GET: /allocation_candidates?resources=VCPU:1&member_of=!(^o^)
+  status: 400
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_strings:
+      - "Invalid query string parameters: Expected 'member_of' parameter to contain valid UUID(s)."
+
+- name: negative agg error on invalid usage of in prefix
+  GET: /allocation_candidates?resources=VCPU:1&member_of=in:$ENVIRON['AGGA_UUID'],!$ENVIRON['AGGB_UUID']
+  status: 400
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_strings:
+      - "Invalid query string parameters: Expected 'member_of' parameter to contain valid UUID(s)."
+
+- name: negative agg
+  GET: /allocation_candidates?resources=VCPU:1&member_of=!$ENVIRON['AGGC_UUID']
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+    # CN1 is excluded
+    $.allocation_requests.`len`: 2
+    $.provider_summaries.`len`: 2
+    $.allocation_requests..allocations["$ENVIRON['CN2_UUID']"].resources.VCPU: 1
+    $.allocation_requests..allocations["$ENVIRON['CN3_UUID']"].resources.VCPU: 1
+
+- name: negative agg multiple
+  GET: /allocation_candidates?resources=VCPU:1&member_of=!in:$ENVIRON['AGGB_UUID'],$ENVIRON['AGGC_UUID']
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+    # Both CN1 and CN2 are excluded
+    $.allocation_requests.`len`: 1
+    $.provider_summaries.`len`: 1
+    $.allocation_requests..allocations["$ENVIRON['CN3_UUID']"].resources.VCPU: 1
+
+- name: negative agg with positive agg
+  GET: /allocation_candidates?resources=VCPU:1&member_of=!$ENVIRON['AGGB_UUID']&member_of=$ENVIRON['AGGC_UUID']
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+    # Only CN1 is returned
+    $.allocation_requests.`len`: 1
+    $.provider_summaries.`len`: 1
+    $.allocation_requests..allocations["$ENVIRON['CN1_UUID']"].resources.VCPU: 1
+
+- name: negative agg multiple with positive agg
+  GET: /allocation_candidates?resources=VCPU:1&member_of=!in:$ENVIRON['AGGB_UUID'],$ENVIRON['AGGC_UUID']&member_of=$ENVIRON['AGGA_UUID']
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+    # no rp is returned
+    $.allocation_requests.`len`: 0
+    $.provider_summaries.`len`: 0
+
+# This request is equivalent to the one in "negative agg with positive agg"
+- name: negative agg with the same agg on positive get rp
+  GET: /allocation_candidates?resources=VCPU:1&member_of=!$ENVIRON['AGGB_UUID']&member_of=in:$ENVIRON['AGGB_UUID'],$ENVIRON['AGGC_UUID']
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+    $.allocation_requests.`len`: 1
+    $.provider_summaries.`len`: 1
+    $.allocation_requests..allocations["$ENVIRON['CN1_UUID']"].resources.VCPU: 1
+
+- name: negative agg with the same agg on positive no rp
+  GET: /allocation_candidates?resources=VCPU:1&member_of=!$ENVIRON['AGGB_UUID']&member_of=$ENVIRON['AGGB_UUID']
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+    # no rp is returned
+    $.allocation_requests.`len`: 0
+    $.provider_summaries.`len`: 0

placement/tests/functional/gabbits/microversion.yaml

@@ -41,13 +41,13 @@ tests:
   response_json_paths:
       $.errors[0].title: Not Acceptable
 
-- name: latest microversion is 1.31
+- name: latest microversion is 1.32
   GET: /
   request_headers:
       openstack-api-version: placement latest
   response_headers:
       vary: /openstack-api-version/
-      openstack-api-version: placement 1.31
+      openstack-api-version: placement 1.32
 
 - name: other accept header bad version
   GET: /

placement/tests/functional/gabbits/resource-provider-aggregates.yaml

@@ -26,6 +26,13 @@ tests:
       uuid: 5202c48f-c960-4eec-bde3-89c4f22a17b9
   status: 200
 
+- name: post new provider 3
+  POST: /resource_providers
+  data:
+      name: rp_3
+      uuid: 0621521c-ad3a-4f9c-9b72-2933788fab19
+  status: 200
+
 - name: get by aggregates no result
   GET: '/resource_providers?member_of=in:83a3d69d-8920-48e2-8914-cadfd8fa2f91'
   response_json_paths:
@@ -179,3 +186,114 @@ tests:
       # Only rp2 returned since it's the only one associated with the duplicated agg
       $.resource_providers.`len`: 1
       $.resource_providers[0].uuid: /893337e9-1e55-49f0-bcfe-6a2f16fbf2f7/
+
+# Tests for negative aggregate membership from microversion 1.32
+# Now the aggregation map is as below
+# {
+#  893337e9-1e55-49f0-bcfe-6a2f16fbf2f7 (rp_1):
+#      [83a3d69d-8920-48e2-8914-cadfd8fa2f91, 282d469e-29e2-4a8a-8f2e-31b3202b696a]
+#  5202c48f-c960-4eec-bde3-89c4f22a17b9 (rp_2)
+#      [83a3d69d-8920-48e2-8914-cadfd8fa2f91, 99652f11-9f77-46b9-80b7-4b1989be9f8c]
+#  0621521c-ad3a-4f9c-9b72-2933788fab19 (rp_3):
+#      []
+# }
+- name: negative agg error on old microversion with ! prefix
+  GET: /resource_providers?member_of=!282d469e-29e2-4a8a-8f2e-31b3202b696a
+  status: 400
+  request_headers:
+    openstack-api-version: placement 1.31
+  response_strings:
+      - "Forbidden member_of parameters are not supported in the specified microversion"
+
+- name: negative agg error on old microversion with !in prefix
+  GET: /allocation_candidates?resources=VCPU:1&member_of=!in:282d469e-29e2-4a8a-8f2e-31b3202b696a
+  status: 400
+  request_headers:
+    openstack-api-version: placement 1.31
+  response_strings:
+      - "Forbidden member_of parameters are not supported in the specified microversion"
+
+- name: negative agg error on invalid agg
+  GET: /resource_providers?member_of=!(^o^)
+  status: 400
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_strings:
+      - "Invalid query string parameters: Expected 'member_of' parameter to contain valid UUID(s)."
+
+- name: negative agg error on invalid usage of in prefix
+  GET: /resource_providers?resources=VCPU:1&member_of=in:99652f11-9f77-46b9-80b7-4b1989be9f8c,!282d469e-29e2-4a8a-8f2e-31b3202b696a
+  status: 400
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_strings:
+      - "Invalid query string parameters: Expected 'member_of' parameter to contain valid UUID(s)."
+
+- name: negative agg
+  GET: /resource_providers?member_of=!282d469e-29e2-4a8a-8f2e-31b3202b696a
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+      # rp_2 is excluded
+      $.resource_providers.`len`: 2
+      $.resource_providers[0].uuid: /5202c48f-c960-4eec-bde3-89c4f22a17b9|0621521c-ad3a-4f9c-9b72-2933788fab19/
+      $.resource_providers[1].uuid: /5202c48f-c960-4eec-bde3-89c4f22a17b9|0621521c-ad3a-4f9c-9b72-2933788fab19/
+
+- name: negative agg multiple
+  GET: /resource_providers?member_of=!282d469e-29e2-4a8a-8f2e-31b3202b696a&member_of=!99652f11-9f77-46b9-80b7-4b1989be9f8c
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+      # Both rp_1 and rp_2 are excluded
+      $.resource_providers.`len`: 1
+      $.resource_providers[0].uuid: 0621521c-ad3a-4f9c-9b72-2933788fab19
+
+- name: negative agg with in prefix
+  GET: /resource_providers?member_of=!in:282d469e-29e2-4a8a-8f2e-31b3202b696a,99652f11-9f77-46b9-80b7-4b1989be9f8c
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+      # The same results as above
+      $.resource_providers.`len`: 1
+      $.resource_providers[0].uuid: 0621521c-ad3a-4f9c-9b72-2933788fab19
+
+- name: negative agg with positive agg
+  GET: /resource_providers?member_of=!282d469e-29e2-4a8a-8f2e-31b3202b696a&member_of=83a3d69d-8920-48e2-8914-cadfd8fa2f91
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+      # only rp_2 is returned
+      $.resource_providers.`len`: 1
+      $.resource_providers[0].uuid: 5202c48f-c960-4eec-bde3-89c4f22a17b9
+
+- name: negative agg multiple with positive agg
+  GET: /resource_providers?member_of=!in:282d469e-29e2-4a8a-8f2e-31b3202b696a,83a3d69d-8920-48e2-8914-cadfd8fa2f91&member_of=99652f11-9f77-46b9-80b7-4b1989be9f8c
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+      # no rp is returned
+      $.resource_providers.`len`: 0
+
+# This request is equivalent to the one in "negative agg with positive agg"
+- name: negative agg with the same agg on positive get rp
+  GET: /resource_providers?member_of=!282d469e-29e2-4a8a-8f2e-31b3202b696a&member_of=in:83a3d69d-8920-48e2-8914-cadfd8fa2f91,282d469e-29e2-4a8a-8f2e-31b3202b696a
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+      $.resource_providers.`len`: 1
+      $.resource_providers[0].uuid: 5202c48f-c960-4eec-bde3-89c4f22a17b9
+
+- name: negative agg with the same agg on positive no rp
+  GET: /resource_providers?member_of=!282d469e-29e2-4a8a-8f2e-31b3202b696a&member_of=282d469e-29e2-4a8a-8f2e-31b3202b696a
+  status: 200
+  request_headers:
+    openstack-api-version: placement 1.32
+  response_json_paths:
+      # no rp is returned
+      $.resource_providers.`len`: 0

placement/tests/unit/test_util.py

@@ -667,6 +667,85 @@ class TestParseQsRequestGroups(testtools.TestCase):
         self.assertRequestGroupsEqual(
             expected, self.do_parse(qs, version=(1, 24)))
 
+    def test_member_of_forbidden_aggs(self):
+        agg1_uuid = uuidsentinel.agg1
+        agg2_uuid = uuidsentinel.agg2
+        agg3_uuid = uuidsentinel.agg3
+        agg4_uuid = uuidsentinel.agg4
+        qs = ('resources=VCPU:2'
+              '&member_of=%s'
+              '&member_of=%s'
+              '&member_of=!%s'
+              '&member_of=!%s' % (
+                  agg1_uuid, agg2_uuid, agg3_uuid, agg4_uuid))
+        expected = [
+            pl.RequestGroup(
+                use_same_provider=False,
+                resources={
+                    'VCPU': 2,
+                },
+                member_of=[
+                    set([agg1_uuid]),
+                    set([agg2_uuid]),
+                ],
+                forbidden_aggs=set(
+                    [agg3_uuid, agg4_uuid]
+                ),
+            ),
+        ]
+        self.assertRequestGroupsEqual(
+            expected, self.do_parse(qs, version=(1, 32)))
+
+    def test_member_of_multiple_forbidden_aggs(self):
+        agg1_uuid = uuidsentinel.agg1
+        agg2_uuid = uuidsentinel.agg2
+        agg3_uuid = uuidsentinel.agg3
+        qs = ('resources=VCPU:2'
+              '&member_of=!in:%s,%s,%s' % (
+                  agg1_uuid, agg2_uuid, agg3_uuid))
+        expected = [
+            pl.RequestGroup(
+                use_same_provider=False,
+                resources={
+                    'VCPU': 2,
+                },
+                forbidden_aggs=set(
+                    [agg1_uuid, agg2_uuid, agg3_uuid]
+                ),
+            ),
+        ]
+        self.assertRequestGroupsEqual(
+            expected, self.do_parse(qs, version=(1, 32)))
+
+    def test_member_of_forbidden_aggs_prior_microversion(self):
+        agg1_uuid = uuidsentinel.agg1
+        agg2_uuid = uuidsentinel.agg2
+        qs = ('resources=VCPU:2'
+              '&member_of=!%s'
+              '&member_of=!%s' % (agg1_uuid, agg2_uuid))
+        self.assertRaises(
+            webob.exc.HTTPBadRequest, self.do_parse, qs, version=(1, 31))
+
+        qs = ('resources=VCPU:2'
+              '&member_of=!in:%s,%s' % (agg1_uuid, agg2_uuid))
+        self.assertRaises(
+            webob.exc.HTTPBadRequest, self.do_parse, qs, version=(1, 31))
+
+    def test_member_of_forbidden_aggs_invalid_usage(self):
+        agg1_uuid = uuidsentinel.agg1
+        agg2_uuid = uuidsentinel.agg2
+        qs = ('resources=VCPU:2'
+              '&member_of=in:%s,!%s' % (agg1_uuid, agg2_uuid))
+        self.assertRaises(
+            webob.exc.HTTPBadRequest, self.do_parse, qs, version=(1, 32))
+
+        agg1_uuid = uuidsentinel.agg1
+        agg2_uuid = uuidsentinel.agg2
+        qs = ('resources=VCPU:2'
+              '&member_of=!%s,!%s' % (agg1_uuid, agg2_uuid))
+        self.assertRaises(
+            webob.exc.HTTPBadRequest, self.do_parse, qs, version=(1, 32))
+
     def test_400_malformed_resources(self):
         # Somewhat duplicates TestNormalizeResourceQsParam.test_400*.
         qs = ('resources=VCPU:0,MEMORY_MB:4096,DISK_GB:10'

placement/util.py

@@ -340,54 +340,82 @@ def normalize_traits_qs_param(val, allow_forbidden=False):
 def normalize_member_of_qs_params(req, suffix=''):
     """Given a webob.Request object, validate that the member_of querystring
     parameters are correct. We begin supporting multiple member_of params in
-    microversion 1.24.
+    microversion 1.24 and forbidden aggregates in microversion 1.32.
 
     :param req: webob.Request object
-    :return: A list containing sets of UUIDs of aggregates to filter on
+    :return: A tuple of
+        required_aggs: A list containing sets of UUIDs of required
+                       aggregates to filter on
+        forbidden_aggs: A set of UUIDs of forbidden aggregates to filter on
     :raises `webob.exc.HTTPBadRequest` if the microversion requested is <1.24
             and the request contains multiple member_of querystring params
+    :raises `webob.exc.HTTPBadRequest` if the microversion requested is <1.32
+            and the request contains forbidden format of member_of querystring
+            params with '!' prefix
     :raises `webob.exc.HTTPBadRequest` if the val parameter is not in the
             expected format.
     """
     want_version = req.environ[placement.microversion.MICROVERSION_ENVIRON]
     multi_member_of = want_version.matches((1, 24))
+    allow_forbidden = want_version.matches((1, 32))
     if not multi_member_of and len(req.GET.getall('member_of' + suffix)) > 1:
         raise webob.exc.HTTPBadRequest(
             'Multiple member_of%s parameters are not supported' % suffix)
-    values = []
+    required_aggs = []
+    forbidden_aggs = set()
     for value in req.GET.getall('member_of' + suffix):
-        values.append(normalize_member_of_qs_param(value))
+        required, forbidden = normalize_member_of_qs_param(value)
-    return values
+        if required:
+            required_aggs.append(required)
+        if forbidden:
+            if not allow_forbidden:
+                raise webob.exc.HTTPBadRequest(
+                    'Forbidden member_of%s parameters are not supported '
+                    'in the specified microversion' % suffix)
+            forbidden_aggs |= forbidden
+    return required_aggs, forbidden_aggs
 
 
 def normalize_member_of_qs_param(value):
     """Parse a member_of query string parameter value.
 
-    Valid values are either a single UUID, or the prefix 'in:' followed by two
+    Valid values are one of either
-    or more comma-separated UUIDs.
+        - a single UUID
+        - the prefix '!' followed by a single UUID
+        - the prefix 'in:' or '!in:' followed by two or more
+          comma-separated UUIDs.
 
-    :param value: A member_of query parameter of either a single UUID, or a
+    :param value: A member_of query parameter
-                  comma-separated string of two or more UUIDs, prefixed with
+    :return: A tuple of:
-                  the "in:" operator
+        required: A set of aggregate UUIDs at least one of which is required
-    :return: A set of UUIDs
+        forbidden: A set of aggregate UUIDs all of which are forbidden
     :raises `webob.exc.HTTPBadRequest` if the value parameter is not in the
             expected format.
     """
-    if "," in value and not value.startswith("in:"):
+    if "," in value and not (
+            value.startswith("in:") or value.startswith("!in:")):
         msg = ("Multiple values for 'member_of' must be prefixed with the "
-               "'in:' keyword. Got: %s") % value
+               "'in:' or '!in:' keyword using the valid microversion. "
+               "Got: %s") % value
         raise webob.exc.HTTPBadRequest(msg)
-    if value.startswith('in:'):
+
-        value = set(value[3:].split(','))
+    required = forbidden = set()
+    if value.startswith('!in:'):
+        forbidden = set(value[4:].split(','))
+    elif value.startswith('!'):
+        forbidden = set([value[1:]])
+    elif value.startswith('in:'):
+        required = set(value[3:].split(','))
     else:
-        value = set([value])
+        required = set([value])
+
     # Make sure the values are actually UUIDs.
-    for aggr_uuid in value:
+    for aggr_uuid in (required | forbidden):
         if not uuidutils.is_uuid_like(aggr_uuid):
             msg = ("Invalid query string parameters: Expected 'member_of' "
                    "parameter to contain valid UUID(s). Got: %s") % aggr_uuid
             raise webob.exc.HTTPBadRequest(msg)
-    return value
+    return required, forbidden
 
 
 def normalize_in_tree_qs_params(value):

releasenotes/notes/negative-aggregate-membership-1dde3cbe27c69279.yaml

@@ -0,0 +1,31 @@
+---
+features:
+  - |
+    Add support for forbidden aggregates in ``member_of`` queryparam
+    in ``GET /resource_providers`` and ``GET /allocation_candidates``.
+    Forbidden aggregates are prefixed with a ``!``.
+
+    This negative expression can also be used in multiple ``member_of``
+    parameters::
+
+        ?member_of=in:<agg1>,<agg2>&member_of=<agg3>&member_of=!<agg4>
+
+    would translate logically to
+
+    "Candidate resource providers must be at least one of agg1 or agg2,
+    definitely in agg3 and definitely *not* in agg4."
+
+    We do NOT support ``!`` within the ``in:`` list::
+
+        ?member_of=in:<agg1>,<agg2>,!<agg3>
+
+    but we support ``!in:`` prefix::
+
+        ?member_of=!in:<agg1>,<agg2>,<agg3>
+
+    which is equivalent to::
+
+        ?member_of=!<agg1>&member_of=!<agg2>&member_of=!<agg3>
+
+    where candidate resource providers must not be in agg1, agg2, or agg3.
+