Fix quota class set APIs

v2.1 API which does not return the 'server_groups' and
'server_group_members' quotas in GET & PUT os-quota-class-sets
API response. v2 API used to return those keys in API response.

Also filter out the network related quotas from os-quota-class-sets
APIs

Fixing this with microversion.
Closes-Bug: #1701211
Closes-Bug: #1693168
implement-blueprint fix-quota-classes-api

Change-Id: I66aeb7a92fb5ee906fead78030bd84a2e97916e8

api-ref/source/os-quota-class-sets.inc

@@ -26,14 +26,18 @@ Nova supports implicit 'default' Quota Class only.
     for more details.
 
 .. warning::
-    There is a bug in the v2.1 API and the legacy v2 compatible API which does
+    There is a bug in the v2.1 API till microversion 2.49 and
-    not return the ``server_groups`` and ``server_group_members`` quotas in
+    the legacy v2 compatible API which does not return the
+    ``server_groups`` and ``server_group_members`` quotas in
     GET and PUT ``os-quota-class-sets`` API response, whereas the v2 API
     used to return those keys in the API response.
     There is workaround to get the ``server_groups`` and
     ``server_group_members`` quotas using
     :ref:`list-default-quotas-for-tenant`
     API but that is per project quota.
+    This issue is fixed in microversion 2.50, here onwards
+    ``server_groups`` and ``server_group_members`` keys are
+    returned in API response body.
 
 Show the quota for Quota Class
 ==============================
@@ -72,11 +76,13 @@ Response
   - ram: ram_quota_class
   - security_group_rules: security_group_rules_quota_class
   - security_groups: security_groups_quota_class
+  - server_groups: server_groups_quota_class
+  - server_group_members: server_group_members_quota_class
   - networks: networks_quota_optional
 
-**Example Show A Quota Class: JSON response**
+**Example Show A Quota Class: JSON response(2.50)**
 
-.. literalinclude:: ../../doc/api_samples/os-quota-class-sets/quota-classes-show-get-resp.json
+.. literalinclude:: ../../doc/api_samples/os-quota-class-sets/v2.50/quota-classes-show-get-resp.json
    :language: javascript
 
 Create or Update Quotas for Quota Class
@@ -117,9 +123,9 @@ Request
   - server_group_members: server_group_members_quota_optional
   - networks: networks_quota_optional
 
-**Example Update Quotas: JSON request**
+**Example Update Quotas: JSON request(2.50)**
 
-.. literalinclude:: ../../doc/api_samples/os-quota-class-sets/quota-classes-update-post-req.json
+.. literalinclude:: ../../doc/api_samples/os-quota-class-sets/v2.50/quota-classes-update-post-req.json
    :language: javascript
 
 Response
@@ -140,9 +146,11 @@ Response
   - ram: ram_quota_class
   - security_group_rules: security_group_rules_quota_class
   - security_groups: security_groups_quota_class
+  - server_groups: server_groups_quota_class
+  - server_group_members: server_group_members_quota_class
   - networks: networks_quota_optional
 
-**Example Update Quotas: JSON response**
+**Example Update Quotas: JSON response(2.50)**
 
-.. literalinclude:: ../../doc/api_samples/os-quota-class-sets/quota-classes-update-post-resp.json
+.. literalinclude:: ../../doc/api_samples/os-quota-class-sets/v2.50/quota-classes-update-post-resp.json
    :language: javascript

api-ref/source/parameters.yaml

@@ -3813,6 +3813,7 @@ networks_quota_optional: &networks_quota_optional
   in: body
   required: false
   type: integer
+  max_version: 2.50
 networks_quota_set_optional:
   <<: *networks_quota_optional
   max_version: 2.35
@@ -4746,12 +4747,15 @@ server_group_id_body:
   in: body
   required: true
   type: string
-server_group_members:
+server_group_members: &server_group_members
   description: |
     The number of allowed members for each server group.
   in: body
   required: true
   type: integer
+server_group_members_quota_class:
+  <<: *server_group_members
+  min_version: 2.50
 server_group_members_quota_details:
   description: |
     The object of detailed server group members, including in_use,
@@ -4777,6 +4781,11 @@ server_groups_list:
   in: body
   required: true
   type: array
+server_groups_quota_class:
+  <<: *server_groups
+  description: |
+    The number of allowed server groups for the quota class.
+  min_version: 2.50
 server_groups_quota_class_optional:
   <<: *server_groups
   description: |

doc/api_samples/os-quota-class-sets/v2.50/quota-classes-show-get-resp.json

@@ -0,0 +1,15 @@
+{
+    "quota_class_set": {
+        "cores": 20,
+        "id": "test_class",
+        "injected_file_content_bytes": 10240,
+        "injected_file_path_bytes": 255,
+        "injected_files": 5,
+        "instances": 10,
+        "key_pairs": 100,
+        "metadata_items": 128,
+        "ram": 51200,
+        "server_groups": 10,
+        "server_group_members": 10
+    }
+}

doc/api_samples/os-quota-class-sets/v2.50/quota-classes-update-post-req.json

@@ -0,0 +1,14 @@
+{
+    "quota_class_set": {
+        "instances": 50,
+        "cores": 50,
+        "ram": 51200,
+        "metadata_items": 128,
+        "injected_files": 5,
+        "injected_file_content_bytes": 10240,
+        "injected_file_path_bytes": 255,
+        "key_pairs": 100,
+        "server_groups": 10,
+        "server_group_members": 10
+    }
+}

doc/api_samples/os-quota-class-sets/v2.50/quota-classes-update-post-resp.json

@@ -0,0 +1,14 @@
+{
+    "quota_class_set": {
+        "cores": 50,
+        "injected_file_content_bytes": 10240,
+        "injected_file_path_bytes": 255,
+        "injected_files": 5,
+        "instances": 50,
+        "key_pairs": 100,
+        "metadata_items": 128,
+        "ram": 51200,
+        "server_groups": 10,
+        "server_group_members": 10
+    }
+}

doc/api_samples/versions/v21-version-get-resp.json

@@ -19,7 +19,7 @@
             }
         ],
         "status": "CURRENT",
-        "version": "2.49",
+        "version": "2.50",
         "min_version": "2.1",
         "updated": "2013-07-23T11:33:21Z"
     }

doc/api_samples/versions/versions-get-resp.json

@@ -22,7 +22,7 @@
                 }
             ],
             "status": "CURRENT",
-            "version": "2.49",
+            "version": "2.50",
             "min_version": "2.1",
             "updated": "2013-07-23T11:33:21Z"
         }

nova/api/openstack/api_version_request.py

@@ -116,6 +116,10 @@ REST_API_VERSION_HISTORY = """REST API Version History:
              the output.
     * 2.48 - Standardize VM diagnostics info.
     * 2.49 - Support tagged attachment of network interfaces and block devices.
+    * 2.50 - Exposes ``server_groups`` and ``server_group_members`` keys in
+             GET & PUT ``os-quota-class-sets`` APIs response.
+             Also filter out Network related quotas from
+             ``os-quota-class-sets`` API
 """
 
 # The minimum and maximum versions of the API supported
@@ -124,7 +128,7 @@ REST_API_VERSION_HISTORY = """REST API Version History:
 # Note(cyeoh): This only applies for the v2.1 API once microversions
 # support is fully merged. It does not affect the V2 API.
 _MIN_API_VERSION = "2.1"
-_MAX_API_VERSION = "2.49"
+_MAX_API_VERSION = "2.50"
 DEFAULT_API_VERSION = _MIN_API_VERSION
 
 # Almost all proxy APIs which related to network, images and baremetal

nova/api/openstack/compute/quota_classes.py

@@ -13,9 +13,10 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-
+import copy
 import webob
 
+from nova.api.openstack import api_version_request
 from nova.api.openstack.compute.schemas import quota_classes
 from nova.api.openstack import extensions
 from nova.api.openstack import wsgi
@@ -29,9 +30,14 @@ from nova import utils
 
 QUOTAS = quota.QUOTAS
 
-# Quotas that are only enabled by specific extensions
+# NOTE(gmann): Quotas which were returned in v2 but in v2.1 those
-EXTENDED_QUOTAS = {'server_groups': 'os-server-group-quotas',
+# were not returned. Fixed in microversion 2.50. Bug#1693168.
-                   'server_group_members': 'os-server-group-quotas'}
+EXTENDED_QUOTAS = ['server_groups', 'server_group_members']
+
+# NOTE(gmann): Network related quotas are filter out in
+# microversion 2.50. Bug#1701211.
+FILTERED_QUOTAS = ["fixed_ips", "floating_ips", "networks",
+                   "security_group_rules", "security_groups"]
 
 
 class QuotaClassSetsController(wsgi.Controller):
@@ -40,20 +46,25 @@ class QuotaClassSetsController(wsgi.Controller):
 
     def __init__(self, **kwargs):
         self.supported_quotas = QUOTAS.resources
-        # TODO(jichenjc): need fix v2 and v2.1 API difference here see bug
-        # 1693168 for more info
-        for resource, extension in EXTENDED_QUOTAS.items():
-            self.supported_quotas.remove(resource)
 
-    def _format_quota_set(self, quota_class, quota_set):
+    def _format_quota_set(self, quota_class, quota_set, req):
         """Convert the quota object to a result dict."""
 
         if quota_class:
             result = dict(id=str(quota_class))
         else:
             result = {}
-
+        original_quotas = copy.deepcopy(self.supported_quotas)
-        for resource in self.supported_quotas:
+        if api_version_request.is_supported(req, min_version="2.50"):
+            original_quotas = [resource for resource in original_quotas
+                               if resource not in FILTERED_QUOTAS]
+        # NOTE(gmann): Before microversion v2.50, v2.1 API does not return the
+        # 'server_groups' & 'server_group_members' key in quota class API
+        # response.
+        else:
+            for resource in EXTENDED_QUOTAS:
+                original_quotas.remove(resource)
+        for resource in original_quotas:
             if resource in quota_set:
                 result[resource] = quota_set[resource]
 
@@ -64,10 +75,11 @@ class QuotaClassSetsController(wsgi.Controller):
         context = req.environ['nova.context']
         context.can(qcs_policies.POLICY_ROOT % 'show', {'quota_class': id})
         values = QUOTAS.get_class_quotas(context, id)
-        return self._format_quota_set(id, values)
+        return self._format_quota_set(id, values, req)
 
     @extensions.expected_errors(400)
-    @validation.schema(quota_classes.update)
+    @validation.schema(quota_classes.update, "2.0", "2.49")
+    @validation.schema(quota_classes.update_v250, "2.50")
     def update(self, req, id, body):
         context = req.environ['nova.context']
         context.can(qcs_policies.POLICY_ROOT % 'update', {'quota_class': id})
@@ -87,4 +99,4 @@ class QuotaClassSetsController(wsgi.Controller):
                 db.quota_class_create(context, quota_class, key, value)
 
         values = QUOTAS.get_class_quotas(context, quota_class)
-        return self._format_quota_set(None, values)
+        return self._format_quota_set(None, values, req)

nova/api/openstack/compute/rest_api_version_history.rst

@@ -579,3 +579,19 @@ user documentation.
   be reflected in the config drive.
 
   Tagged volume attachment is not supported for shelved-offloaded instances.
+
+
+2.50
+----
+
+  The ``server_groups`` and ``server_group_members`` keys are exposed in GET & PUT
+  ``os-quota-class-sets`` APIs Response body.
+  Networks related quotas have been filtered out from os-quota-class. Below quotas
+  are filtered out and not available in ``os-quota-class-sets`` APIs from this
+  microversion onwards.
+
+  - "fixed_ips"
+  - "floating_ips"
+  - "networks",
+  - "security_group_rules"
+  - "security_groups"

nova/api/openstack/compute/schemas/quota_classes.py

@@ -11,6 +11,7 @@
 #    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 #    License for the specific language governing permissions and limitations
 #    under the License.
+import copy
 
 from nova.api.openstack.compute.schemas import quota_sets
 
@@ -26,3 +27,12 @@ update = {
     'required': ['quota_class_set'],
     'additionalProperties': False,
 }
+
+update_v250 = copy.deepcopy(update)
+del update_v250['properties']['quota_class_set']['properties']['fixed_ips']
+del update_v250['properties']['quota_class_set']['properties']['floating_ips']
+del update_v250['properties']['quota_class_set']['properties'][
+    'security_groups']
+del update_v250['properties']['quota_class_set']['properties'][
+    'security_group_rules']
+del update_v250['properties']['quota_class_set']['properties']['networks']

nova/tests/functional/api_sample_tests/api_samples/os-quota-class-sets/v2.50/quota-classes-show-get-resp.json.tpl

@@ -0,0 +1,15 @@
+{
+    "quota_class_set": {
+        "cores": 20,
+        "id": "%(set_id)s",
+        "injected_file_content_bytes": 10240,
+        "injected_file_path_bytes": 255,
+        "injected_files": 5,
+        "instances": 10,
+        "key_pairs": 100,
+        "metadata_items": 128,
+        "ram": 51200,
+        "server_groups": 10,
+        "server_group_members": 10
+    }
+}

nova/tests/functional/api_sample_tests/api_samples/os-quota-class-sets/v2.50/quota-classes-update-post-req.json.tpl

@@ -0,0 +1,14 @@
+{
+    "quota_class_set": {
+        "instances": 50,
+        "cores": 50,
+        "ram": 51200,
+        "metadata_items": 128,
+        "injected_files": 5,
+        "injected_file_content_bytes": 10240,
+        "injected_file_path_bytes": 255,
+        "key_pairs": 100,
+        "server_groups": 10,
+        "server_group_members": 10
+    }
+}

nova/tests/functional/api_sample_tests/api_samples/os-quota-class-sets/v2.50/quota-classes-update-post-resp.json.tpl

@@ -0,0 +1,14 @@
+{
+    "quota_class_set": {
+        "cores": 50,
+        "injected_file_content_bytes": 10240,
+        "injected_file_path_bytes": 255,
+        "injected_files": 5,
+        "instances": 50,
+        "key_pairs": 100,
+        "metadata_items": 128,
+        "ram": 51200,
+        "server_groups": 10,
+        "server_group_members": 10
+    }
+}

nova/tests/functional/api_sample_tests/test_quota_classes.py

@@ -35,3 +35,8 @@ class QuotaClassesSampleJsonTests(api_sample_base.ApiSampleTestBaseV21):
                                 {})
         self._verify_response('quota-classes-update-post-resp',
                               {}, response, 200)
+
+
+class QuotaClassesV250SampleJsonTests(QuotaClassesSampleJsonTests):
+    microversion = '2.50'
+    scenarios = [('v2_50', {'api_major_version': 'v2.1'})]

nova/tests/unit/api/openstack/compute/test_quota_classes.py

@@ -12,7 +12,7 @@
 #    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 #    License for the specific language governing permissions and limitations
 #    under the License.
-
+import copy
 import webob
 
 from nova.api.openstack.compute import extension_info
@@ -23,23 +23,26 @@ from nova import test
 from nova.tests.unit.api.openstack import fakes
 
 
-def quota_set(class_name):
-    return {'quota_class_set': {'id': class_name, 'metadata_items': 128,
-                                'ram': 51200, 'floating_ips': 10,
-                                'fixed_ips': -1, 'instances': 10,
-                                'injected_files': 5, 'cores': 20,
-                                'injected_file_content_bytes': 10240,
-                                'security_groups': 10,
-                                'security_group_rules': 20, 'key_pairs': 100,
-                                'injected_file_path_bytes': 255}}
-
-
 class QuotaClassSetsTestV21(test.TestCase):
     validation_error = exception.ValidationError
+    api_version = '2.1'
+    quota_resources = {'metadata_items': 128,
+                       'ram': 51200, 'floating_ips': 10,
+                       'fixed_ips': -1, 'instances': 10,
+                       'injected_files': 5, 'cores': 20,
+                       'injected_file_content_bytes': 10240,
+                       'security_groups': 10,
+                       'security_group_rules': 20, 'key_pairs': 100,
+                       'injected_file_path_bytes': 255}
+
+    def quota_set(self, class_name):
+        quotas = copy.deepcopy(self.quota_resources)
+        quotas['id'] = class_name
+        return {'quota_class_set': quotas}
 
     def setUp(self):
         super(QuotaClassSetsTestV21, self).setUp()
-        self.req = fakes.HTTPRequest.blank('')
+        self.req = fakes.HTTPRequest.blank('', version=self.api_version)
         self._setup()
 
     def _setup(self):
@@ -47,60 +50,46 @@ class QuotaClassSetsTestV21(test.TestCase):
         self.controller = quota_classes_v21.QuotaClassSetsController(
             extension_info=ext_info)
 
-    def test_format_quota_set(self):
+    def _check_filtered_extended_quota(self, quota_set):
-        raw_quota_set = {
+        self.assertNotIn('server_groups', quota_set)
-            'instances': 10,
+        self.assertNotIn('server_group_members', quota_set)
-            'cores': 20,
+        self.assertEqual(10, quota_set['floating_ips'])
-            'ram': 51200,
+        self.assertEqual(-1, quota_set['fixed_ips'])
-            'floating_ips': 10,
+        self.assertEqual(10, quota_set['security_groups'])
-            'fixed_ips': -1,
+        self.assertEqual(20, quota_set['security_group_rules'])
-            'metadata_items': 128,
-            'injected_files': 5,
-            'injected_file_path_bytes': 255,
-            'injected_file_content_bytes': 10240,
-            'security_groups': 10,
-            'security_group_rules': 20,
-            'key_pairs': 100,
-            }
 
+    def test_format_quota_set(self):
         quota_set = self.controller._format_quota_set('test_class',
-                                                      raw_quota_set)
+                                                      self.quota_resources,
+                                                      self.req)
         qs = quota_set['quota_class_set']
 
         self.assertEqual(qs['id'], 'test_class')
         self.assertEqual(qs['instances'], 10)
         self.assertEqual(qs['cores'], 20)
         self.assertEqual(qs['ram'], 51200)
-        self.assertEqual(qs['floating_ips'], 10)
-        self.assertEqual(qs['fixed_ips'], -1)
         self.assertEqual(qs['metadata_items'], 128)
         self.assertEqual(qs['injected_files'], 5)
         self.assertEqual(qs['injected_file_path_bytes'], 255)
         self.assertEqual(qs['injected_file_content_bytes'], 10240)
-        self.assertEqual(qs['security_groups'], 10)
-        self.assertEqual(qs['security_group_rules'], 20)
         self.assertEqual(qs['key_pairs'], 100)
+        self._check_filtered_extended_quota(qs)
 
     def test_quotas_show(self):
         res_dict = self.controller.show(self.req, 'test_class')
 
-        self.assertEqual(res_dict, quota_set('test_class'))
+        self.assertEqual(res_dict, self.quota_set('test_class'))
 
     def test_quotas_update(self):
-        body = {'quota_class_set': {'instances': 50, 'cores': 50,
+        expected_body = {'quota_class_set': self.quota_resources}
-                                    'ram': 51200, 'floating_ips': 10,
+        request_quota_resources = copy.deepcopy(self.quota_resources)
-                                    'fixed_ips': -1, 'metadata_items': 128,
+        request_quota_resources['server_groups'] = 10
-                                    'injected_files': 5,
+        request_quota_resources['server_group_members'] = 10
-                                    'injected_file_content_bytes': 10240,
+        request_body = {'quota_class_set': request_quota_resources}
-                                    'injected_file_path_bytes': 255,
-                                    'security_groups': 10,
-                                    'security_group_rules': 20,
-                                    'key_pairs': 100}}
-
         res_dict = self.controller.update(self.req, 'test_class',
-                                          body=body)
+                                          body=request_body)
 
-        self.assertEqual(res_dict, body)
+        self.assertEqual(res_dict, expected_body)
 
     def test_quotas_update_with_empty_body(self):
         body = {}
@@ -139,6 +128,35 @@ class QuotaClassSetsTestV21(test.TestCase):
                           self.req, 'test_class', body=body)
 
 
+class QuotaClassSetsTestV250(QuotaClassSetsTestV21):
+    api_version = '2.50'
+    quota_resources = {'metadata_items': 128,
+                       'ram': 51200, 'instances': 10,
+                       'injected_files': 5, 'cores': 20,
+                       'injected_file_content_bytes': 10240,
+                       'key_pairs': 100,
+                       'injected_file_path_bytes': 255,
+                       'server_groups': 10,
+                       'server_group_members': 10}
+
+    def _check_filtered_extended_quota(self, quota_set):
+        self.assertEqual(10, quota_set['server_groups'])
+        self.assertEqual(10, quota_set['server_group_members'])
+        self.assertNotIn('floating_ips', quota_set)
+        self.assertNotIn('fixed_ips', quota_set)
+        self.assertNotIn('security_groups', quota_set)
+        self.assertNotIn('security_group_rules', quota_set)
+        self.assertNotIn('networks', quota_set)
+
+    def test_quotas_update_with_filtered_quota(self):
+        filtered_quotas = ["fixed_ips", "floating_ips", "networks",
+                           "security_group_rules", "security_groups"]
+        for resource in filtered_quotas:
+            body = {'quota_class_set': {resource: 10}}
+            self.assertRaises(self.validation_error, self.controller.update,
+                              self.req, 'test_class', body=body)
+
+
 class QuotaClassesPolicyEnforcementV21(test.NoDBTestCase):
 
     def setUp(self):

releasenotes/notes/add-server-groups-keys-in-quota-class-set-response-4a91ef4b2683e31c.yaml

@@ -0,0 +1,20 @@
+---
+fixes:
+  - |
+    Fix bug 1693168. v2.1 API which does not return the ``server_groups`` and
+    ``server_group_members`` quotas in GET and PUT ``os-quota-class-sets`` API
+    response. v2 API used to return those keys in the API response.
+    Microversion 2.50 restored that behavior. In microversion 2.50, the
+    ``server_groups`` and ``server_group_members`` keys are exposed in
+    GET and PUT ``os-quota-class-sets`` APIs Response body.
+    Fix bug 1701211. Network related quotas are filtered out of
+    ``os-quota-class-sets`` APIs and not available from microversion
+    2.50 onwards.
+    Filtered quotas are:
+
+    - 'fixed_ips'
+    - 'floating ips'
+    - 'security_groups'
+    - 'security_group_rules'
+    - 'networks'
+