Standardization of VM diagnostics info API.

Before this patch, VM diagnostics response was just a
'blob' of data returned by each hypervisor. New API
version makes diagnostics response standardized.
New response has a set of fields which each hypervisor
will try to fill. If hypervisor unable to provide a
specific field then this field will be reported as 'None'.

Tempest tests: I7757c5beeea3d3b0bc15a51cafc5ea2ada65e76c

DocImpact: admin guide docs should be updated to mention
standardized version of the diagnostics response

blueprint: restore-vm-diagnostics

Change-Id: If0b1493cc5c1c7f0d9896dd68342ad4dea4f7da2

api-ref/source/diagnostics.inc

@@ -31,12 +31,39 @@ Request
 Response
 --------
 
-The response format for diagnostics is backend hypervisor specific,
+Starting from **microversion 2.48** diagnostics response is standardized
-and not well defined. This should be considered a debug interface
+across all virt drivers. The response should be considered a debug interface
-only, and not relied upon by programmatic tools.
+only and not relied upon by programmatic tools. All response fields are listed
+below. If the virt driver is unable to provide a specific field then this field
+will be reported as ``None`` in the response.
+
+.. rest_parameters:: parameters.yaml
+
+  - config_drive: config_drive_diagnostics
+  - state: vm_state_diagnostics
+  - driver: driver_diagnostics
+  - hypervisor: hypervisor_diagnostics
+  - hypervisor_os: hypervisor_os_diagnostics
+  - uptime: uptime_diagnostics
+  - num_cpus: num_cpus_diagnostics
+  - num_disks: num_disks_diagnostics
+  - num_nics: num_nics_diagnostics
+  - memory_details: memory_details_diagnostics
+  - cpu_details: cpu_details_diagnostics
+  - disk_details: disk_details_diagnostics
+  - nic_details: nic_details_diagnostics
 
 **Example Server diagnostics**
 
+.. literalinclude:: ../../doc/api_samples/os-server-diagnostics/v2.48/server-diagnostics-get-resp.json
+   :language: javascript
+
+.. warning::
+   Before **microversion 2.48** the response format for diagnostics was not
+   well defined. Each hypervisor had its own format.
+
+**Example Old Server diagnostics**
+
 Below is an example of diagnostics for a libvirt based instance. The unit of the return
 value is hypervisor specific, but in this case the unit of vnet1_rx* and
 vnet1_tx* is octets.

api-ref/source/parameters.yaml

@@ -1516,6 +1516,13 @@ config_drive:
   in: body
   required: false
   type: boolean
+config_drive_diagnostics:
+  description: |
+    Indicates whether or not a config drive was used for this server.
+  in: body
+  required: true
+  type: boolean
+  min_version: 2.48
 config_drive_resp:
   description: |
     Indicates whether or not a config drive was used for this server.
@@ -1602,6 +1609,20 @@ cores_quota_optional:
   in: body
   required: false
   type: integer
+cpu_details_diagnostics:
+  description: |
+    The list of dictionaries with detailed information about VM CPUs.
+    Following fields are presented in each dictionary:
+
+    - ``id`` - the ID of CPU (Integer)
+
+    - ``time`` - CPU Time in nano seconds (Integer)
+
+    - ``utilisation`` - CPU utilisation in percents (Integer)
+  in: body
+  required: true
+  type: array
+  min_version: 2.48
 cpu_info:
   description: |
     A dictionary that contains cpu information like ``arch``, ``model``,
@@ -1800,6 +1821,24 @@ disk_config:
   in: body
   required: true
   type: string
+disk_details_diagnostics:
+  description: |
+    The list of dictionaries with detailed information about VM disks.
+    Following fields are presented in each dictionary:
+
+    - ``read_bytes`` - Disk reads in bytes (Integer)
+
+    - ``read_requests`` - Read requests (Integer)
+
+    - ``write_bytes`` - Disk writes in bytes (Integer)
+
+    - ``write_requests`` - Write requests (Integer)
+
+    - ``errors_count`` - Disk errors (Integer)
+  in: body
+  required: true
+  type: array
+  min_version: 2.48
 disk_over_commit:
   description: |
     Set to ``True`` to enable over commit when the destination host is checked for
@@ -1833,6 +1872,19 @@ display_name_optional:
   in: body
   required: false
   type: string
+driver_diagnostics:
+  description: |
+    The driver on which the VM is running. Possible values are:
+
+    - ``libvirt``
+    - ``xenapi``
+    - ``hyperv``
+    - ``vmwareapi``
+    - ``ironic``
+  in: body
+  required: true
+  type: string
+  min_version: 2.48
 ended_at:
   description: |
     The date and time when the server was deleted.
@@ -2701,6 +2753,14 @@ hypervisor_count:
   in: body
   required: true
   type: integer
+hypervisor_diagnostics:
+  description: |
+    The hypervisor on which the VM is running. Examples for libvirt driver
+    may be: ``qemu``, ``kvm`` or ``xen``.
+  in: body
+  required: true
+  type: string
+  min_version: 2.48
 hypervisor_free_disk_gb:
   description: |
     The free disk remaining on this hypervisor(in GB).
@@ -2730,6 +2790,13 @@ hypervisor_links:
   type: array
   min_version: 2.33
   required: false
+hypervisor_os_diagnostics:
+  description: |
+    The hypervisor OS.
+  in: body
+  type: string
+  required: true
+  min_version: 2.48
 hypervisor_service:
   description: |
     The hypervisor service object.
@@ -3341,6 +3408,19 @@ members:
   in: body
   required: true
   type: array
+memory_details_diagnostics:
+  description: |
+    The dictionary with information about VM memory usage.
+    Following fields are presented in the dictionary:
+
+    - ``maximum`` - Amount of memory provisioned for the VM in MB (Integer)
+
+    - ``used`` - Amount of memory that is currently used by the guest operating
+      system and its applications in MB (Integer)
+  in: body
+  required: true
+  type: array
+  min_version: 2.48
 memory_mb:
   description: |
     The memory of this hypervisor(in MB).
@@ -3624,6 +3704,57 @@ new_file:
   in: body
   required: true
   type: string
+nic_details_diagnostics:
+  description: |
+    The list of dictionaries with detailed information about VM NICs.
+    Following fields are presented in each dictionary:
+
+    - ``mac_address`` - Mac address of the interface (String)
+
+    - ``rx_octets`` - Received octets (Integer)
+
+    - ``rx_errors`` - Received errors (Integer)
+
+    - ``rx_drop`` - Received packets dropped (Integer)
+
+    - ``rx_packets`` - Received packets (Integer)
+
+    - ``rx_rate`` - Receive rate in bytes (Integer)
+
+    - ``tx_octets`` - Transmitted Octets (Integer)
+
+    - ``tx_errors`` - Transmit errors (Integer)
+
+    - ``tx_drop`` - Transmit dropped packets (Integer)
+
+    - ``tx_packets`` - Transmit packets (Integer)
+
+    - ``tx_rate`` - Transmit rate in bytes (Integer)
+  in: body
+  required: true
+  type: array
+  min_version: 2.48
+num_cpus_diagnostics:
+  description: |
+    The number of vCPUs.
+  in: body
+  required: true
+  type: integer
+  min_version: 2.48
+num_disks_diagnostics:
+  description: |
+    The number of disks.
+  in: body
+  required: true
+  type: integer
+  min_version: 2.48
+num_nics_diagnostics:
+  description: |
+    The number of vNICs.
+  in: body
+  required: true
+  type: integer
+  min_version: 2.48
 on_shared_storage:
   description: |
     Server on shared storage.
@@ -5009,6 +5140,13 @@ uptime:
   in: body
   required: true
   type: string
+uptime_diagnostics:
+  description: |
+    The amount of time in seconds that the VM has been running.
+  in: body
+  required: true
+  type: integer
+  min_version: 2.48
 uptime_simple_tenant_usage:
   description: |
     The uptime of the server.
@@ -5148,6 +5286,20 @@ virtual_interfaces:
   in: body
   required: true
   type: array
+vm_state_diagnostics:
+  description: |
+    A string enum denoting the current state of the VM. Possible values are:
+
+    - ``pending``
+    - ``running``
+    - ``paused``
+    - ``shutdown``
+    - ``crashed``
+    - ``suspended``
+  in: body
+  required: true
+  type: string
+  min_version: 2.48
 vm_state_optional:
   description: |
     The VM state.

doc/api_samples/os-server-diagnostics/v2.48/server-diagnostics-get-resp.json

@@ -0,0 +1,46 @@
+{
+    "config_drive": true,
+    "cpu_details": [
+        {
+            "id": 0,
+            "time": 17300000000,
+            "utilisation": 15
+        }
+    ],
+    "disk_details": [
+        {
+            "errors_count": 1,
+            "read_bytes": 262144,
+            "read_requests": 112,
+            "write_bytes": 5778432,
+            "write_requests": 488
+        }
+    ],
+    "driver": "libvirt",
+    "hypervisor": "kvm",
+    "hypervisor_os": "ubuntu",
+    "memory_details": {
+        "maximum": 524288,
+        "used": 0
+    },
+    "nic_details": [
+        {
+            "mac_address": "01:23:45:67:89:ab",
+            "rx_drop": 200,
+            "rx_errors": 100,
+            "rx_octets": 2070139,
+            "rx_packets": 26701,
+            "rx_rate": 300,
+            "tx_drop": 500,
+            "tx_errors": 400,
+            "tx_octets": 140208,
+            "tx_packets": 662,
+            "tx_rate": 600
+        }
+    ],
+    "num_cpus": 1,
+    "num_disks": 1,
+    "num_nics": 1,
+    "state": "running",
+    "uptime": 46664
+}

doc/api_samples/versions/v21-version-get-resp.json

@@ -19,7 +19,7 @@
             }
         ],
         "status": "CURRENT",
-        "version": "2.47",
+        "version": "2.48",
         "min_version": "2.1",
         "updated": "2013-07-23T11:33:21Z"
     }

doc/api_samples/versions/versions-get-resp.json

@@ -22,7 +22,7 @@
                 }
             ],
             "status": "CURRENT",
-            "version": "2.47",
+            "version": "2.48",
             "min_version": "2.1",
             "updated": "2013-07-23T11:33:21Z"
         }

nova/api/openstack/api_version_request.py

@@ -114,6 +114,7 @@ REST_API_VERSION_HISTORY = """REST API Version History:
              rather than a link.  If the user is prevented from retrieving
              the flavor extra-specs by policy, simply omit the field from
              the output.
+    * 2.48 - Standardize VM diagnostics info.
 """
 
 # The minimum and maximum versions of the API supported
@@ -122,7 +123,7 @@ REST_API_VERSION_HISTORY = """REST API Version History:
 # Note(cyeoh): This only applies for the v2.1 API once microversions
 # support is fully merged. It does not affect the V2 API.
 _MIN_API_VERSION = "2.1"
-_MAX_API_VERSION = "2.47"
+_MAX_API_VERSION = "2.48"
 DEFAULT_API_VERSION = _MIN_API_VERSION
 
 # Almost all proxy APIs which related to network, images and baremetal

nova/api/openstack/compute/rest_api_version_history.rst

@@ -556,3 +556,11 @@ user documentation.
   indexing extra-specs, then the ``extra_specs`` field will not be included in the
   flavor information.
 
+2.48
+----
+
+  Before version 2.48, VM diagnostics response was just a 'blob' of data
+  returned by each hypervisor. From this version VM diagnostics response is
+  standardized. It has a set of fields which each hypervisor will try to fill.
+  If a hypervisor driver unable to provide a specific field then this field
+  will be reported as 'None'.

nova/api/openstack/compute/server_diagnostics.py

@@ -15,19 +15,25 @@
 
 import webob
 
+from nova.api.openstack import api_version_request
 from nova.api.openstack import common
+from nova.api.openstack.compute.views import server_diagnostics
 from nova.api.openstack import extensions
 from nova.api.openstack import wsgi
 from nova import compute
 from nova import exception
+from nova.i18n import _
 from nova.policies import server_diagnostics as sd_policies
 
 
 class ServerDiagnosticsController(wsgi.Controller):
-    def __init__(self):
+    _view_builder_class = server_diagnostics.ViewBuilder
+
+    def __init__(self, *args, **kwargs):
+        super(ServerDiagnosticsController, self).__init__(*args, **kwargs)
         self.compute_api = compute.API()
 
-    @extensions.expected_errors((404, 409, 501))
+    @extensions.expected_errors((400, 404, 409, 501))
     def index(self, req, server_id):
         context = req.environ["nova.context"]
         context.can(sd_policies.BASE_POLICY_NAME)
@@ -35,16 +41,27 @@ class ServerDiagnosticsController(wsgi.Controller):
         instance = common.get_instance(self.compute_api, context, server_id)
 
         try:
-            # NOTE(gmann): To make V21 same as V2 API, this method will call
+            if api_version_request.is_supported(req, min_version='2.48'):
-            # 'get_diagnostics' instead of 'get_instance_diagnostics'.
+                diagnostics = self.compute_api.get_instance_diagnostics(
-            # In future, 'get_instance_diagnostics' needs to be called to
+                    context, instance)
-            # provide VM diagnostics in a defined format for all driver.
+                return self._view_builder.instance_diagnostics(diagnostics)
-            # BP - https://blueprints.launchpad.net/nova/+spec/v3-diagnostics.
+
             return self.compute_api.get_diagnostics(context, instance)
         except exception.InstanceInvalidState as state_error:
             common.raise_http_conflict_for_instance_invalid_state(state_error,
                     'get_diagnostics', server_id)
         except exception.InstanceNotReady as e:
             raise webob.exc.HTTPConflict(explanation=e.format_message())
+        except exception.InstanceDiagnosticsNotSupported:
+            # NOTE(snikitin): During upgrade we may face situation when env
+            # has new API and old compute. New compute returns a
+            # Diagnostics object. Old compute returns a dictionary. So we
+            # can't perform a request correctly if compute is too old.
+            msg = _('Compute node is too old. You must complete the '
+                    'upgrade process to be able to get a standardized '
+                    'diagnostics data which is available since v2.48. However '
+                    'you still able to get a diagnostics data in old format '
+                    'which is available till v2.47.')
+            raise webob.exc.HTTPBadRequest(explanation=msg)
         except NotImplementedError:
             common.raise_feature_not_supported()

nova/api/openstack/compute/views/server_diagnostics.py

@@ -0,0 +1,62 @@
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from nova.api.openstack import common
+
+INSTANCE_DIAGNOSTICS_PRIMITIVE_FIELDS = (
+    'state', 'driver', 'hypervisor', 'hypervisor_os', 'uptime', 'config_drive',
+    'num_cpus', 'num_nics', 'num_disks'
+)
+
+INSTANCE_DIAGNOSTICS_LIST_FIELDS = {
+    'disk_details': ('read_bytes', 'read_requests', 'write_bytes',
+                     'write_requests', 'errors_count'),
+    'cpu_details': ('id', 'time', 'utilisation'),
+    'nic_details': ('mac_address', 'rx_octets', 'rx_errors', 'rx_drop',
+                    'rx_packets', 'rx_rate', 'tx_octets', 'tx_errors',
+                    'tx_drop', 'tx_packets', 'tx_rate')
+}
+
+INSTANCE_DIAGNOSTICS_OBJECT_FIELDS = {'memory_details': ('maximum', 'used')}
+
+
+class ViewBuilder(common.ViewBuilder):
+    @staticmethod
+    def _get_obj_field(obj, field):
+        if obj and obj.obj_attr_is_set(field):
+            return getattr(obj, field)
+        return None
+
+    def instance_diagnostics(self, diagnostics):
+        """Return a dictionary with instance diagnostics."""
+        diagnostics_dict = {}
+        for field in INSTANCE_DIAGNOSTICS_PRIMITIVE_FIELDS:
+            diagnostics_dict[field] = self._get_obj_field(diagnostics, field)
+
+        for list_field in INSTANCE_DIAGNOSTICS_LIST_FIELDS:
+            diagnostics_dict[list_field] = []
+            list_obj = getattr(diagnostics, list_field)
+
+            for obj in list_obj:
+                obj_dict = {}
+                for field in INSTANCE_DIAGNOSTICS_LIST_FIELDS[list_field]:
+                    obj_dict[field] = self._get_obj_field(obj, field)
+                diagnostics_dict[list_field].append(obj_dict)
+
+        for obj_field in INSTANCE_DIAGNOSTICS_OBJECT_FIELDS:
+            diagnostics_dict[obj_field] = {}
+            obj = self._get_obj_field(diagnostics, obj_field)
+            for field in INSTANCE_DIAGNOSTICS_OBJECT_FIELDS[obj_field]:
+                diagnostics_dict[obj_field][field] = self._get_obj_field(
+                    obj, field)
+
+        return diagnostics_dict

nova/tests/functional/api_sample_tests/api_samples/os-server-diagnostics/v2.48/server-diagnostics-get-resp.json.tpl

@@ -0,0 +1,46 @@
+{
+    "config_drive": true,
+    "cpu_details": [
+        {
+            "id": 0,
+            "time": 17300000000,
+            "utilisation": 15
+        }
+    ],
+    "disk_details": [
+        {
+            "errors_count": 1,
+            "read_bytes": 262144,
+            "read_requests": 112,
+            "write_bytes": 5778432,
+            "write_requests": 488
+        }
+    ],
+    "driver": "libvirt",
+    "hypervisor": "kvm",
+    "hypervisor_os": "ubuntu",
+    "memory_details": {
+        "maximum": 524288,
+        "used": 0
+    },
+    "nic_details": [
+        {
+            "mac_address": "01:23:45:67:89:ab",
+            "rx_drop": 200,
+            "rx_errors": 100,
+            "rx_octets": 2070139,
+            "rx_packets": 26701,
+            "rx_rate": 300,
+            "tx_drop": 500,
+            "tx_errors": 400,
+            "tx_octets": 140208,
+            "tx_packets": 662,
+            "tx_rate": 600
+        }
+    ],
+    "num_cpus": 1,
+    "num_disks": 1,
+    "num_nics": 1,
+    "state": "running",
+    "uptime": 46664
+}

nova/tests/functional/api_sample_tests/test_server_diagnostics.py

@@ -24,3 +24,8 @@ class ServerDiagnosticsSamplesJsonTest(test_servers.ServersSampleBase):
         response = self._do_get('servers/%s/diagnostics' % uuid)
         self._verify_response('server-diagnostics-get-resp', {},
                               response, 200)
+
+
+class ServerDiagnosticsSamplesJsonTestV248(ServerDiagnosticsSamplesJsonTest):
+    microversion = '2.48'
+    scenarios = [('v2_48', {'api_major_version': 'v2.1'})]

nova/tests/unit/api/openstack/compute/test_server_diagnostics.py

@@ -15,53 +15,58 @@
 
 import mock
 from oslo_serialization import jsonutils
+from webob import exc
 
-from nova.api.openstack import compute
 from nova.api.openstack.compute import server_diagnostics
+from nova.api.openstack import wsgi as os_wsgi
 from nova.compute import api as compute_api
 from nova import exception
+from nova import objects
 from nova import test
 from nova.tests.unit.api.openstack import fakes
 
+import oslo_messaging
 
 UUID = 'abc'
 
 
-def fake_get_diagnostics(self, _context, instance_uuid):
-    return {'data': 'Some diagnostic info'}
-
-
 def fake_instance_get(self, _context, instance_uuid, expected_attrs=None):
     if instance_uuid != UUID:
         raise Exception("Invalid UUID")
-    return {'uuid': instance_uuid}
+    return objects.Instance(uuid=instance_uuid, host='123')
 
 
 class ServerDiagnosticsTestV21(test.NoDBTestCase):
+    mock_diagnostics_method = 'get_diagnostics'
+    api_version = '2.1'
 
     def _setup_router(self):
-        self.router = compute.APIRouterV21()
+        self.router = fakes.wsgi_app_v21()
 
     def _get_request(self):
         return fakes.HTTPRequest.blank(
-                   '/fake/servers/%s/diagnostics' % UUID)
+            '/v2/fake/servers/%s/diagnostics' % UUID,
+            version=self.api_version,
+            headers = {os_wsgi.API_VERSION_REQUEST_HEADER:
+                           'compute %s' % self.api_version})
 
     def setUp(self):
         super(ServerDiagnosticsTestV21, self).setUp()
         self._setup_router()
 
-    @mock.patch.object(compute_api.API, 'get_diagnostics',
+    @mock.patch.object(compute_api.API, 'get', fake_instance_get)
-                       fake_get_diagnostics)
+    def _test_get_diagnostics(self, expected, return_value):
-    @mock.patch.object(compute_api.API, 'get',
-                       fake_instance_get)
-    def test_get_diagnostics(self):
         req = self._get_request()
-        res = req.get_response(self.router)
+        with mock.patch.object(compute_api.API, self.mock_diagnostics_method,
+                               return_value=return_value):
+            res = req.get_response(self.router)
         output = jsonutils.loads(res.body)
-        self.assertEqual(output, {'data': 'Some diagnostic info'})
+        self.assertEqual(expected, output)
+
+    def test_get_diagnostics(self):
+        diagnostics = {'data': 'Some diagnostics info'}
+        self._test_get_diagnostics(diagnostics, diagnostics)
 
-    @mock.patch.object(compute_api.API, 'get_diagnostics',
-                fake_get_diagnostics)
     @mock.patch.object(compute_api.API, 'get',
                 side_effect=exception.InstanceNotFound(instance_id=UUID))
     def test_get_diagnostics_with_non_existed_instance(self, mock_get):
@@ -69,40 +74,123 @@ class ServerDiagnosticsTestV21(test.NoDBTestCase):
         res = req.get_response(self.router)
         self.assertEqual(res.status_int, 404)
 
-    @mock.patch.object(compute_api.API, 'get_diagnostics',
-                side_effect=exception.InstanceInvalidState('fake message'))
     @mock.patch.object(compute_api.API, 'get', fake_instance_get)
-    def test_get_diagnostics_raise_conflict_on_invalid_state(self,
+    def test_get_diagnostics_raise_conflict_on_invalid_state(self):
-                                                  mock_get_diagnostics):
         req = self._get_request()
-        res = req.get_response(self.router)
+        with mock.patch.object(compute_api.API, self.mock_diagnostics_method,
+                side_effect=exception.InstanceInvalidState('fake message')):
+            res = req.get_response(self.router)
         self.assertEqual(409, res.status_int)
 
-    @mock.patch.object(compute_api.API, 'get_diagnostics',
-                       side_effect=exception.InstanceNotReady('fake message'))
     @mock.patch.object(compute_api.API, 'get', fake_instance_get)
-    def test_get_diagnostics_raise_instance_not_ready(self,
+    def test_get_diagnostics_raise_instance_not_ready(self):
-                                                      mock_get_diagnostics):
         req = self._get_request()
-        res = req.get_response(self.router)
+        with mock.patch.object(compute_api.API, self.mock_diagnostics_method,
+                side_effect=exception.InstanceNotReady('fake message')):
+            res = req.get_response(self.router)
         self.assertEqual(409, res.status_int)
 
-    @mock.patch.object(compute_api.API, 'get_diagnostics',
-                side_effect=NotImplementedError)
     @mock.patch.object(compute_api.API, 'get', fake_instance_get)
-    def test_get_diagnostics_raise_no_notimplementederror(self,
+    def test_get_diagnostics_raise_no_notimplementederror(self):
-                      mock_get_diagnostics):
         req = self._get_request()
-        res = req.get_response(self.router)
+        with mock.patch.object(compute_api.API, self.mock_diagnostics_method,
+                               side_effect=NotImplementedError):
+            res = req.get_response(self.router)
         self.assertEqual(501, res.status_int)
 
 
+class ServerDiagnosticsTestV248(ServerDiagnosticsTestV21):
+    mock_diagnostics_method = 'get_instance_diagnostics'
+    api_version = '2.48'
+
+    def test_get_diagnostics(self):
+        return_value = objects.Diagnostics(
+            config_drive=False,
+            state='running',
+            driver='libvirt',
+            uptime=5,
+            hypervisor='hypervisor',
+            # hypervisor_os is unset
+            cpu_details=[
+                objects.CpuDiagnostics(id=0, time=1111, utilisation=11),
+                objects.CpuDiagnostics(id=1, time=None, utilisation=22),
+                objects.CpuDiagnostics(id=2, time=3333, utilisation=None),
+                objects.CpuDiagnostics(id=None, time=4444, utilisation=44)],
+            nic_details=[objects.NicDiagnostics(
+                mac_address='de:ad:be:ef:00:01',
+                rx_drop=1,
+                rx_errors=2,
+                rx_octets=3,
+                rx_packets=4,
+                rx_rate=5,
+                tx_drop=6,
+                tx_errors=7,
+                tx_octets=8,
+                # tx_packets is unset
+                tx_rate=None)],
+            disk_details=[objects.DiskDiagnostics(
+                errors_count=1,
+                read_bytes=2,
+                read_requests=3,
+                # write_bytes is unset
+                write_requests=None)],
+            num_cpus=4,
+            num_disks=1,
+            num_nics=1,
+            memory_details=objects.MemoryDiagnostics(maximum=8192, used=3072))
+
+        expected = {
+            'config_drive': False,
+            'state': 'running',
+            'driver': 'libvirt',
+            'uptime': 5,
+            'hypervisor': 'hypervisor',
+            'hypervisor_os': None,
+            'cpu_details': [{'id': 0, 'time': 1111, 'utilisation': 11},
+                            {'id': 1, 'time': None, 'utilisation': 22},
+                            {'id': 2, 'time': 3333, 'utilisation': None},
+                            {'id': None, 'time': 4444, 'utilisation': 44}],
+            'nic_details': [{'mac_address': 'de:ad:be:ef:00:01',
+                             'rx_drop': 1,
+                             'rx_errors': 2,
+                             'rx_octets': 3,
+                             'rx_packets': 4,
+                             'rx_rate': 5,
+                             'tx_drop': 6,
+                             'tx_errors': 7,
+                             'tx_octets': 8,
+                             'tx_packets': None,
+                             'tx_rate': None}],
+            'disk_details': [{'errors_count': 1,
+                              'read_bytes': 2,
+                              'read_requests': 3,
+                              'write_bytes': None,
+                              'write_requests': None}],
+            'num_cpus': 4,
+            'num_disks': 1,
+            'num_nics': 1,
+            'memory_details': {'maximum': 8192, 'used': 3072}}
+
+        self._test_get_diagnostics(expected, return_value)
+
+    @mock.patch.object(oslo_messaging.RPCClient, 'can_send_version',
+                       return_value=False)
+    @mock.patch.object(compute_api.API, 'get', fake_instance_get)
+    def test_get_diagnostics_old_compute(self, mock_version):
+        """Checks case when env has new api and old compute."""
+
+        controller = server_diagnostics.ServerDiagnosticsController()
+        req = self._get_request()
+        self.assertRaises(exc.HTTPBadRequest, controller.index, req, UUID)
+
+
 class ServerDiagnosticsEnforcementV21(test.NoDBTestCase):
+    api_version = '2.1'
 
     def setUp(self):
         super(ServerDiagnosticsEnforcementV21, self).setUp()
         self.controller = server_diagnostics.ServerDiagnosticsController()
-        self.req = fakes.HTTPRequest.blank('')
+        self.req = fakes.HTTPRequest.blank('', version=self.api_version)
 
     def test_get_diagnostics_policy_failed(self):
         rule_name = "os_compute_api:os-server-diagnostics"
@@ -113,3 +201,7 @@ class ServerDiagnosticsEnforcementV21(test.NoDBTestCase):
         self.assertEqual(
             "Policy doesn't allow %s to be performed." % rule_name,
             exc.format_message())
+
+
+class ServerDiagnosticsEnforcementV248(ServerDiagnosticsEnforcementV21):
+    api_version = '2.48'

nova/tests/unit/compute/test_compute.py

@@ -4040,8 +4040,8 @@ class ComputeTestCase(BaseTestCase,
                            'write_bytes': 5778432,
                            'write_requests': 488}],
             driver='libvirt',
-            hypervisor='fake-hypervisor',
+            hypervisor='kvm',
-            hypervisor_os='fake-os',
+            hypervisor_os='ubuntu',
             memory_details={'maximum': 524288, 'used': 0},
             nic_details=[{'mac_address': '01:23:45:67:89:ab',
                           'rx_octets': 2070139,

nova/virt/fake.py

@@ -349,8 +349,8 @@ class FakeDriver(driver.ComputeDriver):
 
     def get_instance_diagnostics(self, instance):
         diags = diagnostics_obj.Diagnostics(
-            state='running', driver='libvirt', hypervisor='fake-hypervisor',
+            state='running', driver='libvirt', hypervisor='kvm',
-            hypervisor_os='fake-os', uptime=46664, config_drive=True)
+            hypervisor_os='ubuntu', uptime=46664, config_drive=True)
         diags.add_cpu(id=0, time=17300000000, utilisation=15)
         diags.add_nic(mac_address='01:23:45:67:89:ab',
                       rx_octets=2070139,

releasenotes/notes/bp-restore-vm-diagnostics-544b56bbb0167071.yaml

@@ -0,0 +1,6 @@
+---
+features:
+  - Added microversion v2.48 which standardize VM diagnostics response.
+    It has a set of fields which each hypervisor will try to fill.
+    If a hypervisor driver unable to provide a specific field then this field
+    will be reported as 'None'.