Assume number of services can be large (10000 as in the bug description),
this patch removes second service_get_all call.
zone_hosts changed from dict of lists to dict of sets.
The HostAPI instance from the API controller is also passed to the
get_availability_zones method so it does not have to recreate it
per call (this is both for a slight performance gain but mostly also
for test sanity).
On devstack with 10000 services patch decreased response time twice.
openstack availability zone list --long --timing
...
Before:
+-------------------------------------------------------------------------------------------+-----------+
| URL | Seconds |
+-------------------------------------------------------------------------------------------+-----------+
| GET http://192.168.0.45/identity | 0.006816 |
| POST http://192.168.0.45/identity/v3/auth/tokens | 0.456708 |
| POST http://192.168.0.45/identity/v3/auth/tokens | 0.087485 |
| GET http://172.18.237.203/compute/v2.1/os-availability-zone/detail | 95.667192 |
| GET http://172.18.237.203/volume/v2/e2671d37ee2c4374bd1533645261f1d4/os-availability-zone | 0.036528 |
| Total | 96.254729 |
+-------------------------------------------------------------------------------------------+-----------+
After:
+-------------------------------------------------------------------------------------------+-----------+
| URL | Seconds |
+-------------------------------------------------------------------------------------------+-----------+
| GET http://192.168.0.45/identity | 0.020215 |
| POST http://192.168.0.45/identity/v3/auth/tokens | 0.102987 |
| POST http://192.168.0.45/identity/v3/auth/tokens | 0.111899 |
| GET http://172.18.237.203/compute/v2.1/os-availability-zone/detail | 39.346657 |
| GET http://172.18.237.203/volume/v2/e2671d37ee2c4374bd1533645261f1d4/os-availability-zone | 0.026403 |
| Total | 39.608161 |
+-------------------------------------------------------------------------------------------+-----------+
The test_availability_zone_detail unit test is updated to assert that
services are only retrieved twice (once for enabled, once for disabled).
While in there, the expected response dict is formatted for readability
and a duplicate zone/host is added to make sure duplicates are handled
for available services. To ensure service_get_all is called only twice,
the low-level DB API service_get_all stub is replaced with a mock and
the mock is changed to be on the HostAPI.service_get_all method which
is (1) what the API controller code is actually using and (2) allows the
test to only mock the instance of the HostAPI being tested - trying to
mock the DB API service_get_all method causes intermittent failures
in unrelated tests because of the global nature of that mock.
There is another opportunity for optimizing get_availability_zones which
is marked with a TODO but left for a separate patch.
Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>
Partial-Bug: #1801897
Change-Id: Ib9a9a9a79499272d740a64cc0b909f0299a237d1
(cherry picked from commit 74cefe4266)
(cherry picked from commit c280d747fb)
fa275544c2