Currently when there is no limit set on a volume list query we retrieve
all volumes and then limit them locally using osapi_max_limit. Similar
thing happens when we are using the marker for next pages, we get all
volumes from that marker until the last volume and then limit it
locally.
We should be limiting it on the DB side so we only retrieve the data we
are actually going to return to the API caller.
This patch always limits the data retrieved from the DB and for the
offset to keep working as it was before we need to do the offset on the
DB side as well.
For reference some tests were performed:
On a deployment with 60,000 volumes, 370,000 volume_metadata items and
240,000 volume_glance_metadata items in cinder db. Before the patch
this will use nearly 10G memory. With the patch we will just use about
500M.
Co-Authored-By: wangxiyuan <wangxiyuan@huawei.com>
Closes-bug:#1483165i
Change-Id: Ie903e546074fe118299e8e1acfb9c88c8a10d78c
28879f9a78