Support for VMware store to use multiple datastore backends.
Spec (approved):
https://review.openstack.org/#/c/146723/
tl;dr:
1. Adds a new config option vmware_datastores to configure
multiple datastores.
2. Implements a selection logic based on priority to choose
from the list of datastores.
3. Modifies StoreLocation parsing logic to identify datastore
related info from location URI.
DocImpact
Implements-Blueprint: vmware-store-multiple-datastores
Change-Id: I176f1143cd2d9b0a01a0f4f4256e7ac7d9b09afd
In Python2.6, urlparse module does not parse the query string from
from the url for arbitrary schemes. http://bugs.python.org/issue9374
Thus, our test utility method does not sort the qs keys for vmware
store url scheme (which is vsphere). This causes random failures in
py26 unit tests.
Change-Id: I1f5da01bfb346b547c1f70b7de9b18585c197f29
Uploading an image to vSphere backend without checking the session
results in broken pipe socket error. When this happens, glance-api
sends a 400 response because the IOError is not handled by the
store.
We address this issue by :-
1. Checking if session is authenticated before uploading the image.
2. Handle IOError and check the response code.
Closes-Bug: #1402354
Change-Id: I66b6dfddfb2ddd089488f3f79f3917fd69739fc9
Storage capabilities is used to indicate the static and dynamic ability
of the storage driver object based on current driver implementation or
particular driver configuration and backend status.
Use storage capabilities glance_store can do more proper operations on
backend to support upper layer request, like to enable or disable add()
function to glance, or if allow glance reuse driver instance for all
request according to whether the driver and/or backend is stateless.
This patch implemented some initial capabilities for existing drivers,
and change the foundational code to make them be aware. Mainly it
contains:
1. Implemented essential code to enable driver capabilities, adding
necessary capabilities.
2. Added a generic checker on necessary storage operations, to make sure
the capabilities of the driver are capable of handling requested
operation. We can enhance the check logic as needed easily in future.
3. Added a callback based schedule logic to update dynamic capabilities
of store when operator enabled it by a option.
4. Refactoring on existing disablement logic on driver add() interface,
to use consistent capabilities way to handle it, removed add_disabled().
5. Therefor the related exception conversion logic for other interfaces
are redundant, due to now we can raise proper exception directly from
the checker.
6. Added the logic to recreate drive object if the storage and/or driver
isn't stateless.
Few minor changes need to be added to Glance side:
Change Ibbc85b6bc2ea98c564d316db2874d7df5aac32a6 .
docImpact
Implements: blueprint store-capabilities
Change-Id: Iedf0d4f829e46ca64c3f4fc6a7dfee54d9b0605b
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
VMware store retries to upload the image data but fails when
seeking to reposition the file. Since the image_file is not
seekable, we can never retry from the beginning. Also, the
backend datastore server does not support resumable upload.
Thus,
1. Removing the retry logic in store.add().
2. Replace non-existing UnexpectedStatus with BackendException.
Closes-Bug: #1413852
Change-Id: I453a14cb832c07c8cd932a3db7b4a34a80a73116
The store object is currently instanciated for each API call. The
constructor of the store object triggers the configuration of the
store. The session token to communicate with vCenter is generated
in this configuration step which can take some time depending on
the network.
This patch prevents the creation of the session throughout the API
calls.
Change-Id: Ic9520fa856264fda1ef3cb05f1395a8002192499
Closes-Bug: #1350010
This change switched glance_store to using oslo.utils. It is not good to
add dependency on oslo-incubator if we can avoid it.
Change-Id: I9efe3d57e0ce64296c7a75838bd2ee19249f0fa8
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Change I579084460e7f61ab4042632d17ec0f045fa6f5af changed the VMware
store to use chunked encoding to upload data to the underlying backend.
We should do that only if the image size is not provided or zero and not
all the time.
This patch addressed the issue by checking the image_size before upload.
Merged in Glance: Change-Id: If348be7fd24fd05146b0c86bef45b79f600cc0f7
Closes-Bug: #1336970
Change-Id: I58efc8981b38d1e445f9acde669802b97e4b2077
1. Changed drivers to prevent using global CONF but access it from
client by interface.
2. Corrected swift driver to involve options from swift driver utils,
allow glance_store exports and registers them for client.
3. Added missing test cases for the functions of swift driver util.
Change-Id: I8fac6800efde202e29179791ea05c4814ec58435
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
The current implementation can lead to session timeouts. This
will occur when accessing the datastore directly through HTTP.
This patch provides a retry mechanism by recreating the session
when getting a 401 status code.
This patch backports Change I54cc9e30c9bc374a2cf82dec4beb9b06594835f8
Change-Id: I245d10c66082215010109e661e799293caec6191
1. The change exposed all options of store and driver by oslo.config
entry-point.
2. Refactor store and driver options registering logic to leverage this
unified option discover function.
Change-Id: Ibbd0f27abd0c0eb9a88d8d30de1e89264a11f53d
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Given the existing, known issues, with python namespaces, pip and
setuptools, we've decided to rename glance.store into glance_store.
Change-Id: I3f02ce78b3d64f34744e5116652bfbb4f3062cbf
A randomized PYTHONHASHSEED caused the glance API to return certain data
in a nondeterministic order, breaking some tests. This patch sorts the
returned data before testing against a pre-sorted expected values.
Change-Id: Ie294b64738bd33b31aea915183c30b5b8db2eb65