Commit Graph

20 Commits

Author SHA1 Message Date
Sabari Kumar Murugesan
aa10f66ee0 VMware: Support Multiple Datastores
Support for VMware store to use multiple datastore backends.

Spec (approved):
https://review.openstack.org/#/c/146723/

tl;dr:
1. Adds a new config option vmware_datastores to configure
   multiple datastores.
2. Implements a selection logic based on priority to choose
   from the list of datastores.
3. Modifies StoreLocation parsing logic to identify datastore
   related info from location URI.

DocImpact

Implements-Blueprint: vmware-store-multiple-datastores

Change-Id: I176f1143cd2d9b0a01a0f4f4256e7ac7d9b09afd
2015-03-06 20:11:56 +00:00
Sabari Kumar Murugesan
e278d14838 Fix sorting query string keys for arbitrary url schemes
In Python2.6, urlparse module does not parse the query string from
from the url for arbitrary schemes. http://bugs.python.org/issue9374

Thus, our test utility method does not sort the qs keys for vmware
store url scheme (which is vsphere). This causes random failures in
py26 unit tests.

Change-Id: I1f5da01bfb346b547c1f70b7de9b18585c197f29
2015-02-25 19:47:02 -08:00
Jenkins
45b55585c7 Merge "Check VMware session before uploading image" 2015-02-19 22:37:17 +00:00
Sabari Kumar Murugesan
e1ee81a37f Check VMware session before uploading image
Uploading an image to vSphere backend without checking the session
results in broken pipe socket error. When this happens, glance-api
sends a 400 response because the IOError is not handled by the
store.

We address this issue by :-
1. Checking if session is authenticated before uploading the image.
2. Handle IOError and check the response code.

Closes-Bug: #1402354

Change-Id: I66b6dfddfb2ddd089488f3f79f3917fd69739fc9
2015-02-09 12:15:52 -08:00
Zhi Yan Liu
138875b7c3 Add capabilities to storage driver
Storage capabilities is used to indicate the static and dynamic ability
of the storage driver object based on current driver implementation or
particular driver configuration and backend status.

Use storage capabilities glance_store can do more proper operations on
backend to support upper layer request, like to enable or disable add()
function to glance, or if allow glance reuse driver instance for all
request according to whether the driver and/or backend is stateless.

This patch implemented some initial capabilities for existing drivers,
and change the foundational code to make them be aware. Mainly it
contains:

1. Implemented essential code to enable driver capabilities, adding
necessary capabilities.

2. Added a generic checker on necessary storage operations, to make sure
the capabilities of the driver are capable of handling requested
operation. We can enhance the check logic as needed easily in future.

3. Added a callback based schedule logic to update dynamic capabilities
of store when operator enabled it by a option.

4. Refactoring on existing disablement logic on driver add() interface,
to use consistent capabilities way to handle it, removed add_disabled().

5. Therefor the related exception conversion logic for other interfaces
are redundant, due to now we can raise proper exception directly from
the checker.

6. Added the logic to recreate drive object if the storage and/or driver
isn't stateless.

Few minor changes need to be added to Glance side:
Change Ibbc85b6bc2ea98c564d316db2874d7df5aac32a6 .

docImpact
Implements: blueprint store-capabilities

Change-Id: Iedf0d4f829e46ca64c3f4fc6a7dfee54d9b0605b
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
2015-02-08 12:41:47 +08:00
Jenkins
94ad18b31f Merge "Remove retry on failed uploads to VMware datastore" 2015-02-03 15:14:43 +00:00
Sabari Kumar Murugesan
632ce4000a Remove retry on failed uploads to VMware datastore
VMware store retries to upload the image data but fails when
seeking to reposition the file. Since the image_file is not
seekable, we can never retry from the beginning. Also, the
backend datastore server does not support resumable upload.

Thus,
1. Removing the retry logic in store.add().
2. Replace non-existing UnexpectedStatus with BackendException.

Closes-Bug: #1413852

Change-Id: I453a14cb832c07c8cd932a3db7b4a34a80a73116
2015-01-31 23:49:19 +00:00
Louis Taylor
94554c606f Move from oslo.utils to oslo_utils
oslo_utils has been moved out of the oslo namespace.

bp drop-namespace-packages

Change-Id: Id4ccf076983592026d35407107e79cf1575248ce
2015-01-09 20:16:20 +00:00
Arnaud Legendre
ec6db9fd47 VMware store: Re-use api session token
The store object is currently instanciated for each API call. The
constructor of the store object triggers the configuration of the
store. The session token to communicate with vCenter is generated
in this configuration step which can take some time depending on
the network.

This patch prevents the creation of the session throughout the API
calls.

Change-Id: Ic9520fa856264fda1ef3cb05f1395a8002192499
Closes-Bug: #1350010
2014-11-18 14:51:09 -08:00
Zhi Yan Liu
0584204175 Switch to using oslo.utils
This change switched glance_store to using oslo.utils. It is not good to
add dependency on oslo-incubator if we can avoid it.

Change-Id: I9efe3d57e0ce64296c7a75838bd2ee19249f0fa8
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
2014-09-29 13:11:48 +08:00
Arnaud Legendre
4917fe64b3 VMware store: Use the Content-Length if available
Change I579084460e7f61ab4042632d17ec0f045fa6f5af changed the VMware
store to use chunked encoding to upload data to the underlying backend.
We should do that only if the image size is not provided or zero and not
all the time.
This patch addressed the issue by checking the image_size before upload.

Merged in Glance: Change-Id: If348be7fd24fd05146b0c86bef45b79f600cc0f7
Closes-Bug: #1336970

Change-Id: I58efc8981b38d1e445f9acde669802b97e4b2077
2014-09-25 14:42:21 -07:00
Zhi Yan Liu
22dbccb491 Enhance configuration handling
1. Changed drivers to prevent using global CONF but access it from
client by interface.
2. Corrected swift driver to involve options from swift driver utils,
allow glance_store exports and registers them for client.
3. Added missing test cases for the functions of swift driver util.

Change-Id: I8fac6800efde202e29179791ea05c4814ec58435
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
2014-09-24 11:56:11 +08:00
Oleksii Chuprykov
71ae81d69c Fix H402
Fix `one line docstring needs punctuation.`
Remove H402 from ignore list.

Change-Id: I0cd3b88467067cdb10d41f86cc94e075da141ac0
2014-09-20 21:13:41 +03:00
Jenkins
783a1c8bbd Merge "Handle session timeout in the VMware store" 2014-09-13 12:38:38 +00:00
Arnaud Legendre
a8b2f3a215 Handle session timeout in the VMware store
The current implementation can lead to session timeouts. This
will occur when accessing the datastore directly through HTTP.
This patch provides a retry mechanism by recreating the session
when getting a 401 status code.

This patch backports Change I54cc9e30c9bc374a2cf82dec4beb9b06594835f8

Change-Id: I245d10c66082215010109e661e799293caec6191
2014-09-09 18:46:45 -07:00
Zhi Yan Liu
d6a5dde0ea Add entry-point for oslo.config options and update registering logic
1. The change exposed all options of store and driver by oslo.config
entry-point.
2. Refactor store and driver options registering logic to leverage this
unified option discover function.

Change-Id: Ibbd0f27abd0c0eb9a88d8d30de1e89264a11f53d
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
2014-09-10 09:44:29 +08:00
Flavio Percoco
a1c59a1575 Rename glance.store to glance_store
Given the existing, known issues, with python namespaces, pip and
setuptools, we've decided to rename glance.store into glance_store.

Change-Id: I3f02ce78b3d64f34744e5116652bfbb4f3062cbf
2014-08-28 10:09:36 +02:00
Flavio Percoco
9d4eda50f2 Fix collection order issues and unit test failures
A randomized PYTHONHASHSEED caused the glance API to return certain data
in a nondeterministic order, breaking some tests. This patch sorts the
returned data before testing against a pre-sorted expected values.

Change-Id: Ie294b64738bd33b31aea915183c30b5b8db2eb65
2014-08-27 12:54:28 +02:00
Flavio Percoco
80af4f414a Fix flake8 errors 2014-07-09 15:58:38 +02:00
Flavio Percoco
03b43598d0 Migrate vmware store
The commit migrates vmware store from glance to glance/store
2014-06-25 19:19:48 +02:00