openstack/openstacksdk
Code Issues Proposed changes

Implement Swift Proxy object and example

Browse Source 
This implements the proxy that is exposed as Connection.object_store,
and includes some example usage as well as documentation and a user
guide.

python -m examples.object_store --list-containers
python -m examples.object_store --list-objects <container>
python -m examples.object_store --upload-directory <directory> --pattern
<glob pattern>

Ex: ... --upload-directory pictures/ --pattern "*.jpg"

Change-Id: I8739ebca2ac77ea4a4d6f4e3ff30a3a253d8b636
This commit is contained in:
Brian Curtin 2014-10-30 14:41:57 -05:00
parent a250d8893f
commit ad9a4c1878
9 changed files with 993 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
doc/source/code/connection.py Normal file
View File

@ -0,0 +1,6 @@
from openstack import connection
conn = connection.Connection(auth_url="http://openstack:5000/v3",
project_name="big_project",
user_name="SDK_user",
password="Super5ecretPassw0rd")

15
doc/source/highlevel/object_store.rst Normal file
View File

@ -0,0 +1,15 @@
Object Store API
================
For details on how to use this API, see :doc:`/userguides/object_store`
.. automodule:: openstack.object_store.v1._proxy
The Object Store Class
----------------------
The Object Store high-level interface is exposed as the ``object_store``
object on :class:`~openstack.connection.Connection` objects.
.. autoclass:: openstack.object_store.v1._proxy.Proxy
:members:

16
doc/source/index.rst
View File

@ -11,6 +11,22 @@ Welcome!
contributing
glossary
User Guides
-----------
.. toctree::
:maxdepth: 1
userguides/object_store
High-Level Interface
--------------------
.. toctree::
:maxdepth: 1
highlevel/object_store
Resource Level Classes
----------------------

224
doc/source/userguides/object_store.rst Normal file
View File

@ -0,0 +1,224 @@
Using the OpenStack Object Store API
====================================
The Object Store API operates on two things: containers and objects.
Before working with the ``object_store`` API, you'll need to obtain a
:class:`~openstack.connection.Connection` object like so.
.. literalinclude:: /code/connection.py
Working with Containers
-----------------------
Listing Containers
******************
To list existing containers, use the
:meth:`~openstack.object_store.v1._proxy.Proxy.containers` method. ::
>>> for cont in conn.object_store.containers():
... print cont
...
Container: {u'count': 5, u'bytes': 500, u'name': u'my container'}
Container: {u'count': 0, u'bytes': 0, u'name': u'empty container'}
Container: {u'count': 100, u'bytes': 1000000, u'name': u'another container'}
The ``containers`` method returns a generator which yields
:class:`~openstack.object_store.v1.container.Container` objects. It handles
pagination for you, which can be adjusted via the ``limit`` argument.
By default, the ``containers`` method will yield as many containers as the
service will return, and it will continue requesting until it receives
no more. ::
>>> for cont in conn.object_store.containers(limit=500):
... print(cont)
...
<500 Containers>
... another request transparently made to the Object Store service
<500 more Containers>
...
Creating Containers
*******************
To create a container, use the
:meth:`~openstack.object_store.v1._proxy.Proxy.create_container` method. ::
>>> cont = conn.object_store.create_container("new container".decode("utf8"))
>>> cont
Container: {'name': u'new container'}
You can also create containers by passing in a
:class:`~openstack.object_store.v1.container.Container` resource. This is
helpful if you wanted to create another container which uses the same metadata
settings that another container has. ::
>>> from copy import copy
>>> print cont.name, cont.read_ACL
MyContainer .r:mysite.com
>>> new_cont = copy(cont)
>>> new_cont.name = "copied container"
>>> conn.object_store.create_container(new_cont)
Container: {u'name': 'copied container', 'x-container-read': '.r:mysite.com'}
Working with Container Metadata
*******************************
To get the metadata for a container, use the
:meth:`~openstack.object_store.v1._proxy.Proxy.get_container_metadata` method.
This method either takes the name of a container, or a
:class:`~openstack.object_store.v1.container.Container` object, and it returns
a `Container` object with all of its metadata attributes set. ::
>>> cont = conn.object_store.get_container_metadata("new container".decode("utf8"))
Container: {'content-length': '0', 'x-container-object-count': '0',
'name': u'new container', 'accept-ranges': 'bytes',
'x-trans-id': 'tx22c5de63466e4c05bb104-0054740c39',
'date': 'Tue, 25 Nov 2014 04:57:29 GMT',
'x-timestamp': '1416889793.23520', 'x-container-read': '.r:mysite.com',
'x-container-bytes-used': '0', 'content-type': 'text/plain; charset=utf-8'}
To set the metadata for a container, use the
:meth:`~openstack.object_store.v1._proxy.Proxy.set_container_metadata` method.
This method takes a :class:`~openstack.object_store.v1.container.Container`
object. For example, to grant another user write access to this container,
you can set the
:attr:`~openstack.object_store.v1.container.Container.write_ACL` on a
resource and pass it to `set_container_metadata`. ::
>>> cont.write_ACL = "big_project:another_user"
>>> conn.object_store.set_container_metadata(cont)
Container: {'content-length': '0', 'x-container-object-count': '0',
'name': u'my new container', 'accept-ranges': 'bytes',
'x-trans-id': 'txc3ee751f971d41de9e9f4-0054740ec1',
'date': 'Tue, 25 Nov 2014 05:08:17 GMT',
'x-timestamp': '1416889793.23520', 'x-container-read': '.r:mysite.com',
'x-container-bytes-used': '0', 'content-type': 'text/plain; charset=utf-8',
'x-container-write': 'big_project:another_user'}
Working with Objects
--------------------
Objects are held in containers. From an API standpoint, you work with
them using similarly named methods, typically with an additional argument
to specify their container.
Listing Objects
***************
To list the objects that exist in a container, use the
:meth:`~openstack.object_store.v1._proxy.Proxy.objects` method.
If you have a :class:`~openstack.object_store.v1.container.Container`
object, you can pass it to ``objects``. ::
>>> print cont.name
pictures
>>> for obj in conn.object_store.objects(cont):
... print obj
...
Object: {u'hash': u'0522d4ccdf9956badcb15c4087a0c4cb',
u'name': u'pictures/selfie.jpg', u'bytes': 15744,
'last-modified': u'2014-10-31T06:33:36.618640',
u'last_modified': u'2014-10-31T06:33:36.618640',
u'content_type': u'image/jpeg', 'container': u'pictures',
'content-type': u'image/jpeg'}
...
Similar to the :meth:`~openstack.object_store.v1._proxy.Proxy.containers`
method, ``objects`` returns a generator which yields
:class:`~openstack.object_store.v1.obj.Object` objects stored in the
container. It also handles pagination for you, which you can adjust
with the ``limit`` parameter, otherwise making each request for the maximum
that your Object Store will return.
If you have the name of a container instead of an object, you can also
pass that to the ``objects`` method. ::
>>> for obj in conn.object_store.objects("pictures".decode("utf8"),
limit=100):
... print obj
...
<100 Objects>
... another request transparently made to the Object Store service
<100 more Objects>
Getting Object Data
*******************
Once you have an :class:`~openstack.object_store.v1.obj.Object`, you get
the data stored inside of it with the
:meth:`~openstack.object_store.v1._proxy.Proxy.get_object_data` method. ::
>>> print ob.name
message.txt
>>> data = conn.object_store.get_object_data(ob)
>>> print data
Hello, world!
Additionally, if you want to save the object to disk, the
:meth:`~openstack.object_store.v1._proxy.Proxy.save_object` convenience
method takes an :class:`~openstack.object_store.v1.obj.Object` and a
``path`` to write the contents to. ::
>>> conn.object_store.save_object(ob, "the_message.txt")
Creating Objects
****************
Once you have data you'd like to store in the Object Store service, you use
the :meth:`~openstack.object_store.v1._proxy.Proxy.create_object` method.
This method takes the ``data`` to be stored, along with an ``obj`` and
``container``. The ``obj`` can either be the name of an object or an
:class:`~openstack.object_store.v1.obj.Object` instance, and ``container``
can either be the name of a container or an
:class:`~openstack.object_store.v1.container.Container` instance. ::
>>> hello = conn.object_store.create_object("Hello, world!",
"helloworld.txt".decode("utf8"),
"My Container".decode("utf8"))
>>> print hello
Object: {'content-length': '0', 'container': u'My Container',
'name': u'helloworld.txt',
'last-modified': 'Tue, 25 Nov 2014 17:39:29 GMT',
'etag': '5eb63bbbe01eeed093cb22bb8f5acdc3',
'x-trans-id': 'tx3035d41b03334aeaaf3dd-005474bed0',
'date': 'Tue, 25 Nov 2014 17:39:28 GMT',
'content-type': 'text/html; charset=UTF-8'}
If you have an existing object and want to update its data, you can easily
do that by passing new ``data`` along with existing
:class:`~openstack.object_store.v1.obj.Object` and
:class:`~openstack.object_store.v1.container.Container` instances. ::
>>> conn.object_store.create_object("Hola, mundo!", hello, cont)
Working with Object Metadata
****************************
Working with metadata on objects is identical to how it's done with
containers. You use the
:meth:`~openstack.object_store.v1._proxy.Proxy.get_object_metadata` and
:meth:`~openstack.object_store.v1._proxy.Proxy.set_object_metadata` methods.
The metadata attributes to be set can be found on the
:class:`~openstack.object_store.v1.obj.Object` object. ::
>>> secret.delete_after = 300
>>> secret = conn.object_store.set_object_metadata(secret)
We set the :attr:`~openstack.object_store.obj.Object.delete_after`
value to 500 seconds, causing the object to be deleted in 300 seconds,
or five minutes. That attribute corresponds to the ``X-Delete-After``
header value, which you can see is returned when we retreive the updated
metadata. ::
>>> conn.object_store.get_object_metadata(ob)
Object: {'content-length': '11', 'container': u'Secret Container',
'name': u'selfdestruct.txt', 'x-delete-after': 300,
'accept-ranges': 'bytes', 'last-modified': 'Tue, 25 Nov 2014 17:50:45 GMT',
'etag': '5eb63bbbe01eeed093cb22bb8f5acdc3',
'x-timestamp': '1416937844.36805',
'x-trans-id': 'tx5c3fd94adf7c4e1b8f334-005474c17b',
'date': 'Tue, 25 Nov 2014 17:50:51 GMT', 'content-type': 'text/plain'}

104
examples/object_store.py Normal file
View File

@ -0,0 +1,104 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from __future__ import print_function
import glob
import os
import sys
from examples import common
from openstack import connection
CONTAINER_HEADER = ("Name{0}| Bytes Used{1}| "
"Num Objects".format(13 * " ", 1 * " "))
CONTAINER_FORMAT = ("{0.name: <16} | {0.bytes: <10} | {0.count}")
OBJECT_HEADER = ("Name{0}| Bytes {1}| "
"Content-Type".format(27 * " ", 2 * " "))
OBJECT_FORMAT = ("{0.name: <30} | {0.bytes: <7} | {0.content_type}")
def list_containers(conn):
print(CONTAINER_HEADER)
print("=" * len(CONTAINER_HEADER))
for container in conn.object_store.containers():
print(CONTAINER_FORMAT.format(container))
def list_objects(conn, container):
print(OBJECT_HEADER)
print("=" * len(OBJECT_HEADER))
for obj in conn.object_store.objects(container.decode("utf8")):
print(OBJECT_FORMAT.format(obj))
def upload_directory(conn, directory, pattern):
"""Upload a directory to object storage.
Given an OpenStack connection, a directory, and a file glob pattern,
upload all files matching the pattern from that directory into a
container named after the directory containing the files.
"""
container_name = os.path.basename(os.path.realpath(directory))
container = conn.object_store.create_container(
container_name.decode("utf8"))
for root, dirs, files in os.walk(directory):
for file in glob.iglob(os.path.join(root, pattern)):
with open(file, "rb") as f:
ob = conn.object_store.create_object(data=f.read(),
obj=file.decode("utf8"),
container=container)
print("Uploaded {0.name}".format(ob))
def main():
# Add on to the common parser with a few options of our own.
parser = common.option_parser()
parser.add_argument("--list-containers", dest="list_containers",
action="store_true")
parser.add_argument("--list-objects", dest="container")
parser.add_argument("--upload-directory", dest="directory")
parser.add_argument("--pattern", dest="pattern")
opts = parser.parse_args()
args = {
'auth_plugin': opts.auth_plugin,
'auth_url': opts.auth_url,
'project_name': opts.project_name,
'domain_name': opts.domain_name,
'project_domain_name': opts.project_domain_name,
'user_domain_name': opts.user_domain_name,
'user_name': opts.user_name,
'password': opts.password,
'verify': opts.verify,
'token': opts.token,
}
conn = connection.Connection(**args)
if opts.list_containers:
return list_containers(conn)
elif opts.container:
return list_objects(conn, opts.container)
elif opts.directory and opts.pattern:
return upload_directory(conn, opts.directory.decode("utf8"),
opts.pattern)
else:
print(parser.print_help())
return -1
if __name__ == "__main__":
sys.exit(main())

168
openstack/object_store/v1/_proxy.py
View File

@ -10,8 +10,176 @@
# License for the specific language governing permissions and limitations
# under the License.
from openstack.object_store.v1 import container as _container
from openstack.object_store.v1 import obj as _obj
class Proxy(object):
def __init__(self, session):
self.session = session
def get_account_metadata(self, container=None):
"""Get metatdata for this account.
:param container: The container to retreive metadata for.
:type container:
:class:`~openstack.object_store.v1.container.Container`
"""
# TODO(briancurtin): should just use Container.head directly?
if container is None:
container = _container.Container()
container.head(self.session)
return container
def set_account_metadata(self, container):
"""Set metatdata for this account.
:param container: The container to set metadata for.
:type container:
:class:`~openstack.object_store.v1.container.Container`
"""
container.update(self.session)
return container
def containers(self, limit=None, marker=None, **kwargs):
"""Return a generator that yields the account's Container objects.
:param int limit: Set the limit of how many containers to retrieve.
:param str marker: The name of the container to begin iterating from.
"""
return _container.Container.list(self.session, limit=limit,
marker=marker, **kwargs)
def get_container_metadata(self, container):
"""Get metatdata for a container.
:param container: The container to retreive metadata for.
:type container:
:class:`~openstack.object_store.v1.container.Container`
"""
container = _container.Container.from_id(container)
# TODO(briancurtin): may want to check if the container has a
# name at this point. If it doesn't, this call will work but it's
# actually getting *account* metadata.
container.head(self.session)
return container
def set_container_metadata(self, container):
"""Set metatdata for a container.
:param container: The container to set metadata for.
:type container:
:class:`~openstack.object_store.v1.container.Container`
"""
container.create(self.session)
return container
def create_container(self, container):
"""Create a container,
:param container: A container name or object.
:type container:
:class:`~openstack.object_store.v1.container.Container`
"""
container = _container.Container.from_id(container)
container.create(self.session)
return container
def delete_container(self, container):
"""Delete a container.
:param container: A container name or object.
:type container:
:class:`~openstack.object_store.v1.container.Container`
"""
container = _container.Container.from_id(container)
container.delete(self.session)
def objects(self, container, limit=None, marker=None, **kwargs):
"""Return a generator that yields the Container's objects.
:param container: A container name or object.
:type container:
:class:`~openstack.object_store.v1.container.Container`
"""
container = _container.Container.from_id(container)
objs = _obj.Object.list(self.session, limit=limit, marker=marker,
path_args={"container": container.name},
**kwargs)
# TODO(briancurtin): Objects have to know their container at this
# point, otherwise further operations like getting their metadata
# or downloading them is a hassle because the end-user would have
# to maintain both the container and the object separately.
for ob in objs:
ob.container = container.name
yield ob
def get_object_data(self, obj):
"""Retreive the data contained inside an object.
:param obj: The object to retreive.
:type obj: :class:`~openstack.object_store.v1.obj.Object`
"""
return obj.get(self.session)
def save_object(self, obj, path):
"""Save the data contained inside an object to disk.
:param obj: The object to save to disk.
:type obj: :class:`~openstack.object_store.v1.obj.Object`
:param path str: Location to write the object contents.
"""
with open(path, "w") as out:
out.write(self.get_object_data(obj))
def create_object(self, data, obj, container=None, **kwargs):
"""Create an object within the object store.
:param data: The data to store.
:param obj: The name of the object to create, or an obj.Object
:type obj: :class:`~openstack.object_store.v1.obj.Object`
"""
obj = _obj.Object.from_id(obj)
# If we were given an Object complete with an underlying Container,
# this attribute access will succeed. Otherwise we'll need to set
# a container value on `obj` out of the `container` value.
name = getattr(obj, "container")
if not name:
cnt = _container.Container.from_id(container)
obj.container = cnt.name
obj.create(self.session, data)
return obj
def copy_object(self):
"""Copy an object."""
raise NotImplementedError
def delete_object(self, obj):
"""Delete an object.
:param obj: The object to delete.
:type obj: :class:`~openstack.object_store.v1.obj.Object`
"""
obj.delete(self.session)
def get_object_metadata(self, obj):
"""Get metatdata for an object.
:param obj: The object to retreive metadata from.
:type obj: :class:`~openstack.object_store.v1.obj.Object`
"""
obj.head(self.session)
return obj
def set_object_metadata(self, obj):
"""Set metatdata for an object.
:param obj: The object to set metadata for.
:type obj: :class:`~openstack.object_store.v1.obj.Object`
"""
obj.create(self.session)
return obj

2
openstack/object_store/v1/container.py
View File

@ -96,7 +96,7 @@ class Container(resource.Resource):
#: Content-Type header, if present.
detect_content_type = resource.prop("x-detect-content-type", type=bool)
#: In combination with Expect: 100-Continue, specify an
#: "If-None-Match: *" header to query whether the server already
#: "If-None-Match: \*" header to query whether the server already
#: has a copy of the object before any data is sent.
if_none_match = resource.prop("if-none-match")

18
openstack/object_store/v1/obj.py
View File

@ -53,7 +53,7 @@ class Object(resource.Resource):
#: See http://www.ietf.org/rfc/rfc2616.txt.
if_match = resource.prop("if-match", type=dict)
#: In combination with Expect: 100-Continue, specify an
#: "If-None-Match: *" header to query whether the server already
#: "If-None-Match: \*" header to query whether the server already
#: has a copy of the object before any data is sent.
if_none_match = resource.prop("if-none-match", type=dict)
#: See http://www.ietf.org/rfc/rfc2616.txt.
@ -167,3 +167,19 @@ class Object(resource.Resource):
headers=headers).content
return resp
def create(self, session, data=None):
"""Create a remote resource from this instance."""
if not self.allow_create:
raise exceptions.MethodNotSupported('create')
url = utils.urljoin("", self.base_path % self, self.id)
if data is not None:
resp = session.put(url, service=self.service, data=data,
accept="bytes").headers
else:
resp = session.post(url, service=self.service, data=None,
accept=None).headers
self._attrs.update(resp)

442
openstack/tests/object_store/v1/test_proxy.py Normal file
View File

@ -0,0 +1,442 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import json
import httpretty
import mock
import six
from openstack.object_store.v1 import _proxy
from openstack.object_store.v1 import container
from openstack.object_store.v1 import obj
from openstack import session
from openstack.tests import base
from openstack.tests import fakes
from openstack.tests import test_proxy_base
from openstack import transport
class TestObjectStoreProxy(test_proxy_base.TestProxyBase):
def setUp(self):
super(TestObjectStoreProxy, self).setUp()
self.proxy = _proxy.Proxy(self.session)
class Test_account_metadata(TestObjectStoreProxy):
def _test_container_object(self, method, verb):
container = mock.MagicMock()
result = method(container)
self.assertIs(result, container)
getattr(container, verb).assert_called_once_with(self.session)
def test_get_account_metadata(self):
self._test_container_object(self.proxy.get_account_metadata, "head")
def test_set_account_metadata(self):
self._test_container_object(self.proxy.set_account_metadata, "update")
@mock.patch("openstack.object_store.v1._proxy._container.Container")
def test_get_account_metadata_no_arg(self, mock_container):
created_container = mock.MagicMock()
mock_container.return_value = created_container
self.proxy.get_account_metadata()
mock_container.assert_called_once_with()
created_container.head.assert_called_once_with(self.session)
class Test_containers(TestObjectStoreProxy, base.TestTransportBase):
TEST_URL = fakes.FakeAuthenticator.ENDPOINT
def setUp(self):
super(Test_containers, self).setUp()
self.transport = transport.Transport(accept=transport.JSON)
self.auth = fakes.FakeAuthenticator()
self.session = session.Session(self.transport, self.auth)
self.proxy = _proxy.Proxy(self.session)
self.containers_body = []
for i in range(3):
self.containers_body.append({six.text_type("name"):
six.text_type("container%d" % i)})
@httpretty.activate
def test_all_containers(self):
self.stub_url(httpretty.GET,
path=[container.Container.base_path],
responses=[httpretty.Response(
body=json.dumps(self.containers_body),
status=200, content_type="application/json"),
httpretty.Response(body=json.dumps([]),
status=200, content_type="application/json")])
count = 0
for actual, expected in zip(self.proxy.containers(),
self.containers_body):
self.assertEqual(actual, expected)
count += 1
self.assertEqual(count, len(self.containers_body))
@httpretty.activate
def test_containers_limited(self):
limit = len(self.containers_body) + 1
limit_param = "?limit=%d" % limit
self.stub_url(httpretty.GET,
path=[container.Container.base_path + limit_param],
json=self.containers_body)
count = 0
for actual, expected in zip(self.proxy.containers(limit=limit),
self.containers_body):
self.assertEqual(actual, expected)
count += 1
self.assertEqual(count, len(self.containers_body))
# Since we've chosen a limit larger than the body, only one request
# should be made, so it should be the last one.
self.assertIn(limit_param, httpretty.last_request().path)
@httpretty.activate
def test_containers_with_marker(self):
marker = six.text_type("container2")
marker_param = "?marker=%s" % marker
self.stub_url(httpretty.GET,
path=[container.Container.base_path + marker_param],
json=self.containers_body)
count = 0
for actual, expected in zip(self.proxy.containers(marker=marker),
self.containers_body):
# Make sure the marker made it into the actual request.
self.assertIn(marker_param, httpretty.last_request().path)
self.assertEqual(actual, expected)
count += 1
self.assertEqual(count, len(self.containers_body))
# Since we have to make one request beyond the end, because no
# limit was provided, make sure the last container appears as
# the marker in this last request.
self.assertIn(self.containers_body[-1]["name"],
httpretty.last_request().path)
class Test_container_metadata(TestObjectStoreProxy):
@mock.patch("openstack.resource.Resource.from_id")
def test_get_container_metadata_object(self, mock_fi):
container = mock.MagicMock()
mock_fi.return_value = container
result = self.proxy.get_container_metadata(container)
self.assertIs(result, container)
container.head.assert_called_once_with(self.session)
@mock.patch("openstack.resource.Resource.from_id")
def test_get_container_metadata_name(self, mock_fi):
name = six.text_type("my_container")
created_container = mock.MagicMock()
created_container.name = name
mock_fi.return_value = created_container
result = self.proxy.get_container_metadata(name)
self.assertEqual(result.name, name)
created_container.head.assert_called_once_with(self.session)
def test_set_container_metadata_object(self):
container = mock.MagicMock()
result = self.proxy.set_container_metadata(container)
self.assertIs(result, container)
container.create.assert_called_once_with(self.session)
class Test_create_container(TestObjectStoreProxy):
@mock.patch("openstack.resource.Resource.from_id")
def test_container_object(self, mock_fi):
container = mock.MagicMock()
mock_fi.return_value = container
result = self.proxy.create_container(container)
self.assertIs(result, container)
container.create.assert_called_once_with(self.session)
@mock.patch("openstack.resource.Resource.from_id")
def test_container_name(self, mock_fi):
name = six.text_type("my_container")
created_container = mock.MagicMock()
created_container.name = name
mock_fi.return_value = created_container
result = self.proxy.create_container(name)
self.assertEqual(result.name, name)
created_container.create.assert_called_once_with(self.session)
class Test_delete_container(TestObjectStoreProxy):
@mock.patch("openstack.resource.Resource.from_id")
def test_container_object(self, mock_fi):
container = mock.MagicMock()
mock_fi.return_value = container
result = self.proxy.delete_container(container)
self.assertIsNone(result)
container.delete.assert_called_once_with(self.session)
@mock.patch("openstack.resource.Resource.from_id")
def test_container_name(self, mock_fi):
name = six.text_type("my_container")
created_container = mock.MagicMock()
created_container.name = name
mock_fi.return_value = created_container
result = self.proxy.delete_container(name)
self.assertIsNone(result)
created_container.delete.assert_called_once_with(self.session)
class Test_objects(TestObjectStoreProxy, base.TestTransportBase):
TEST_URL = fakes.FakeAuthenticator.ENDPOINT
def setUp(self):
super(Test_objects, self).setUp()
self.transport = transport.Transport(accept=transport.JSON)
self.auth = fakes.FakeAuthenticator()
self.session = session.Session(self.transport, self.auth)
self.proxy = _proxy.Proxy(self.session)
self.container_name = six.text_type("my_container")
self.objects_body = []
for i in range(3):
self.objects_body.append({six.text_type("name"):
six.text_type("object%d" % i)})
# Returned object bodies have their container inserted.
self.returned_objects = []
for ob in self.objects_body:
ob[six.text_type("container")] = self.container_name
self.returned_objects.append(ob)
self.assertEqual(len(self.objects_body), len(self.returned_objects))
@httpretty.activate
def test_all_objects(self):
self.stub_url(httpretty.GET,
path=[obj.Object.base_path %
{"container": self.container_name}],
responses=[httpretty.Response(
body=json.dumps(self.objects_body),
status=200, content_type="application/json"),
httpretty.Response(body=json.dumps([]),
status=200, content_type="application/json")])
count = 0
for actual, expected in zip(self.proxy.objects(self.container_name),
self.returned_objects):
self.assertEqual(actual, expected)
count += 1
self.assertEqual(count, len(self.returned_objects))
@httpretty.activate
def test_objects_limited(self):
limit = len(self.objects_body) + 1
limit_param = "?limit=%d" % limit
self.stub_url(httpretty.GET,
path=[obj.Object.base_path %
{"container": self.container_name} + limit_param],
json=self.objects_body)
count = 0
for actual, expected in zip(self.proxy.objects(self.container_name,
limit=limit),
self.returned_objects):
self.assertEqual(actual, expected)
count += 1
self.assertEqual(count, len(self.returned_objects))
# Since we've chosen a limit larger than the body, only one request
# should be made, so it should be the last one.
self.assertIn(limit_param, httpretty.last_request().path)
@httpretty.activate
def test_objects_with_marker(self):
marker = six.text_type("object2")
marker_param = "?marker=%s" % marker
self.stub_url(httpretty.GET,
path=[obj.Object.base_path %
{"container": self.container_name} + marker_param],
json=self.objects_body)
count = 0
for actual, expected in zip(self.proxy.objects(self.container_name,
marker=marker),
self.returned_objects):
# Make sure the marker made it into the actual request.
self.assertIn(marker_param, httpretty.last_request().path)
self.assertEqual(actual, expected)
count += 1
self.assertEqual(count, len(self.returned_objects))
# Since we have to make one request beyond the end, because no
# limit was provided, make sure the last container appears as
# the marker in this last request.
self.assertIn(self.returned_objects[-1]["name"],
httpretty.last_request().path)
class Test_get_object_data(TestObjectStoreProxy):
def test_get(self):
the_data = "here's some data"
ob = mock.MagicMock()
ob.get.return_value = the_data
result = self.proxy.get_object_data(ob)
self.assertEqual(result, the_data)
ob.get.assert_called_once_with(self.session)
class Test_save_object(TestObjectStoreProxy):
@mock.patch("openstack.object_store.v1._proxy.Proxy.get_object_data")
def test_save(self, mock_get):
the_data = "here's some data"
mock_get.return_value = the_data
ob = mock.MagicMock()
fake_open = mock.mock_open()
file_path = "blarga/somefile"
with mock.patch("openstack.object_store.v1._proxy.open",
fake_open, create=True):
self.proxy.save_object(ob, file_path)
fake_open.assert_called_once_with(file_path, "w")
fake_handle = fake_open()
fake_handle.write.assert_called_once_with(the_data)
class Test_create_object(TestObjectStoreProxy):
def setUp(self):
super(Test_create_object, self).setUp()
self.the_data = six.b("here's some data")
self.container_name = six.text_type("my_container")
self.object_name = six.text_type("my_object")
@mock.patch("openstack.object_store.v1.obj.Object.from_id")
def test_create_with_obj_name_real_container(self, mock_fi):
created_object = mock.MagicMock()
created_object.name = self.object_name
# Since we're using a MagicMock, we have to explicitly set this to
# None otherwise when it gets accessed it'll have a value which
# is not what we want to happen.
created_object.container = None
mock_fi.return_value = created_object
cont = container.Container.new(name=self.container_name)
result = self.proxy.create_object(self.the_data, self.object_name,
cont)
self.assertIs(result, created_object)
self.assertEqual(result.name, self.object_name)
self.assertEqual(result.container, self.container_name)
created_object.create.assert_called_once_with(self.session,
self.the_data)
def test_create_with_real_obj_real_container(self):
ob = obj.Object.new(name=self.object_name)
ob.create = mock.MagicMock()
cont = container.Container.new(name=self.container_name)
result = self.proxy.create_object(self.the_data, ob, cont)
self.assertIs(result, ob)
self.assertEqual(result.name, self.object_name)
self.assertEqual(result.container, self.container_name)
ob.create.assert_called_once_with(self.session, self.the_data)
def test_create_with_full_obj_no_container_arg(self):
ob = obj.Object.new(name=self.object_name,
container=self.container_name)
ob.create = mock.MagicMock()
result = self.proxy.create_object(self.the_data, ob)
self.assertIs(result, ob)
self.assertEqual(result.name, self.object_name)
self.assertEqual(result.container, self.container_name)
ob.create.assert_called_once_with(self.session, self.the_data)
class Test_object_metadata(TestObjectStoreProxy):
@mock.patch("openstack.resource.Resource.from_id")
def test_get_object_metadata(self, mock_fi):
ob = mock.MagicMock()
mock_fi.return_value = ob
result = self.proxy.get_object_metadata(ob)
self.assertIs(result, ob)
ob.head.assert_called_once_with(self.session)
def test_set_object_metadata(self):
ob = mock.MagicMock()
result = self.proxy.set_object_metadata(ob)
self.assertIs(result, ob)
ob.create.assert_called_once_with(self.session)
class Test_delete_object(TestObjectStoreProxy):
def test_delete_object(self):
ob = mock.MagicMock()
result = self.proxy.delete_object(ob)
self.assertIsNone(result)
ob.delete.assert_called_once_with(self.session)
class Test_copy_object(TestObjectStoreProxy):
def test_copy_object(self):
self.assertRaises(NotImplementedError, self.proxy.copy_object)