899 lines
31 KiB
ReStructuredText
899 lines
31 KiB
ReStructuredText
================================
|
|
The swiftclient.SwiftService API
|
|
================================
|
|
|
|
A higher-level API aimed at allowing developers an easy way to perform multiple
|
|
operations asynchronously using a configurable thread pool. Documentation for
|
|
each service method call can be found here: :mod:`swiftclient.service`.
|
|
|
|
Authentication
|
|
--------------
|
|
|
|
This section covers the various options for authenticating with a swift
|
|
object store. The combinations of options required for each authentication
|
|
version are detailed below. Once again, these are just a subset of those that
|
|
can be used to successfully authenticate, but they are the most common and
|
|
recommended.
|
|
|
|
The relevant authentication options are presented as python dictionaries that
|
|
should be added to any other options you are supplying to your ``SwiftService``
|
|
instance. As indicated in the python code, you can also set these options as
|
|
environment variables that will be loaded automatically if the relevant option
|
|
is not specified.
|
|
|
|
The ``SwiftService`` authentication attempts to automatically select
|
|
the auth version based on the combination of options specified, but
|
|
supplying options from multiple different auth versions can cause unexpected
|
|
behaviour.
|
|
|
|
.. note::
|
|
|
|
Leftover environment variables are a common source of confusion when
|
|
authorization fails.
|
|
|
|
Keystone V3
|
|
~~~~~~~~~~~
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
...
|
|
"auth_version": environ.get('ST_AUTH_VERSION'), # Should be '3'
|
|
"os_username": environ.get('OS_USERNAME'),
|
|
"os_password": environ.get('OS_PASSWORD'),
|
|
"os_project_name": environ.get('OS_PROJECT_NAME'),
|
|
"os_project_domain_name": environ.get('OS_PROJECT_DOMAIN_NAME'),
|
|
"os_auth_url": environ.get('OS_AUTH_URL'),
|
|
...
|
|
}
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
...
|
|
"auth_version": environ.get('ST_AUTH_VERSION'), # Should be '3'
|
|
"os_username": environ.get('OS_USERNAME'),
|
|
"os_password": environ.get('OS_PASSWORD'),
|
|
"os_project_id": environ.get('OS_PROJECT_ID'),
|
|
"os_project_domain_id": environ.get('OS_PROJECT_DOMAIN_ID'),
|
|
"os_auth_url": environ.get('OS_AUTH_URL'),
|
|
...
|
|
}
|
|
|
|
Keystone V2
|
|
~~~~~~~~~~~
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
...
|
|
"auth_version": environ.get('ST_AUTH_VERSION'), # Should be '2.0'
|
|
"os_username": environ.get('OS_USERNAME'),
|
|
"os_password": environ.get('OS_PASSWORD'),
|
|
"os_tenant_name": environ.get('OS_TENANT_NAME'),
|
|
"os_auth_url": environ.get('OS_AUTH_URL'),
|
|
...
|
|
}
|
|
|
|
Legacy Auth
|
|
~~~~~~~~~~~
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
...
|
|
"auth_version": environ.get('ST_AUTH_VERSION'), # Should be '1.0'
|
|
"auth": environ.get('ST_AUTH'),
|
|
"user": environ.get('ST_USER'),
|
|
"key": environ.get('ST_KEY'),
|
|
...
|
|
}
|
|
|
|
Configuration
|
|
-------------
|
|
|
|
When you create an instance of a ``SwiftService``, you can override a collection
|
|
of default options to suit your use case. Typically, the defaults are sensible to
|
|
get us started, but depending on your needs you might want to tweak them to
|
|
improve performance (options affecting large objects and thread counts can
|
|
significantly alter performance in the right situation).
|
|
|
|
Service level defaults and some extra options can also be overridden on a
|
|
per-operation (or even in some cases per-object) basis, and you will call out
|
|
which options affect which operations later in the document.
|
|
|
|
The configuration of the service API is performed using an options dictionary
|
|
passed to the ``SwiftService`` during initialisation. The options available
|
|
in this dictionary are described below, along with their defaults:
|
|
|
|
Options
|
|
~~~~~~~
|
|
|
|
``retries``: ``5``
|
|
The number of times that the library should attempt to retry HTTP
|
|
actions before giving up and reporting a failure.
|
|
|
|
``container_threads``: ``10``
|
|
|
|
``object_dd_threads``: ``10``
|
|
|
|
``object_uu_threads``: ``10``
|
|
|
|
``segment_threads``: ``10``
|
|
The above options determine the size of the available thread pools for
|
|
performing swift operations. Container operations (such as listing a
|
|
container) operate in the container threads, and a similar pattern
|
|
applies to object and segment threads.
|
|
|
|
.. note::
|
|
|
|
Object threads are separated into two separate thread pools:
|
|
``uu`` and ``dd``. This stands for "upload/update" and "download/delete",
|
|
and the corresponding actions will be run on separate threads pools.
|
|
|
|
``segment_size``: ``None``
|
|
If specified, this option enables uploading of large objects. Should the
|
|
object being uploaded be larger than 5G in size, this option is
|
|
mandatory otherwise the upload will fail. This option should be
|
|
specified as a size in bytes.
|
|
|
|
``use_slo``: ``False``
|
|
Used in combination with the above option, ``use_slo`` will upload large
|
|
objects as static rather than dynamic. Only static large objects provide
|
|
error checking for the downloaded object, so we recommend this option.
|
|
|
|
``segment_container``: ``None``
|
|
Allows the user to select the container into which large object segments
|
|
will be uploaded. We do not recommend changing this value as it could make
|
|
locating orphaned segments more difficult in the case of errors.
|
|
|
|
``leave_segments``: ``False``
|
|
Setting this option to true means that when deleting or overwriting a large
|
|
object, its segments will be left in the object store and must be cleaned
|
|
up manually. This option can be useful when sharing large object segments
|
|
between multiple objects in more advanced scenarios, but must be treated
|
|
with care, as it could lead to ever increasing storage usage.
|
|
|
|
``changed``: ``None``
|
|
This option affects uploads and simply means that those objects which
|
|
already exist in the object store will not be overwritten if the ``mtime``
|
|
and size of the source is the same as the existing object.
|
|
|
|
``skip_identical``: ``False``
|
|
A slightly more thorough case of the above, but rather than ``mtime`` and size
|
|
uses an object's ``MD5 sum``.
|
|
|
|
``yes_all``: ``False``
|
|
This options affects only download and delete, and in each case must be
|
|
specified in order to download/delete the entire contents of an account.
|
|
This option has no effect on any other calls.
|
|
|
|
``no_download``: ``False``
|
|
This option only affects download and means that all operations proceed as
|
|
normal with the exception that no data is written to disk.
|
|
|
|
``header``: ``[]``
|
|
Used with upload and post operations to set headers on objects. Headers
|
|
are specified as colon separated strings, e.g. "content-type:text/plain".
|
|
|
|
``meta``: ``[]``
|
|
Used to set metadata on an object similarly to headers.
|
|
|
|
.. note::
|
|
Setting metadata is a destructive operation, so when updating one
|
|
of many metadata values all desired metadata for an object must be re-applied.
|
|
|
|
``long``: ``False``
|
|
Affects only list operations, and results in more metrics being made
|
|
available in the results at the expense of lower performance.
|
|
|
|
``fail_fast``: ``False``
|
|
Applies to delete and upload operations, and attempts to abort queued
|
|
tasks in the event of errors.
|
|
|
|
``prefix``: ``None``
|
|
Affects list operations; only objects with the given prefix will be
|
|
returned/affected. It is not advisable to set at the service level, as
|
|
those operations that call list to discover objects on which they should
|
|
operate will also be affected.
|
|
|
|
``delimiter``: ``None``
|
|
Affects list operations, and means that listings only contain results up
|
|
to the first instance of the delimiter in the object name. This is useful
|
|
for working with objects containing '/' in their names to simulate folder
|
|
structures.
|
|
|
|
``dir_marker``: ``False``
|
|
Affects uploads, and allows empty 'pseudofolder' objects to be created
|
|
when the source of an upload is ``None``.
|
|
|
|
``checksum``: ``True``
|
|
Affects uploads and downloads. If set check md5 sum for the transfer.
|
|
|
|
``shuffle``: ``False``
|
|
When downloading objects, the default behaviour of the CLI is to shuffle
|
|
lists of objects in order to spread the load on storage drives when multiple
|
|
clients are downloading the same files to multiple locations (e.g. in the
|
|
event of distributing an update). When using the ``SwiftService`` directly,
|
|
object downloads are scheduled in the same order as they appear in the container
|
|
listing. When combined with a single download thread this means that objects
|
|
are downloaded in lexically-sorted order. Setting this option to ``True``
|
|
gives the same shuffling behaviour as the CLI.
|
|
|
|
``destination``: ``None``
|
|
When copying objects, this specifies the destination where the object
|
|
will be copied to. The default of None means copy will be the same as
|
|
source.
|
|
|
|
``fresh_metadata``: ``None``
|
|
When copying objects, this specifies that the object metadata on the
|
|
source will *not* be applied to the destination object - the
|
|
destination object will have a new fresh set of metadata that includes
|
|
*only* the metadata specified in the meta option if any at all.
|
|
|
|
Other available options can be found in ``swiftclient/service.py`` in the
|
|
source code for ``python-swiftclient``. Each ``SwiftService`` method also allows
|
|
for an optional dictionary to override those specified at init time, and the
|
|
appropriate docstrings show which options modify each method's behaviour.
|
|
|
|
Available Operations
|
|
--------------------
|
|
|
|
Each operation provided by the service API may raise a ``SwiftError`` or
|
|
``ClientException`` for any call that fails completely (or a call which
|
|
performs only one operation at an account or container level). In the case of a
|
|
successful call an operation returns one of the following:
|
|
|
|
* A dictionary detailing the results of a single operation.
|
|
* An iterator that produces result dictionaries (for calls that perform
|
|
multiple sub-operations).
|
|
|
|
A result dictionary can indicate either the success or failure of an individual
|
|
operation (detailed in the ``success`` key), and will either contain the
|
|
successful result, or an ``error`` key detailing the error encountered
|
|
(usually an instance of Exception).
|
|
|
|
An example result dictionary is given below:
|
|
|
|
.. code-block:: python
|
|
|
|
result = {
|
|
'action': 'download_object',
|
|
'success': True,
|
|
'container': container,
|
|
'object': obj,
|
|
'path': path,
|
|
'start_time': start_time,
|
|
'finish_time': finish_time,
|
|
'headers_receipt': headers_receipt,
|
|
'auth_end_time': conn.auth_end_time,
|
|
'read_length': bytes_read,
|
|
'attempts': conn.attempts
|
|
}
|
|
|
|
All the possible ``action`` values are detailed below:
|
|
|
|
.. code-block:: python
|
|
|
|
[
|
|
'stat_account',
|
|
'stat_container',
|
|
'stat_object',
|
|
'post_account',
|
|
'post_container',
|
|
'post_object',
|
|
'list_part', # list yields zero or more 'list_part' results
|
|
'download_object',
|
|
'create_container', # from upload
|
|
'create_dir_marker', # from upload
|
|
'upload_object',
|
|
'upload_segment',
|
|
'delete_container',
|
|
'delete_object',
|
|
'delete_segment', # from delete_object operations
|
|
'capabilities',
|
|
]
|
|
|
|
Stat
|
|
~~~~
|
|
|
|
Stat can be called against an account, a container, or a list of objects to
|
|
get account stats, container stats or information about the given objects. In
|
|
the first two cases a dictionary is returned containing the results of the
|
|
operation, and in the case of a list of object names being supplied, an
|
|
iterator over the results generated for each object is returned.
|
|
|
|
Information returned includes the amount of data used by the given
|
|
object/container/account and any headers or metadata set (this includes
|
|
user set data as well as content-type and modification times).
|
|
|
|
See :mod:`swiftclient.service.SwiftService.stat` for docs generated from the
|
|
method docstring.
|
|
|
|
Valid calls for this method are as follows:
|
|
|
|
* ``stat([options])``: Returns stats for the configured account.
|
|
* ``stat(<container>, [options])``: Returns stats for the given container.
|
|
* ``stat(<container>, <object_list>, [options])``: Returns stats for each
|
|
of the given objects in the given container (through the returned
|
|
iterator).
|
|
|
|
Results from stat are dictionaries indicating the success or failure of each
|
|
operation. In the case of a successful stat against an account or container,
|
|
the method returns immediately with one of the following results:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': 'stat_account',
|
|
'success': True,
|
|
'items': items,
|
|
'headers': headers
|
|
}
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': 'stat_container',
|
|
'container': <container>,
|
|
'success': True,
|
|
'items': items,
|
|
'headers': headers
|
|
}
|
|
|
|
In the case of stat called against a list of objects, the method returns a
|
|
generator that returns the results of individual object stat operations as they
|
|
are performed on the thread pool:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': 'stat_object',
|
|
'object': <object_name>,
|
|
'container': <container>,
|
|
'success': True,
|
|
'items': items,
|
|
'headers': headers
|
|
}
|
|
|
|
In the case of a failure the dictionary returned will indicate that the
|
|
operation was not successful, and will include the keys below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': <'stat_object'|'stat_container'|'stat_account'>,
|
|
'object': <'object_name'>, # Only for stat with objects list
|
|
'container': <container>, # Only for stat with objects list or container
|
|
'success': False,
|
|
'error': <error>,
|
|
'traceback': <trace>,
|
|
'error_timestamp': <timestamp>
|
|
}
|
|
|
|
.. topic:: Example
|
|
|
|
The code below demonstrates the use of ``stat`` to retrieve the headers for
|
|
a given list of objects in a container using 20 threads. The code creates a
|
|
mapping from object name to headers which is then pretty printed to the log.
|
|
|
|
.. literalinclude:: ../../examples/stat.py
|
|
:language: python
|
|
|
|
List
|
|
~~~~
|
|
|
|
List can be called against an account or a container to retrieve the containers
|
|
or objects contained within them. Each call returns an iterator that returns
|
|
pages of results (by default, up to 10000 results in each page).
|
|
|
|
See :mod:`swiftclient.service.SwiftService.list` for docs generated from the
|
|
method docstring.
|
|
|
|
If the given container or account does not exist, the list method will raise
|
|
a ``SwiftError``, but for all other success/failures a dictionary is returned.
|
|
Each successfully listed page returns a dictionary as described below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': <'list_account_part'|'list_container_part'>,
|
|
'container': <container>, # Only for listing a container
|
|
'prefix': <prefix>, # The prefix of returned objects/containers
|
|
'success': True,
|
|
'listing': [Item], # A list of results
|
|
# (only in the event of success)
|
|
'marker': <marker> # The last item name in the list
|
|
# (only in the event of success)
|
|
}
|
|
|
|
Where an item contains the following keys:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'name': <name>,
|
|
'bytes': 10485760,
|
|
'last_modified': '2014-12-11T12:02:38.774540',
|
|
'hash': 'fb938269cbeabe4c234e1127bbd3b74a',
|
|
'content_type': 'application/octet-stream',
|
|
'meta': <metadata> # Full metadata listing from stat'ing each object
|
|
# this key only exists if 'long' is specified in options
|
|
}
|
|
|
|
Any failure listing an account or container that exists will return a failure
|
|
dictionary as described below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': <'list_account_part'|'list_container_part'>,,
|
|
'container': container, # Only for listing a container
|
|
'prefix': options['prefix'],
|
|
'success': success,
|
|
'marker': marker,
|
|
'error': error,
|
|
'traceback': <trace>,
|
|
'error_timestamp': <timestamp>
|
|
}
|
|
|
|
.. topic:: Example
|
|
|
|
The code below demonstrates the use of ``list`` to list all items in a
|
|
container that are over 10MiB in size:
|
|
|
|
.. literalinclude:: ../../examples/list.py
|
|
:language: python
|
|
|
|
Post
|
|
~~~~
|
|
|
|
Post can be called against an account, container or list of objects in order to
|
|
update the metadata attached to the given items. In the first two cases a single
|
|
dictionary is returned containing the results of the operation, and in the case
|
|
of a list of objects being supplied, an iterator over the results generated for
|
|
each object post is returned.
|
|
|
|
Each element of the object list may be a plain string of the object name, or a
|
|
``SwiftPostObject`` that allows finer control over the options and metadata
|
|
applied to each of the individual post operations. When a string is given for
|
|
the object name, the options and metadata applied are a combination of those
|
|
supplied to the call to ``post()`` and the defaults of the ``SwiftService``
|
|
object.
|
|
|
|
If the given container or account does not exist, the ``post`` method will
|
|
raise a ``SwiftError``. Successful metadata update results are dictionaries as
|
|
described below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': <'post_account'|'post_container'|'post_object'>,
|
|
'success': True,
|
|
'container': <container>,
|
|
'object': <object>,
|
|
'headers': {},
|
|
'response_dict': <HTTP response details>
|
|
}
|
|
|
|
.. note::
|
|
Updating user metadata keys will not only add any specified keys, but
|
|
will also remove user metadata that has previously been set. This means
|
|
that each time user metadata is updated, the complete set of desired
|
|
key-value pairs must be specified.
|
|
|
|
.. topic:: Example
|
|
|
|
The code below demonstrates the use of ``post`` to set an archive folder in
|
|
a given container to expire after a 24 hour delay:
|
|
|
|
.. literalinclude:: ../../examples/post.py
|
|
:language: python
|
|
|
|
Download
|
|
~~~~~~~~
|
|
|
|
Download can be called against an entire account, a single container, or a list
|
|
of objects in a given container. Each element of the object list is a string
|
|
detailing the full name of an object to download.
|
|
|
|
In order to download the full contents of an entire account, you must set the
|
|
value of ``yes_all`` to ``True`` in the ``options`` dictionary supplied to
|
|
either the ``SwiftService`` instance or the call to ``download``.
|
|
|
|
If the given container or account does not exist, the ``download`` method will
|
|
raise a ``SwiftError``, otherwise an iterator over the results generated for
|
|
each object download is returned.
|
|
|
|
See :mod:`swiftclient.service.SwiftService.download` for docs generated from the
|
|
method docstring.
|
|
|
|
For each successfully downloaded object, the results returned by the iterator
|
|
will be a dictionary as described below (results are not returned for completed
|
|
container or object segment downloads):
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': 'download_object',
|
|
'container': <container>,
|
|
'object': <object name>,
|
|
'success': True,
|
|
'path': <local path to downloaded object>,
|
|
'pseudodir': <if true, the download created an empty directory>,
|
|
'start_time': <time download started>,
|
|
'end_time': <time download completed>,
|
|
'headers_receipt': <time the headers from the object were retrieved>,
|
|
'auth_end_time': <time authentication completed>,
|
|
'read_length': <bytes_read>,
|
|
'attempts': <attempt count>,
|
|
'response_dict': <HTTP response details>
|
|
}
|
|
|
|
Any failure uploading an object will return a failure dictionary as described
|
|
below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': 'download_object',
|
|
'container': <container>,
|
|
'object': <object name>,
|
|
'success': False,
|
|
'path': <local path of the failed download>,
|
|
'pseudodir': <if true, the failed download was an empty directory>,
|
|
'attempts': <attempt count>,
|
|
'error': <error>,
|
|
'traceback': <trace>,
|
|
'error_timestamp': <timestamp>,
|
|
'response_dict': <HTTP response details>
|
|
}
|
|
|
|
.. topic:: Example
|
|
|
|
The code below demonstrates the use of ``download`` to download all PNG
|
|
images from a dated archive folder in a given container:
|
|
|
|
.. literalinclude:: ../../examples/download.py
|
|
:language: python
|
|
|
|
Upload
|
|
~~~~~~
|
|
|
|
Upload is always called against an account and container and with a list of
|
|
objects to upload. Each element of the object list may be a plain string
|
|
detailing the path of the object to upload, or a ``SwiftUploadObject`` that
|
|
allows finer control over some aspects of the individual operations.
|
|
|
|
When a simple string is supplied to specify a file to upload, the name of the
|
|
object uploaded is the full path of the specified file and the options used for
|
|
the upload are those supplied to the call to ``upload``.
|
|
|
|
Constructing a ``SwiftUploadObject`` allows the user to supply an object name
|
|
for the uploaded file, and modify the options used by ``upload`` at the
|
|
granularity of individual files.
|
|
|
|
If the given container or account does not exist, the ``upload`` method will
|
|
raise a ``SwiftError``, otherwise an iterator over the results generated for
|
|
each object upload is returned.
|
|
|
|
See :mod:`swiftclient.service.SwiftService.upload` for docs generated from the
|
|
method docstring.
|
|
|
|
For each successfully uploaded object (or object segment), the results returned
|
|
by the iterator will be a dictionary as described below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': 'upload_object',
|
|
'container': <container>,
|
|
'object': <object name>,
|
|
'success': True,
|
|
'status': <'uploaded'|'skipped-identical'|'skipped-changed'>,
|
|
'attempts': <attempt count>,
|
|
'response_dict': <HTTP response details>
|
|
}
|
|
|
|
{
|
|
'action': 'upload_segment',
|
|
'for_container': <container>,
|
|
'for_object': <object name>,
|
|
'segment_index': <segment_index>,
|
|
'segment_size': <segment_size>,
|
|
'segment_location': <segment_path>
|
|
'segment_etag': <etag>,
|
|
'log_line': <object segment n>
|
|
'success': True,
|
|
'response_dict': <HTTP response details>,
|
|
'attempts': <attempt count>
|
|
}
|
|
|
|
Any failure uploading an object will return a failure dictionary as described
|
|
below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': 'upload_object',
|
|
'container': <container>,
|
|
'object': <object name>,
|
|
'success': False,
|
|
'attempts': <attempt count>,
|
|
'error': <error>,
|
|
'traceback': <trace>,
|
|
'error_timestamp': <timestamp>,
|
|
'response_dict': <HTTP response details>
|
|
}
|
|
|
|
{
|
|
'action': 'upload_segment',
|
|
'for_container': <container>,
|
|
'for_object': <object name>,
|
|
'segment_index': <segment_index>,
|
|
'segment_size': <segment_size>,
|
|
'segment_location': <segment_path>,
|
|
'log_line': <object segment n>,
|
|
'success': False,
|
|
'error': <error>,
|
|
'traceback': <trace>,
|
|
'error_timestamp': <timestamp>,
|
|
'response_dict': <HTTP response details>,
|
|
'attempts': <attempt count>
|
|
}
|
|
|
|
.. topic:: Example
|
|
|
|
The code below demonstrates the use of ``upload`` to upload all files and
|
|
folders in a given directory, and rename each object by replacing the root
|
|
directory name with 'my-<d>-objects', where <d> is the name of the uploaded
|
|
directory:
|
|
|
|
.. literalinclude:: ../../examples/upload.py
|
|
:language: python
|
|
|
|
Delete
|
|
~~~~~~
|
|
|
|
Delete can be called against an account or a container to remove the containers
|
|
or objects contained within them. Each call to ``delete`` returns an iterator
|
|
over results of each resulting sub-request.
|
|
|
|
If the number of requested delete operations is large and the target swift
|
|
cluster is running the bulk middleware, the call to ``SwiftService.delete`` will
|
|
make use of bulk operations and the returned result iterator will return
|
|
``bulk_delete`` results rather than individual ``delete_object``,
|
|
``delete_container`` or ``delete_segment`` results.
|
|
|
|
See :mod:`swiftclient.service.SwiftService.delete` for docs generated from the
|
|
method docstring.
|
|
|
|
For each successfully deleted container, object or segment, the results returned
|
|
by the iterator will be a dictionary as described below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': <'delete_object'|'delete_segment'>,
|
|
'container': <container>,
|
|
'object': <object name>,
|
|
'success': True,
|
|
'attempts': <attempt count>,
|
|
'response_dict': <HTTP response details>
|
|
}
|
|
|
|
{
|
|
'action': 'delete_container',
|
|
'container': <container>,
|
|
'success': True,
|
|
'response_dict': <HTTP response details>,
|
|
'attempts': <attempt count>
|
|
}
|
|
|
|
{
|
|
'action': 'bulk_delete',
|
|
'container': <container>,
|
|
'objects': <[objects]>,
|
|
'success': True,
|
|
'attempts': <attempt count>,
|
|
'response_dict': <HTTP response details>
|
|
}
|
|
|
|
Any failure in a delete operation will return a failure dictionary as described
|
|
below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': ('delete_object'|'delete_segment'),
|
|
'container': <container>,
|
|
'object': <object name>,
|
|
'success': False,
|
|
'attempts': <attempt count>,
|
|
'error': <error>,
|
|
'traceback': <trace>,
|
|
'error_timestamp': <timestamp>,
|
|
'response_dict': <HTTP response details>
|
|
}
|
|
|
|
{
|
|
'action': 'delete_container',
|
|
'container': <container>,
|
|
'success': False,
|
|
'error': <error>,
|
|
'traceback': <trace>,
|
|
'error_timestamp': <timestamp>,
|
|
'response_dict': <HTTP response details>,
|
|
'attempts': <attempt count>
|
|
}
|
|
|
|
{
|
|
'action': 'bulk_delete',
|
|
'container': <container>,
|
|
'objects': <[objects]>,
|
|
'success': False,
|
|
'attempts': <attempt count>,
|
|
'error': <error>,
|
|
'traceback': <trace>,
|
|
'error_timestamp': <timestamp>,
|
|
'response_dict': <HTTP response details>
|
|
}
|
|
|
|
.. topic:: Example
|
|
|
|
The code below demonstrates the use of ``delete`` to remove a given list of
|
|
objects from a specified container. As the objects are deleted the
|
|
transaction ID of the relevant request is printed along with the object name
|
|
and number of attempts required. By printing the transaction ID, the printed
|
|
operations can be easily linked to events in the swift server logs:
|
|
|
|
.. literalinclude:: ../../examples/delete.py
|
|
:language: python
|
|
|
|
Copy
|
|
~~~~
|
|
|
|
Copy can be called to copy an object or update the metadata on the given items.
|
|
|
|
Each element of the object list may be a plain string of the object name, or a
|
|
``SwiftCopyObject`` that allows finer control over the options applied to each
|
|
of the individual copy operations (destination, fresh_metadata, options).
|
|
|
|
Destination should be in format /container/object; if not set, the object will be
|
|
copied onto itself. Fresh_metadata sets mode of operation on metadata. If not set,
|
|
current object user metadata will be copied/preserved; if set, all current user
|
|
metadata will be removed.
|
|
|
|
Returns an iterator over the results generated for each object copy (and may
|
|
also include the results of creating destination containers).
|
|
|
|
When a string is given for the object name, destination and fresh metadata will
|
|
default to None and None, which result in adding metadata to existing objects.
|
|
|
|
Successful copy results are dictionaries as described below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': 'copy_object',
|
|
'success': True,
|
|
'container': <container>,
|
|
'object': <object>,
|
|
'destination': <destination>,
|
|
'headers': {},
|
|
'fresh_metadata': <boolean>,
|
|
'response_dict': <HTTP response details>
|
|
}
|
|
|
|
Any failure in a copy operation will return a failure dictionary as described
|
|
below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': 'copy_object',
|
|
'success': False,
|
|
'container': <container>,
|
|
'object': <object>,
|
|
'destination': <destination>,
|
|
'headers': {},
|
|
'fresh_metadata': <boolean>,
|
|
'response_dict': <HTTP response details>,
|
|
'error': <error>,
|
|
'traceback': <traceback>,
|
|
'error_timestamp': <timestamp>
|
|
}
|
|
|
|
.. topic:: Example
|
|
|
|
The code below demonstrates the use of ``copy`` to add new user metadata for
|
|
objects a and b, and to copy object c to d (with added metadata).
|
|
|
|
.. literalinclude:: ../../examples/copy.py
|
|
:language: python
|
|
|
|
Capabilities
|
|
~~~~~~~~~~~~
|
|
|
|
Capabilities can be called against an account or a particular proxy URL in
|
|
order to determine the capabilities of the swift cluster. These capabilities
|
|
include details about configuration options and the middlewares that are
|
|
installed in the proxy pipeline.
|
|
|
|
See :mod:`swiftclient.service.SwiftService.capabilities` for docs generated from
|
|
the method docstring.
|
|
|
|
For each successful call to list capabilities, a result dictionary will be
|
|
returned with the contents described below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'action': 'capabilities',
|
|
'timestamp': <time of the call>,
|
|
'success': True,
|
|
'capabilities': <dictionary containing capability details>
|
|
}
|
|
|
|
The contents of the capabilities dictionary contain the core swift capabilities
|
|
under the key ``swift``; all other keys show the configuration options for
|
|
additional middlewares deployed in the proxy pipeline. An example capabilities
|
|
dictionary is given below:
|
|
|
|
.. code-block:: python
|
|
|
|
{
|
|
'account_quotas': {},
|
|
'bulk_delete': {
|
|
'max_deletes_per_request': 10000,
|
|
'max_failed_deletes': 1000
|
|
},
|
|
'bulk_upload': {
|
|
'max_containers_per_extraction': 10000,
|
|
'max_failed_extractions': 1000
|
|
},
|
|
'container_quotas': {},
|
|
'container_sync': {'realms': {}},
|
|
'formpost': {},
|
|
'keystoneauth': {},
|
|
'slo': {
|
|
'max_manifest_segments': 1000,
|
|
'max_manifest_size': 2097152,
|
|
'min_segment_size': 1048576
|
|
},
|
|
'swift': {
|
|
'account_autocreate': True,
|
|
'account_listing_limit': 10000,
|
|
'allow_account_management': True,
|
|
'container_listing_limit': 10000,
|
|
'extra_header_count': 0,
|
|
'max_account_name_length': 256,
|
|
'max_container_name_length': 256,
|
|
'max_file_size': 5368709122,
|
|
'max_header_size': 8192,
|
|
'max_meta_count': 90,
|
|
'max_meta_name_length': 128,
|
|
'max_meta_overall_size': 4096,
|
|
'max_meta_value_length': 256,
|
|
'max_object_name_length': 1024,
|
|
'policies': [
|
|
{'default': True, 'name': 'Policy-0'}
|
|
],
|
|
'strict_cors_mode': False,
|
|
'version': '2.2.2'
|
|
},
|
|
'tempurl': {
|
|
'methods': ['GET', 'HEAD', 'PUT']
|
|
}
|
|
}
|
|
|
|
.. topic:: Example
|
|
|
|
The code below demonstrates the use of ``capabilities`` to determine if the
|
|
Swift cluster supports static large objects, and if so, the maximum number
|
|
of segments that can be described in a single manifest file, along with the
|
|
size restrictions on those objects:
|
|
|
|
.. literalinclude:: ../../examples/capabilities.py
|
|
:language: python
|