python-swiftclient/doc/source/introduction.rst

============
Introduction
============

Where to Start?
~~~~~~~~~~~~~~~

The ``python-swiftclient`` project comprises a command line tool and two
separate APIs for accessing swift programmatically. Choosing the most
appropriate method for a given use case is the first problem a user needs to
solve.

Use Cases
---------

Alongside the command line tool, the ``python-swiftclient`` includes two
levels of API:

* A low level client API that provides simple Python wrappers around the
  various authentication mechanisms and the individual HTTP requests.
* A high level service API that provides methods for performing common
  operations in parallel on a thread pool.

Example use cases:

* Uploading and retrieving data
    Use the command line tool if you are simply uploading and downloading
    files and directories to and from your filesystem. The command line tool
    can be integrated into a shell script to automate tasks.

* Integrating into an automated Python workflow
    Use the ``SwiftService`` API to perform operations offered by the CLI
    if your use case requires integration with a Python-based workflow.
    This method offers greater control and flexibility over individual object
    operations, such as the metadata set on each object. The ``SwiftService``
    class provides methods to perform multiple sets of operations against a
    swift object store using a configurable shared thread pool. A single
    instance of the ``SwiftService`` class can be shared between multiple
    threads in your own code.

* Developing an application in Python to access a swift object store
    Use the ``SwiftService`` API to develop Python applications that use
    swift to store and retrieve objects. A ``SwiftService`` instance provides
    a configurable thread pool for performing all operations supported by the
    CLI.

* Fine-grained control over threading or the requests being performed
    Use the ``Connection`` API if your use case requires fine grained control
    over advanced features or you wish to use your own existing threading
    model. Examples of advanced features requiring the use of the
    ``Connection`` API include creating an SLO manifest that references
    already existing objects, or fine grained control over the query strings
    supplied with each HTTP request.

Important considerations
~~~~~~~~~~~~~~~~~~~~~~~~

This section covers some important considerations, helpful hints, and things to
avoid when integrating an object store into your workflow.

An object store is not a filesystem
-----------------------------------

It cannot be stressed enough that your usage of the object store should reflect
the proper use case, and not treat the storage like a traditional filesystem.
There are two main restrictions to bear in mind when designing an application
that uses an object store:

* You cannot rename objects. Due to fact that the name of an object is one
  of the factors that determines where the object and its replicas are stored,
  renaming would require multiple copies of the data to be moved between
  physical storage devices. If you want to rename an object you must upload
  to the new location, or make a server side copy request to the new location,
  and then delete the original.

* You cannot modify objects. Objects are stored in multiple locations and
  are checked for integrity based on the MD5 sum calculated during
  upload. In order to modify the contents of an object, the entire desired
  contents must be re-uploaded. In certain special cases it is possible to
  work around this restriction using large objects, but no general
  file-like access is available to modify a stored object.

Objects cannot be locked
------------------------

There is no mechanism to perform a combination of reading the
data/metadata from an object and writing an update to that data/metadata in an
atomic way. Any user with access to a container could update the contents or
metadata associated with an object at any time.

Workflows that assume that no updates have been made since the last read of an
object should be discouraged. Enabling a workflow of this type requires an
external object locking mechanism and/or cooperation between all clients
accessing the data.