diff --git a/doc/source/api/authentication.rst b/doc/source/api/authentication.rst new file mode 100644 index 0000000000..12e9bfb6a7 --- /dev/null +++ b/doc/source/api/authentication.rst @@ -0,0 +1,54 @@ +============== +Authentication +============== + +The owner of an Object Storage account controls access to that account +and its containers and objects. An owner is the user who has the +''admin'' role for that tenant. The tenant is also known as the project +or account. As the account owner, you can modify account metadata and +create, modify, and delete containers and objects. + +To identify yourself as the account owner, include an authentication +token in the ''X-Auth-Token'' header in the API request. + +Depending on the token value in the ''X-Auth-Token'' header, one of the +following actions occur: + +- ''X-Auth-Token'' contains the token for the account owner. + + The request is permitted and has full access to make changes to the + account. + +- The ''X-Auth-Token'' header is omitted or it contains a token for a + non-owner or a token that is not valid. + + The request fails with a 401 Unauthorized or 403 Forbidden response. + + You have no access to accounts or containers, unless an access + control list (ACL) explicitly grants access. + + The account owner can grant account and container access to users + through access control lists (ACLs). + +The following list describes the authentication services that you can +use with Object Storage: + +- OpenStack Identity (keystone): For Object Storage, account is synonymous with + project or tenant ID. + +- Tempauth middleware: Object Storage includes this middleware. User and account + management is performed in Object Storage itself. + +- Swauth middleware: Stored in github, this custom middleware is modeled on + Tempauth. Usage is similar to Tempauth. + +- Other custom middleware: Write it yourself to fit your environment. + +Specifically, you use the ''X-Auth-Token'' header to pass an +authentication token to an API request. + +Authentication tokens expire after a time period that the authentication +service defines. When a token expires, use of the token causes requests +to fail with a 401 Unauthorized response. To continue, you must obtain a +new token. + diff --git a/doc/source/api/container_quotas.rst b/doc/source/api/container_quotas.rst new file mode 100644 index 0000000000..a41561274e --- /dev/null +++ b/doc/source/api/container_quotas.rst @@ -0,0 +1,30 @@ +================ +Container quotas +================ + +You can set quotas on the size and number of objects stored in a +container by setting the following metadata: + +- ``X-Container-Meta-Quota-Bytes``. The size, in bytes, of objects that + can be stored in a container. + +- ``X-Container-Meta-Quota-Count``. The number of objects that can be + stored in a container. + +When you exceed a container quota, subsequent requests to create objects +fail with a 413 Request Entity Too Large error. + +The Object Storage system uses an eventual consistency model. When you +create a new object, the container size and object count might not be +immediately updated. Consequently, you might be allowed to create +objects even though you have actually exceeded the quota. + +At some later time, the system updates the container size and object +count to the actual values. At this time, subsequent requests fails. In +addition, if you are currently under the +``X-Container-Meta-Quota-Bytes`` limit and a request uses chunked +transfer encoding, the system cannot know if the request will exceed the +quota so the system allows the request. However, once the quota is +exceeded, any subsequent uploads that use chunked transfer encoding +fail. + diff --git a/doc/source/api/discoverability.rst b/doc/source/api/discoverability.rst new file mode 100644 index 0000000000..56912ca652 --- /dev/null +++ b/doc/source/api/discoverability.rst @@ -0,0 +1,37 @@ +=============== +Discoverability +=============== + +Your Object Storage system might not enable all features that you read about because your service provider chooses which features to enable. + +To discover which features are enabled in your Object Storage system, +use the ``/info`` request. However, your service provider might have +disabled the ``/info`` request, or you might be using an older version +that does not support the ``/info`` request. + +To use the ``/info`` request, send a **GET** request using the ``/info`` +path to the Object Store endpoint as shown in this example: + +.. code:: + + # curl https://storage.clouddrive.com/info + +This example shows a truncated response body: + +.. code:: + + { + "swift":{ + "version":"1.11.0" + }, + "staticweb":{ + + }, + "tempurl":{ + + } + } + +This output shows that the Object Storage system has enabled the static +website and temporary URL features. + diff --git a/doc/source/api/form_post_middleware.rst b/doc/source/api/form_post_middleware.rst new file mode 100644 index 0000000000..482be1c5e5 --- /dev/null +++ b/doc/source/api/form_post_middleware.rst @@ -0,0 +1,214 @@ +==================== +Form POST middleware +==================== + +To discover whether your Object Storage system supports this feature, +check with your service provider or send a **GET** request using the ``/info`` +path. + +You can upload objects directly to the Object Storage system from a +browser by using the form **POST** middleware. This middleware uses +account secret keys to generate a cryptographic signature for the +request. This means that you do not need to send an authentication token +in the ``X-Auth-Token`` header to perform the request. + +The form **POST** middleware uses the same secret keys as the temporary +URL middleware uses. For information about how to set these keys, see account secret keys. + +For information about the form **POST** middleware configuration +options, see `Form +post `__ +in the *OpenStack Configuration Reference*. + +Form POST format +~~~~~~~~~~~~~~~~ + +To upload objects to a cluster, you can use an HTML form **POST** +request. + +The format of the form **POST** request is: + +**Example 1.14. Form POST format** + +.. code:: + + <![CDATA[ +
+ + + + + + +
+ +
+ ]]> + + +**``action="SWIFT_URL``"** + +Set to full URL where the objects are to be uploaded. The names of +uploaded files are appended to the specified *``SWIFT_URL``*. So, you +can upload directly to the root of a container with a URL like: + +.. code:: + + https://swift-cluster.example.com/v1/my_account/container/ + +Optionally, you can include an object prefix to separate uploads, such +as: + +.. code:: + + https://swift-cluster.example.com/v1/my_account/container/OBJECT_PREFIX + + +**method="POST"** + +Must be ``POST``. + + +**enctype="multipart/form-data"** + +Must be ``multipart/form-data``. + + +**name="redirect" value="*``REDIRECT_URL``*\ "** + +Redirects the browser to the *``REDIRECT_URL``* after the upload +completes. The URL has status and message query parameters added to it, +which specify the HTTP status code for the upload and an optional error +message. The 2\ *``nn``* status code indicates success. + +The *``REDIRECT_URL``* can be an empty string. If so, the ``Location`` +response header is not set. + +**name="max\_file\_size" value="*``BYTES``*\ "** + +Required. Indicates the size, in bytes, of the maximum single file +upload. + +**name="max\_file\_count" value= "*``COUNT``*\ "** + +Required. Indicates the maximum number of files that can be uploaded +with the form. + + +**name="expires" value="*``UNIX_TIMESTAMP``*\ "** + +The UNIX timestamp that specifies the time before which the form must be +submitted before it becomes no longer valid. + + +**name="signature" value="*``HMAC``*\ "** + +The HMAC-SHA1 signature of the form. + + +**type="file" name="*``FILE_NAME``*\ "** + +File name of the file to be uploaded. You can include from one to the +``max_file_count`` value of files. + +The file attributes must appear after the other attributes to be +processed correctly. + +If attributes appear after the file attributes, they are not sent with +the sub-request because all attributes in the file cannot be parsed on +the server side unless the whole file is read into memory; the server +does not have enough memory to service these requests. Attributes that +follow the file attributes are ignored. + +Optionally, if you want the uploaded files to be temporary you can set x-delete-at or x-delete-after attributes by adding one of these as a form input: + +..code:: + + + + + +**type= "submit"** + +Must be ``submit``. + +HMAC-SHA1 signature for form POST +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Form **POST** middleware uses an HMAC-SHA1 cryptographic signature. This +signature includes these elements from the form: + +- The path. Starting with ``/v1/`` onwards and including a container + name and, optionally, an object prefix. In `Example 1.15, “HMAC-SHA1 + signature for form + POST” the path is + ``/v1/my_account/container/object_prefix``. Do not URL-encode the + path at this stage. + +- A redirect URL. If there is no redirect URL, use the empty string. + +- Maximum file size. In `Example 1.15, “HMAC-SHA1 signature for form + POST” the + ``max_file_size`` is ``104857600`` bytes. + +- The maximum number of objects to upload. In `Example 1.15, “HMAC-SHA1 + signature for form + POST” ``max_file_count`` is ``10``. + +- Expiry time. In `Example 1.15, “HMAC-SHA1 signature for form + POST” the expiry time + is set to ``600`` seconds into the future. + +- The secret key. Set as the ``X-Account-Meta-Temp-URL-Key`` header + value. + +The following example code generates a signature for use with form +**POST**: + +**Example 1.15. HMAC-SHA1 signature for form POST** + +.. code:: + + import hmac + from hashlib import sha1 + from time import time + path = '/v1/my_account/container/object_prefix' + redirect = 'https://myserver.com/some-page' + max_file_size = 104857600 + max_file_count = 10 + expires = int(time() + 600) + key = 'MYKEY' + hmac_body = '%s\n%s\n%s\n%s\n%s' % (path, redirect, + max_file_size, max_file_count, expires) + signature = hmac.new(key, hmac_body, sha1).hexdigest() + + +For more information, see `RFC 2104: HMAC: Keyed-Hashing for Message +Authentication `__. + +Form POST example +~~~~~~~~~~~~~~~~~ + +The following example shows how to submit a form by using a cURL +command. In this example, the object prefix is ``photos/`` and the file +being uploaded is called ``flower.jpg``. + +This example uses the **swift-form-signature** script to compute the +``expires`` and ``signature`` values. + +.. code:: + + $ bin/swift-form-signature /v1/my_account/container/photos/ https://example.com/done.html 5373952000 1 200 MYKEY + Expires: 1390825338 + Signature: 35129416ebda2f1a21b3c2b8939850dfc63d8f43 + +.. code:: + + $ curl -i https://swift-cluster.example.com/v1/my_account/container/photos/ -X POST \ + -F max_file_size=5373952000 -F max_file_count=1 -F expires=1390825338 \ + -F signature=35129416ebda2f1a21b3c2b8939850dfc63d8f43 \ + -F redirect=https://example.com/done.html \ + -F file=@flower.jpg + diff --git a/doc/source/api/large_objects.rst b/doc/source/api/large_objects.rst new file mode 100644 index 0000000000..c84b53e2f8 --- /dev/null +++ b/doc/source/api/large_objects.rst @@ -0,0 +1,295 @@ +============= +Large objects +============= + +By default, the content of an object cannot be greater than 5 GB. +However, you can use a number of smaller objects to construct a large +object. The large object is comprised of two types of objects: + +- **Segment objects** store the object content. You can divide your + content into segments, and upload each segment into its own segment + object. Segment objects do not have any special features. You create, + update, download, and delete segment objects just as you would normal + objects. + +- A **manifest object** links the segment objects into one logical + large object. When you download a manifest object, Object Storage + concatenates and returns the contents of the segment objects in the + response body of the request. This behavior extends to the response + headers returned by **GET** and **HEAD** requests. The + ``Content-Length`` response header value is the total size of all + segment objects. Object Storage calculates the ``ETag`` response + header value by taking the ``ETag`` value of each segment, + concatenating them together, and returning the MD5 checksum of the + result. The manifest object types are: + + **Static large objects** + The manifest object content is an ordered list of the names of + the segment objects in JSON format. + + **Dynamic large objects** + The manifest object has no content but it has a + ``X-Object-Manifest`` metadata header. The value of this header + is ``{container}/{prefix}``, where ``{container}`` is the name of + the container where the segment objects are stored, and + ``{prefix}`` is a string that all segment objects have in common. + +Note +~~~~ + +If you make a **COPY** request by using a manifest object as the source, +the new object is a normal, and not a segment, object. If the total size +of the source segment objects exceeds 5 GB, the **COPY** request fails. +However, you can make a duplicate of the manifest object and this new +object can be larger than 5 GB. + +Static large objects +~~~~~~~~~~~~~~~~~~~~ + +To create a static large object, divide your content into pieces and +create (upload) a segment object to contain each piece. + +You must record the ``ETag`` response header that the **PUT** operation +returns. Alternatively, you can calculate the MD5 checksum of the +segment prior to uploading and include this in the ``ETag`` request +header. This ensures that the upload cannot corrupt your data. + +List the name of each segment object along with its size and MD5 +checksum in order. + +Create a manifest object. Include the *``?multipart-manifest=put``* +query string at the end of the manifest object name to indicate that +this is a manifest object. + +The body of the **PUT** request on the manifest object comprises a json +list, where each element contains the following attributes: + +- ``path``. The container and object name in the format: + ``{container-name}/{object-name}`` + +- ``etag``. The MD5 checksum of the content of the segment object. This + value must match the ``ETag`` of that object. + +- ``size_bytes``. The size of the segment object. This value must match + the ``Content-Length`` of that object. + +**Example Static large object manifest list** + +This example shows three segment objects. You can use several containers +and the object names do not have to conform to a specific pattern, in +contrast to dynamic large objects. + +.. code:: + + [ + { + "path": "mycontainer/objseg1", + "etag": "0228c7926b8b642dfb29554cd1f00963", + "size_bytes": 1468006 + }, + { + "path": "mycontainer/pseudodir/seg-obj2", + "etag": "5bfc9ea51a00b790717eeb934fb77b9b", + "size_bytes": 1572864 + }, + { + "path": "other-container/seg-final", + "etag": "b9c3da507d2557c1ddc51f27c54bae51", + "size_bytes": 256 + } + ] + +| + +The ``Content-Length`` request header must contain the length of the +json content—not the length of the segment objects. However, after the +**PUT** operation completes, the ``Content-Length`` metadata is set to +the total length of all the object segments. A similar situation applies +to the ``ETag``. If used in the **PUT** operation, it must contain the +MD5 checksum of the json content. The ``ETag`` metadata value is then +set to be the MD5 checksum of the concatenated ``ETag`` values of the +object segments. You can also set the ``Content-Type`` request header +and custom object metadata. + +When the **PUT** operation sees the *``?multipart-manifest=put``* query +parameter, it reads the request body and verifies that each segment +object exists and that the sizes and ETags match. If there is a +mismatch, the **PUT**\ operation fails. + +If everything matches, the manifest object is created. The +``X-Static-Large-Object`` metadata is set to ``true`` indicating that +this is a static object manifest. + +Normally when you perform a **GET** operation on the manifest object, +the response body contains the concatenated content of the segment +objects. To download the manifest list, use the +*``?multipart-manifest=get``* query parameter. The resulting list is not +formatted the same as the manifest you originally used in the **PUT** +operation. + +If you use the **DELETE** operation on a manifest object, the manifest +object is deleted. The segment objects are not affected. However, if you +add the *``?multipart-manifest=delete``* query parameter, the segment +objects are deleted and if all are successfully deleted, the manifest +object is also deleted. + +To change the manifest, use a **PUT** operation with the +*``?multipart-manifest=put``* query parameter. This request creates a +manifest object. You can also update the object metadata in the usual +way. + +Dynamic large objects +~~~~~~~~~~~~~~~~~~~~~ + +You must segment objects that are larger than 5 GB before you can upload +them. You then upload the segment objects like you would any other +object and create a dynamic large manifest object. The manifest object +tells Object Storage how to find the segment objects that comprise the +large object. The segments remain individually addressable, but +retrieving the manifest object streams all the segments concatenated. +There is no limit to the number of segments that can be a part of a +single large object. + +To ensure the download works correctly, you must upload all the object +segments to the same container and ensure that each object name is +prefixed in such a way that it sorts in the order in which it should be +concatenated. You also create and upload a manifest file. The manifest +file is a zero-byte file with the extra ``X-Object-Manifest`` +``{container}/{prefix}`` header, where ``{container}`` is the container +the object segments are in and ``{prefix}`` is the common prefix for all +the segments. You must UTF-8-encode and then URL-encode the container +and common prefix in the ``X-Object-Manifest`` header. + +It is best to upload all the segments first and then create or update +the manifest. With this method, the full object is not available for +downloading until the upload is complete. Also, you can upload a new set +of segments to a second location and update the manifest to point to +this new location. During the upload of the new segments, the original +manifest is still available to download the first set of segments. + +**Example Upload segment of large object request: HTTP** + +.. code:: + + PUT /{api_version}/{account}/{container}/{object} HTTP/1.1 + Host: storage.clouddrive.com + X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb + ETag: 8a964ee2a5e88be344f36c22562a6486 + Content-Length: 1 + X-Object-Meta-PIN: 1234 + + +No response body is returned. A status code of 2\ *``nn``* (between 200 +and 299, inclusive) indicates a successful write; status 411 Length +Required denotes a missing ``Content-Length`` or ``Content-Type`` header +in the request. If the MD5 checksum of the data written to the storage +system does NOT match the (optionally) supplied ETag value, a 422 +Unprocessable Entity response is returned. + +You can continue uploading segments like this example shows, prior to +uploading the manifest. + +**Example Upload next segment of large object request: HTTP** + +.. code:: + + PUT /{api_version}/{account}/{container}/{object} HTTP/1.1 + Host: storage.clouddrive.com + X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb + ETag: 8a964ee2a5e88be344f36c22562a6486 + Content-Length: 1 + X-Object-Meta-PIN: 1234 + + +Next, upload the manifest you created that indicates the container the +object segments reside within. Note that uploading additional segments +after the manifest is created causes the concatenated object to be that +much larger but you do not need to recreate the manifest file for +subsequent additional segments. + +**Example Upload manifest request: HTTP** + +.. code:: + + PUT /{api_version}/{account}/{container}/{object} HTTP/1.1 + Host: storage.clouddrive.com + X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb + Content-Length: 0 + X-Object-Meta-PIN: 1234 + X-Object-Manifest: {container}/{prefix} + + +**Example Upload manifest response: HTTP** + +.. code:: + + [...] + + +The ``Content-Type`` in the response for a **GET** or **HEAD** on the +manifest is the same as the ``Content-Type`` set during the **PUT** +request that created the manifest. You can easily change the +``Content-Type`` by reissuing the **PUT** request. + +Comparison of static and dynamic large objects +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +While static and dynamic objects have similar behavior, here are +their differences: + +**Comparing static and dynamic large objects** + +Static large object: Assured end-to-end integrity. The list of segments +includes the MD5 checksum (``ETag``) of each segment. You cannot upload the +manifest object if the ``ETag`` in the list differs from the uploaded segment +object. If a segment is somehow lost, an attempt to download the manifest +object results in an error. You must upload the segment objects before you +upload the manifest object. You cannot add or remove segment objects from the +manifest. However, you can create a completely new manifest object of the same +name with a different manifest list. + +With static large objects, you can upload new segment objects or remove +existing segments. The names must simply match the ``{prefix}`` supplied +in ``X-Object-Manifest``. The segment objects must be at least 1 MB in size +(by default). The final segment object can be any size. At most, 1000 segments +are supported (by default). The manifest list includes the container name of +each object. Segment objects can be in different containers. + +Dynamic large object: End-to-end integrity is not guaranteed. The eventual +consistency model means that although you have uploaded a segment object, it +might not appear in the container listing until later. If you download the +manifest before it appears in the container, it does not form part of the +content returned in response to a **GET** request. + +With dynamic large objects, you can upload manifest and segment objects +in any order. In case a premature download of the manifest occurs, we +recommend users upload the manifest object after the segments. However, +the system does not enforce the order. Segment objects can be any size. All +segment objects must be in the same container. + +Manifest object metadata +------------------------ + +For static large objects, the object has ``X-Static-Large-Object`` set to +``true``. You do not set this metadata directly. Instead the system sets +it when you **PUT** a static manifest object. + +For dynamic object,s the ``X-Object-Manifest`` value is the +``{container}/{prefix}``, which indicates where the segment objects are +located. You supply this request header in the **PUT** operation. + +Copying the manifest object +--------------------------- + +With static large objects, you include the *``?multipart-manifest=get``* +query string in the **COPY** request. The new object contains the same +manifest as the original. The segment objects are not copied. Instead, +both the original and new manifest objects share the same set of segment +objects. + +When creating dynamic large objects, the **COPY** operation does not create +a manifest object. To duplicate a manifest object, use the **GET** operation +to read the value of ``X-Object-Manifest`` and use this value in the +``X-Object-Manifest`` request header in a **PUT** operation. This creates +a new manifest object that shares the same set of segment objects as the +original manifest object. diff --git a/doc/source/api/object_api_v1_overview.rst b/doc/source/api/object_api_v1_overview.rst new file mode 100644 index 0000000000..6a6f9bc3d7 --- /dev/null +++ b/doc/source/api/object_api_v1_overview.rst @@ -0,0 +1,182 @@ +Object Storage API overview +--------------------------- + +OpenStack Object Storage is an object-based storage system that stores +content and metadata as objects. You create, modify, and get objects and +metadata by using the Object Storage API, which is implemented as a set +of Representational State Transfer (REST) web services. + +For an introduction to OpenStack Object Storage, see `Object +Storage `__ +in the *OpenStack Cloud Administrator Guide*. + +You use the HTTPS (SSL) protocol to interact with Object Storage, and +you use standard HTTP calls to perform API operations. You can also use +language-specific APIs, which use the RESTful API, that make it easier +for you to integrate into your applications. + +To assert your right to access and change data in an account, you +identify yourself to Object Storage by using an authentication token. To +get a token, you present your credentials to an authentication service. +The authentication service returns a token and the URL for the account. +Depending on which authentication service that you use, the URL for the +account appears in: + +- **OpenStack Identity Service**. The URL is defined in the service + catalog. + +- **Tempauth**. The URL is provided in the ``X-Storage-Url`` response + header. + +In both cases, the URL is the full URL and includes the account +resource. + +The Object Storage API supports the standard, non-serialized response +format, which is the default, and both JSON and XML serialized response +formats. + +The Object Storage system organizes data in a hierarchy, as follows: + +- **Account**. Represents the top-level of the hierarchy. + + Your service provider creates your account and you own all resources + in that account. The account defines a namespace for containers. A + container might have the same name in two different accounts. + + In the OpenStack environment, *account* is synonymous with a project + or tenant. + +- **Container**. Defines a namespace for objects. An object with the + same name in two different containers represents two different + objects. You can create any number of containers within an account. + + In addition to containing objects, you can also use the container to + control access to objects by using an access control list (ACL). You + cannot store an ACL with individual objects. + + In addition, you configure and control many other features, such as + object versioning, at the container level. + + You can bulk-delete up to 10,000 containers in a single request. + + You can set a storage policy on a container with predefined names + and definitions from your cloud provider. + +- **Object**. Stores data content, such as documents, images, and so + on. You can also store custom metadata with an object. + + With the Object Storage API, you can: + + - Store an unlimited number of objects. Each object can be as large + as 5 GB, which is the default. You can configure the maximum + object size. + + - Upload and store objects of any size with large object creation. + + - Use cross-origin resource sharing to manage object security. + + - Compress files using content-encoding metadata. + + - Override browser behavior for an object using content-disposition metadata. + + - Schedule objects for deletion. + + - Bulk-delete up to 10,000 objects in a single request. + + - Auto-extract archive files. + + - Generate a URL that provides time-limited **GET** access to an + object. + + - Upload objects directly to the Object Storage system from a + browser by using form **POST** middleware + +The account, container, and object hierarchy affects the way you +interact with the Object Storage API. + +Specifically, the resource path reflects this structure and has this +format: + +.. code:: + + /v1/{account}/{container}/{object} + +For example, for the ``flowers/rose.jpg`` object in the ``images`` +container in the ``12345678912345`` account, the resource path is: + +.. code:: + + /v1/12345678912345/images/flowers/rose.jpg + +Notice that the object name contains the ``/`` character. This slash +does not indicate that Object Storage has a sub-hierarchy called +``flowers`` because containers do not store objects in actual +sub-folders. However, the inclusion of ``/`` or a similar convention +inside object names enables you to create pseudo-hierarchical folders +and directories. + +For example, if the endpoint for Object Storage is +``objects.mycloud.com``, the returned URL is +``https://objects.mycloud.com/v1/12345678912345``. + +To access a container, append the container name to the resource path. + +To access an object, append the container and the object name to the +path. + +If you have a large number of containers or objects, you can use query +parameters to page through large lists of containers or objects. Use the +*``marker``*, *``limit``*, and *``end_marker``* query parameters to +control how many items are returned in a list and where the list starts +or ends. + +Object Storage HTTP requests have the following default constraints. +Your service provider might use different default values. + +==== ============= ===== +Item Maximum value Notes +==== ============= ===== + +Number of HTTP headers 90 + +Length of HTTP headers 4096 bytes + +Length per HTTP request line 8192 bytes + +Length of HTTP request 5 GB + +Length of container names 256 bytes Cannot contain the ``/`` character. + +Length of object names 1024 bytes By default, there are no character restrictions. + +You must UTF-8-encode and then URL-encode container and object names +before you call the API binding. If you use an API binding that performs +the URL-encoding for you, do not URL-encode the names before you call +the API binding. Otherwise, you double-encode these names. Check the +length restrictions against the URL-encoded string. + +The API Reference describes the operations that you can perform with the +Object Storage API: + +- `Storage + accounts `__: + Use to perform account-level tasks. + + Lists containers for a specified account. Creates, updates, and + deletes account metadata. Shows account metadata. + +- `Storage + containers `__: + Use to perform container-level tasks. + + Lists objects in a specified container. Creates, shows details for, + and deletes containers. Creates, updates, shows, and deletes + container metadata. + +- `Storage + objects `__: + Use to perform object-level tasks. + + Creates, replaces, shows details for, and deletes objects. Copies + objects with another object with a new or different name. Updates + object metadata. \ No newline at end of file diff --git a/doc/source/api/object_versioning.rst b/doc/source/api/object_versioning.rst new file mode 100644 index 0000000000..0b3cdbe8c4 --- /dev/null +++ b/doc/source/api/object_versioning.rst @@ -0,0 +1,183 @@ +================= +Object versioning +================= + +You can store multiple versions of your content so that you can recover +from unintended overwrites. Object versioning is an easy way to +implement version control, which you can use with any type of content. + +Note +~~~~ + +You cannot version a large-object manifest file, but the large-object +manifest file can point to versioned segments. + +It is strongly recommended that you put non-current objects in a +different container than the container where current object versions +reside. + +To enable object versioning, the cloud provider sets the +``allow_versions`` option to ``TRUE`` in the container configuration +file. + +The ``X-Versions-Location`` header defines the +container that holds the non-current versions of your objects. You +must UTF-8-encode and then URL-encode the container name before you +include it in the ``X-Versions-Location`` header. This header enables +object versioning for all objects in the container. With a comparable +``archive`` container in place, changes to objects in the ``current`` +container automatically create non-current versions in the ``archive`` +container. + +Here's an example: + +#. Create the ``current`` container: + + .. code:: + + # curl -i $publicURL/current -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" -H "X-Versions-Location: archive" + + .. code:: + + HTTP/1.1 201 Created + Content-Length: 0 + Content-Type: text/html; charset=UTF-8 + X-Trans-Id: txb91810fb717347d09eec8-0052e18997 + Date: Thu, 23 Jan 2014 21:28:55 GMT + +#. Create the first version of an object in the ``current`` container: + + .. code:: + + # curl -i $publicURL/current/my_object --data-binary 1 -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" + + .. code:: + + HTTP/1.1 201 Created + Last-Modified: Thu, 23 Jan 2014 21:31:22 GMT + Content-Length: 0 + Etag: d41d8cd98f00b204e9800998ecf8427e + Content-Type: text/html; charset=UTF-8 + X-Trans-Id: tx5992d536a4bd4fec973aa-0052e18a2a + Date: Thu, 23 Jan 2014 21:31:22 GMT + + Nothing is written to the non-current version container when you + initially **PUT** an object in the ``current`` container. However, + subsequent **PUT** requests that edit an object trigger the creation + of a version of that object in the ``archive`` container. + + These non-current versions are named as follows: + + .. code:: + + + + Where ``length`` is the 3-character, zero-padded hexadecimal + character length of the object, ```` is the object name, + and ```` is the time when the object was initially created + as a current version. + +#. Create a second version of the object in the ``current`` container: + + .. code:: + + # curl -i $publicURL/current/my_object --data-binary 2 -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" + + .. code:: + + HTTP/1.1 201 Created + Last-Modified: Thu, 23 Jan 2014 21:41:32 GMT + Content-Length: 0 + Etag: d41d8cd98f00b204e9800998ecf8427e + Content-Type: text/html; charset=UTF-8 + X-Trans-Id: tx468287ce4fc94eada96ec-0052e18c8c + Date: Thu, 23 Jan 2014 21:41:32 GMT + +#. Issue a **GET** request to a versioned object to get the current + version of the object. You do not have to do any request redirects or + metadata lookups. + + List older versions of the object in the ``archive`` container: + + .. code:: + + # curl -i $publicURL/archive?prefix=009my_object -X GET -H "X-Auth-Token: $token" + + .. code:: + + HTTP/1.1 200 OK + Content-Length: 30 + X-Container-Object-Count: 1 + Accept-Ranges: bytes + X-Timestamp: 1390513280.79684 + X-Container-Bytes-Used: 0 + Content-Type: text/plain; charset=utf-8 + X-Trans-Id: tx9a441884997542d3a5868-0052e18d8e + Date: Thu, 23 Jan 2014 21:45:50 GMT + + 009my_object/1390512682.92052 + +Note +~~~~ + + A **POST** request to a versioned object updates only the metadata + for the object and does not create a new version of the object. New + versions are created only when the content of the object changes. + +#. Issue a **DELETE** request to a versioned object to remove the + current version of the object and replace it with the next-most + current version in the non-current container. + + .. code:: + + # curl -i $publicURL/current/my_object -X DELETE -H "X-Auth-Token: $token" + + .. code:: + + HTTP/1.1 204 No Content + Content-Length: 0 + Content-Type: text/html; charset=UTF-8 + X-Trans-Id: tx006d944e02494e229b8ee-0052e18edd + Date: Thu, 23 Jan 2014 21:51:25 GMT + + List objects in the ``archive`` container to show that the archived + object was moved back to the ``current`` container: + + .. code:: + + # curl -i $publicURL/archive?prefix=009my_object -X GET -H "X-Auth-Token: $token" + + .. code:: + + HTTP/1.1 204 No Content + Content-Length: 0 + X-Container-Object-Count: 0 + Accept-Ranges: bytes + X-Timestamp: 1390513280.79684 + X-Container-Bytes-Used: 0 + Content-Type: text/html; charset=UTF-8 + X-Trans-Id: tx044f2a05f56f4997af737-0052e18eed + Date: Thu, 23 Jan 2014 21:51:41 GMT + + This next-most current version carries with it any metadata last set + on it. If want to completely remove an object and you have five + versions of it, you must **DELETE** it five times. + +#. To disable object versioning for the ``current`` container, remove + its ``X-Versions-Location`` metadata header by sending an empty key + value. + + .. code:: + + # curl -i $publicURL/current -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" -H "X-Versions-Location: " + + .. code:: + + HTTP/1.1 202 Accepted + Content-Length: 76 + Content-Type: text/html; charset=UTF-8 + X-Trans-Id: txe2476de217134549996d0-0052e19038 + Date: Thu, 23 Jan 2014 21:57:12 GMT + +

Accepted

The request is accepted for processing.

+ diff --git a/doc/source/api/temporary_url_middleware.rst b/doc/source/api/temporary_url_middleware.rst new file mode 100644 index 0000000000..75b909abf2 --- /dev/null +++ b/doc/source/api/temporary_url_middleware.rst @@ -0,0 +1,174 @@ +======================== +Temporary URL middleware +======================== + +To discover whether your Object Storage system supports this feature, +check with your service provider or send a **GET** request using the ``/info`` +path. + +A temporary URL gives users temporary access to objects. For example, a +website might want to provide a link to download a large object in +Object Storage, but the Object Storage account has no public access. The +website can generate a URL that provides time-limited **GET** access to +the object. When the web browser user clicks on the link, the browser +downloads the object directly from Object Storage, eliminating the need +for the website to act as a proxy for the request. + +Ask your cloud administrator to enable the temporary URL feature. For +information, see `Temporary +URL `__ +in the *OpenStack Configuration Reference*. + +Note +~~~~ + +To use **POST** requests to upload objects to specific Object Storage +locations, use form **POST** instead of temporary URL middleware. See +`Form POST `__ +in the *OpenStack Configuration Reference*. + +Temporary URL format +~~~~~~~~~~~~~~~~~~~~ + +A temporary URL is comprised of the URL for an object with added query +parameters: + +**Example Temporary URL format** + +.. code:: + + https://swift-cluster.example.com/v1/my_account/container/object + ?temp_url_sig=da39a3ee5e6b4b0d3255bfef95601890afd80709 + &temp_url_expires=1323479485 + &filename=My+Test+File.pdf + +The example shows these elements: + + +**Object URL**: Required. The full path URL to the object. + +**temp\_url\_sig**: Required. An HMAC-SHA1 cryptographic signature that defines +the allowed HTTP method, expiration date, full path to the object, and the +secret key for the temporary URL. + +**temp\_url\_expires**: Required. An expiration date as a UNIX Epoch timestamp, +which is an integer value. For example, ``1390852007`` represents +``Mon, 27 Jan 2014 19:46:47 GMT``. + +For more information, see `Epoch & Unix Timestamp Conversion +Tools `__. + +**filename**: Optional. Overrides the default file name. Object Storage +generates a default file name for **GET** temporary URLs that is based on the +object name. Object Storage returns this value in the ``Content-Disposition`` +response header. Browsers can interpret this file name value as a file +attachment to be saved. + +Account secret keys +~~~~~~~~~~~~~~~~~~~ + +Object Storage supports up to two secret keys. You set secret keys at +the account level. + +To set these keys, set one or both of the following request headers to +arbitrary values: + +- ``X-Account-Meta-Temp-URL-Key`` + +- ``X-Account-Meta-Temp-URL-Key-2`` + +The arbitrary values serve as the secret keys. + +Object Storage checks signatures against both keys, if present, to +enable key rotation without invalidating existing temporary URLs. + +For example, use the **swift post** command to set the secret key to +*``MYKEY``*: + +.. code:: + + $ swift post -m "Temp-URL-Key:MYKEY" + +Note +~~~~ + +Changing these headers invalidates any previously generated temporary +URLs within 60 seconds, which is the memcache time for the key. + +HMAC-SHA1 signature for temporary URLs +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Temporary URL middleware uses an HMAC-SHA1 cryptographic signature. This +signature includes these elements: + +- The allowed method. Typically, **GET** or **PUT**. + +- Expiry time. In the example for the HMAC-SHA1 signature for temporary + URLs below, the expiry time is set to ``86400`` seconds (or 1 day) + into the future. + +- The path. Starting with ``/v1/`` onwards and including a container + name and object. In the example below, the path is + ``/v1/my_account/container/object``. Do not URL-encode the path at + this stage. + +- The secret key. Set as the ``X-Account-Meta-Temp-URL-Key`` header + value. + +This sample Python code shows how to compute a signature for use with +temporary URLs: + +**Example HMAC-SHA1 signature for temporary URLs** + +.. code:: + + import hmac + from hashlib import sha1 + from time import time + method = 'GET' + duration_in_seconds = 60*60*24 + expires = int(time() + duration_in_seconds) + path = '/v1/my_account/container/object' + key = 'MYKEY' + hmac_body = '%s\n%s\n%s' % (method, expires, path) + signature = hmac.new(key, hmac_body, sha1).hexdigest() + + +Do not URL-encode the path when you generate the HMAC-SHA1 signature. +However, when you make the actual HTTP request, you should properly +URL-encode the URL. + +The *``MYKEY``* value is the value you set in the +``X-Account-Meta-Temp-URL-Key`` request header on the account. + +For more information, see `RFC 2104: HMAC: Keyed-Hashing for Message +Authentication `__. + +swift-temp-url script +~~~~~~~~~~~~~~~~~~~~~ + +Object Storage provides the **swift-temp-url** script that +auto-generates the *``temp_url_sig``* and *``temp_url_expires``* query +parameters. For example, you might run this command: + +.. code:: + + $ bin/swift-temp-url GET 3600 /v1/my_account/container/object MYKEY + +This command returns the path: + +.. code:: + + /v1/my_account/container/object + ?temp_url_sig=5c4cc8886f36a9d0919d708ade98bf0cc71c9e91 + &temp_url_expires=1374497657 + +To create the temporary URL, prefix this path with the Object Storage +storage host name. For example, prefix the path with +``https://swift-cluster.example.com``, as follows: + +.. code:: + + https://swift-cluster.example.com/v1/my_account/container/object + ?temp_url_sig=5c4cc8886f36a9d0919d708ade98bf0cc71c9e91 + &temp_url_expires=1374497657 diff --git a/doc/source/api/use_content-encoding_metadata.rst b/doc/source/api/use_content-encoding_metadata.rst new file mode 100644 index 0000000000..c12175ab16 --- /dev/null +++ b/doc/source/api/use_content-encoding_metadata.rst @@ -0,0 +1,22 @@ +============================= +Use Content-Encoding metadata +============================= + +When you create an object or update its metadata, you can optionally set +the ``Content-Encoding`` metadata. This metadata enables you to indicate +that the object content is compressed without losing the identity of the +underlying media type (``Content-Type``) of the file, such as a video. + +**Example Content-Encoding header request: HTTP** + +This example assigns an attachment type to the ``Content-Encoding`` +header that indicates how the file is downloaded: + +.. code:: + + PUT //// HTTP/1.1 + Host: storage.clouddrive.com + X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb + Content-Type: video/mp4 + Content-Encoding: gzip + diff --git a/doc/source/api/use_the_content-disposition_metadata.rst b/doc/source/api/use_the_content-disposition_metadata.rst new file mode 100644 index 0000000000..17c671f27f --- /dev/null +++ b/doc/source/api/use_the_content-disposition_metadata.rst @@ -0,0 +1,30 @@ +==================================== +Use the Content-Disposition metadata +==================================== + +To override the default behavior for a browser, use the +``Content-Disposition`` header to specify the override behavior and +assign this header to an object. For example, this header might specify +that the browser use a download program to save this file rather than +show the file, which is the default. + +**Example Override browser default behavior request: HTTP** + +This example assigns an attachment type to the ``Content-Disposition`` +header. This attachment type indicates that the file is to be downloaded +as ``goodbye.txt``: + +.. code:: + + # curl -i $publicURL/marktwain/goodbye -X POST -H "X-Auth-Token: $token" -H "Content-Length: 14" -H "Content-Type: application/octet-stream" -H "Content-Disposition: attachment; filename=goodbye.txt" + +.. code:: + + HTTP/1.1 202 Accepted + Content-Length: 76 + Content-Type: text/html; charset=UTF-8 + X-Trans-Id: txa9b5e57d7f354d7ea9f57-0052e17e13 + Date: Thu, 23 Jan 2014 20:39:47 GMT + +

Accepted

The request is accepted for processing.

+ diff --git a/doc/source/index.rst b/doc/source/index.rst index 3180a60de0..db120c45c0 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -42,7 +42,7 @@ Overview and Concepts .. toctree:: :maxdepth: 1 - Swift's API docs + api/object_api_v1_overview overview_architecture overview_ring overview_policies @@ -84,6 +84,24 @@ Administrator Documentation replication_network logs +Object Storage v1 REST API Documentation +======================================== + +.. toctree:: + :maxdepth: 1 + + api/object_api_v1_overview.rst + api/discoverability.rst + api/authentication.rst + api/container_quotas.rst + api/object_versioning.rst + api/large_objects.rst + api/temporary_url_middleware.rst + api/form_post_middleware.rst + api/use_content-encoding_metadata.rst + api/use_the_content-disposition_metadata.rst + + Source Documentation ====================