Files
swift/doc/source/api/object_versioning.rst
Tim Burke c7283be4fe Add "history" mode to versioned_writes middleware
This change introduces the concept of a "versioning mode" for
versioned_writes. The following modes are supported:

 * stack

    When deleting, check whether any previous versions exist in the
    versions container. If none is found, the object is deleted. If the
    most-recent version in the versions container is not a delete
    marker, it is copied into the versioned container (overwriting the
    current version if one exists) and then deleted from the versions
    container. This preserves the previous behavior.

    If the most-recent version in the versions container is a delete
    marker and a current version exists in the versioned container, the
    current version is deleted. If the most-recent version in the
    versions container is a delete marker and no current version exists
    in the versioned container, we copy the next-most-recent version
    from the versions container into the versioned container (assuming
    it exists and is not a delete marker) and delete both the
    most-recent version (i.e., the delete marker) and the just-copied
    next-most-recent version from the versions container.

    With this mode, DELETEs to versioned containers "undo" operations
    on containers. Previously this was limited to undoing PUTs, but now
    it will also undo DELETEs performed while in "history" mode.

 * history

    When deleting, check whether a current version exists in the
    versioned container. If one is found, it is copied to the versions
    container. Then an empty "delete marker" object is also put into the
    versions container; this records when the object was deleted.
    Finally, the original current version is deleted from the versioned
    container. As a result, subsequent GETs or HEADs will return a 404,
    and container listings for the versioned container do not include
    the object.

    With this mode, DELETEs to versioned containers behave like DELETEs
    to other containers, but with a history of what has happened.

Clients may specify (via a new X-Versions-Mode header) which mode a
container should use. By default, the existing "stack" mode is used.

Upgrade consideration:
======================

Clients should not use the "history" mode until all proxies in the
cluster have been upgraded. Attempting to use the "history" mode during
a rolling upgrade may result in some requests being served by proxies
running old code (which necessarily uses the "stack" mode), leading to
data loss.

Change-Id: I555dc17fefd0aa9ade681aa156da24e018ebe74b
2016-08-15 21:04:29 -07:00

11 KiB

Object versioning

You can store multiple versions of your content so that you can recover from unintended overwrites. Object versioning is an easy way to implement version control, which you can use with any type of content.

Note

You cannot version a large-object manifest file, but the large-object manifest file can point to versioned segments.

Note

It is strongly recommended that you put non-current objects in a different container than the container where current object versions reside.

To allow object versioning within a cluster, the cloud provider should add the versioned_writes filter to the pipeline and set the allow_versioned_writes option to true in the [filter:versioned_writes] section of the proxy-server configuration file.

The X-Versions-Location header defines the container that holds the non-current versions of your objects. You must UTF-8-encode and then URL-encode the container name before you include it in the X-Versions-Location header. This header enables object versioning for all objects in the container. With a comparable archive container in place, changes to objects in the current container automatically create non-current versions in the archive container.

The X-Versions-Mode header defines the behavior of DELETE requests to objects in the versioned container. In the default stack mode, deleting an object will restore the most-recent version from the archive container, overwriting the curent version. Alternatively you may specify history mode, where deleting an object will copy the current version to the archive then remove it from the current container.

Example Using stack Mode

  1. Create the current container:
# curl -i $publicURL/current -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" -H "X-Versions-Location: archive" -H "X-Versions-Mode: stack"
HTTP/1.1 201 Created
Content-Length: 0
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txb91810fb717347d09eec8-0052e18997
Date: Thu, 23 Jan 2014 21:28:55 GMT
  1. Create the first version of an object in the current container:

    # curl -i $publicURL/current/my_object --data-binary 1 -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token"
    HTTP/1.1 201 Created
    Last-Modified: Thu, 23 Jan 2014 21:31:22 GMT
    Content-Length: 0
    Etag: d41d8cd98f00b204e9800998ecf8427e
    Content-Type: text/html; charset=UTF-8
    X-Trans-Id: tx5992d536a4bd4fec973aa-0052e18a2a
    Date: Thu, 23 Jan 2014 21:31:22 GMT

    Nothing is written to the non-current version container when you initially PUT an object in the current container. However, subsequent PUT requests that edit an object trigger the creation of a version of that object in the archive container.

    These non-current versions are named as follows:

    <length><object_name>/<timestamp>

    Where length is the 3-character, zero-padded hexadecimal character length of the object, <object_name> is the object name, and <timestamp> is the time when the object was initially created as a current version.

  2. Create a second version of the object in the current container:

    # curl -i $publicURL/current/my_object --data-binary 2 -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token"
    HTTP/1.1 201 Created
    Last-Modified: Thu, 23 Jan 2014 21:41:32 GMT
    Content-Length: 0
    Etag: d41d8cd98f00b204e9800998ecf8427e
    Content-Type: text/html; charset=UTF-8
    X-Trans-Id: tx468287ce4fc94eada96ec-0052e18c8c
    Date: Thu, 23 Jan 2014 21:41:32 GMT
  3. Issue a GET request to a versioned object to get the current version of the object. You do not have to do any request redirects or metadata lookups.

    List older versions of the object in the archive container:

    # curl -i $publicURL/archive?prefix=009my_object -X GET -H "X-Auth-Token: $token"
    HTTP/1.1 200 OK
    Content-Length: 30
    X-Container-Object-Count: 1
    Accept-Ranges: bytes
    X-Timestamp: 1390513280.79684
    X-Container-Bytes-Used: 0
    Content-Type: text/plain; charset=utf-8
    X-Trans-Id: tx9a441884997542d3a5868-0052e18d8e
    Date: Thu, 23 Jan 2014 21:45:50 GMT
    
    009my_object/1390512682.92052

    Note

    A POST request to a versioned object updates only the metadata for the object and does not create a new version of the object. New versions are created only when the content of the object changes.

  4. Issue a DELETE request to a versioned object to remove the current version of the object and replace it with the next-most current version in the non-current container.

    # curl -i $publicURL/current/my_object -X DELETE -H "X-Auth-Token: $token"
    HTTP/1.1 204 No Content
    Content-Length: 0
    Content-Type: text/html; charset=UTF-8
    X-Trans-Id: tx006d944e02494e229b8ee-0052e18edd
    Date: Thu, 23 Jan 2014 21:51:25 GMT

    List objects in the archive container to show that the archived object was moved back to the current container:

    # curl -i $publicURL/archive?prefix=009my_object -X GET -H "X-Auth-Token: $token"
    HTTP/1.1 204 No Content
    Content-Length: 0
    X-Container-Object-Count: 0
    Accept-Ranges: bytes
    X-Timestamp: 1390513280.79684
    X-Container-Bytes-Used: 0
    Content-Type: text/html; charset=UTF-8
    X-Trans-Id: tx044f2a05f56f4997af737-0052e18eed
    Date: Thu, 23 Jan 2014 21:51:41 GMT

    This next-most current version carries with it any metadata last set on it. If want to completely remove an object and you have five versions of it, you must DELETE it five times.

Example Using history Mode ----------------------------

  1. Create the current container:
# curl -i $publicURL/current -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" -H "X-Versions-Location: archive" -H "X-Versions-Mode: history"
HTTP/1.1 201 Created
Content-Length: 0
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txb91810fb717347d09eec8-0052e18997
Date: Thu, 23 Jan 2014 21:28:55 GMT
  1. Create the first version of an object in the current container:

    # curl -i $publicURL/current/my_object --data-binary 1 -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token"
    HTTP/1.1 201 Created
    Last-Modified: Thu, 23 Jan 2014 21:31:22 GMT
    Content-Length: 0
    Etag: d41d8cd98f00b204e9800998ecf8427e
    Content-Type: text/html; charset=UTF-8
    X-Trans-Id: tx5992d536a4bd4fec973aa-0052e18a2a
    Date: Thu, 23 Jan 2014 21:31:22 GMT

    Nothing is written to the non-current version container when you initially PUT an object in the current container. However, subsequent PUT requests that edit an object trigger the creation of a version of that object in the archive container.

    These non-current versions are named as follows:

    <length><object_name>/<timestamp>

    Where length is the 3-character, zero-padded hexadecimal character length of the object, <object_name> is the object name, and <timestamp> is the time when the object was initially created as a current version.

  2. Create a second version of the object in the current container:

    # curl -i $publicURL/current/my_object --data-binary 2 -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token"
    HTTP/1.1 201 Created
    Last-Modified: Thu, 23 Jan 2014 21:41:32 GMT
    Content-Length: 0
    Etag: d41d8cd98f00b204e9800998ecf8427e
    Content-Type: text/html; charset=UTF-8
    X-Trans-Id: tx468287ce4fc94eada96ec-0052e18c8c
    Date: Thu, 23 Jan 2014 21:41:32 GMT
  3. Issue a GET request to a versioned object to get the current version of the object. You do not have to do any request redirects or metadata lookups.

    List older versions of the object in the archive container:

    # curl -i $publicURL/archive?prefix=009my_object -X GET -H "X-Auth-Token: $token"
    HTTP/1.1 200 OK
    Content-Length: 30
    X-Container-Object-Count: 1
    Accept-Ranges: bytes
    X-Timestamp: 1390513280.79684
    X-Container-Bytes-Used: 0
    Content-Type: text/plain; charset=utf-8
    X-Trans-Id: tx9a441884997542d3a5868-0052e18d8e
    Date: Thu, 23 Jan 2014 21:45:50 GMT
    
    009my_object/1390512682.92052

    Note

    A POST request to a versioned object updates only the metadata for the object and does not create a new version of the object. New versions are created only when the content of the object changes.

  4. Issue a DELETE request to a versioned object to copy the current version of the object to the archive container then delete it from the current container. Subsequent GET requests to the object in the current container will return 404 Not Found.

    # curl -i $publicURL/current/my_object -X DELETE -H "X-Auth-Token: $token"
    HTTP/1.1 204 No Content
    Content-Length: 0
    Content-Type: text/html; charset=UTF-8
    X-Trans-Id: tx006d944e02494e229b8ee-0052e18edd
    Date: Thu, 23 Jan 2014 21:51:25 GMT

    List older versions of the object in the archive container:

    .. code::

    # curl -i $publicURL/archive?prefix=009my_object -X GET -H "X-Auth-Token: $token"

    HTTP/1.1 200 OK
    Content-Length: 90
    X-Container-Object-Count: 3
    Accept-Ranges: bytes
    X-Timestamp: 1390513280.79684
    X-Container-Bytes-Used: 0
    Content-Type: text/html; charset=UTF-8
    X-Trans-Id: tx044f2a05f56f4997af737-0052e18eed
    Date: Thu, 23 Jan 2014 21:51:41 GMT
    
    009my_object/1390512682.92052
    009my_object/1390512692.23062
    009my_object/1390513885.67732

    In addition to the two previous versions of the object, the archive container has a "delete marker" to record when the object was deleted.

    To permanently delete a previous version, issue a DELETE to the version in the archive container.

Disabling Object Versioning

To disable object versioning for the current container, remove its X-Versions-Location metadata header by sending an empty key value.

# curl -i $publicURL/current -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" -H "X-Versions-Location: "
HTTP/1.1 202 Accepted
Content-Length: 76
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txe2476de217134549996d0-0052e19038
Date: Thu, 23 Jan 2014 21:57:12 GMT

<html><h1>Accepted</h1><p>The request is accepted for processing.</p></html>