Merge "Remove bogus responses for dynamic large objects"
This commit is contained in:
Jenkins
Gerrit Code Review
commit
60e524d82e
|
|
@ -43,11 +43,10 @@ format="SVG" scale="60"/>
|
|||
interface. There are also language-specific APIs that utilize
|
||||
the RESTful API but make it much easier for developers to
|
||||
integrate into their applications.</para>
|
||||
<para>For more details on the OpenStack Object Storage service,
|
||||
please refer to <link
|
||||
<para>For more details about the OpenStack Object Storage service,
|
||||
see <link
|
||||
xlink:href="http://docs.openstack.org/developer/swift/"
|
||||
>http://docs.openstack.org/developer/swift/</link>
|
||||
</para>
|
||||
>http://docs.openstack.org/developer/swift/</link>.</para>
|
||||
<para>We welcome feedback, comments, and bug reports at <link
|
||||
xlink:href="http://bugs.launchpad.net/swift"
|
||||
>http://bugs.launchpad.net/swift</link>.</para>
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
PUT /<api version>/<account>/<container>/<object> HTTP/1.1
|
||||
PUT /{api_version}/{account}/{container}/{object} HTTP/1.1
|
||||
Host: storage.clouddrive.com
|
||||
X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
|
||||
ETag: 8a964ee2a5e88be344f36c22562a6486
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
PUT /<api version>/<account>/<container>/<object> HTTP/1.1
|
||||
PUT /{api_version}/{account}/{container}/{object} HTTP/1.1
|
||||
Host: storage.clouddrive.com
|
||||
X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
|
||||
ETag: 8a964ee2a5e88be344f36c22562a6486
|
||||
|
|
|
|||
|
|
@ -1,17 +1,17 @@
|
|||
[
|
||||
{
|
||||
"path": "mycontainer/objseg1",
|
||||
"etag": "0228c7926b8b642dfb29554cd1f00963",
|
||||
"size_bytes": 1468006
|
||||
},
|
||||
{
|
||||
"path": "mycontainer/pseudodir/seg-obj2",
|
||||
"etag": "5bfc9ea51a00b790717eeb934fb77b9b",
|
||||
"size_bytes": 1572864
|
||||
},
|
||||
{
|
||||
"path": "other-container/seg-final",
|
||||
"etag": "b9c3da507d2557c1ddc51f27c54bae51",
|
||||
"size_bytes": 256
|
||||
}
|
||||
]
|
||||
{
|
||||
"path":"mycontainer/objseg1",
|
||||
"etag":"0228c7926b8b642dfb29554cd1f00963",
|
||||
"size_bytes":1468006
|
||||
},
|
||||
{
|
||||
"path":"mycontainer/pseudodir/seg-obj2",
|
||||
"etag":"5bfc9ea51a00b790717eeb934fb77b9b",
|
||||
"size_bytes":1572864
|
||||
},
|
||||
{
|
||||
"path":"other-container/seg-final",
|
||||
"etag":"b9c3da507d2557c1ddc51f27c54bae51",
|
||||
"size_bytes":256
|
||||
}
|
||||
]
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
PUT /<api version>/<account>/<container>/<object> HTTP/1.1
|
||||
PUT /{api_version}/{account}/{container}/{object} HTTP/1.1
|
||||
Host: storage.clouddrive.com
|
||||
X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
|
||||
Content-Length: 0
|
||||
X-Object-Meta-PIN: 1234
|
||||
X-Object-Manifest: container/object/segments
|
||||
X-Object-Manifest: {container}/{prefix}
|
||||
|
|
@ -26,8 +26,7 @@
|
|||
</listitem>
|
||||
<listitem>
|
||||
<para><literal>X-Container-Meta-Quota-Count</literal>. The
|
||||
number of objects that can be stored in a container.
|
||||
</para>
|
||||
number of objects that can be stored in a container.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
<para>When you exceed a container quota, subsequent requests to
|
||||
|
|
|
|||
|
|
@ -40,8 +40,7 @@
|
|||
</listitem>
|
||||
<listitem>
|
||||
<para>The destination container can be a source container
|
||||
for synchronization for another destination container.
|
||||
</para>
|
||||
for synchronization for another destination container.</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para>The destination container can be the original source
|
||||
|
|
|
|||
|
|
@ -18,116 +18,274 @@
|
|||
xml:id="large-object-creation">
|
||||
<title>Large objects</title>
|
||||
<para>By default, the content of an object cannot be greater than
|
||||
5 GB. However, you can use segment objects and manifest
|
||||
objects to store more content.</para>
|
||||
<section xml:id="segment-objects">
|
||||
<title>Segment objects</title>
|
||||
<para>You can divide your content into segments, and upload
|
||||
each segment into its own segment object. Segment objects
|
||||
do not have any special features. You create, update,
|
||||
download, and delete segment objects just as you would
|
||||
normal objects.</para>
|
||||
</section>
|
||||
<section xml:id="manifest-objects">
|
||||
<title>Manifest objects</title>
|
||||
<para>A manifest object points to segment objects. When you
|
||||
download a manifest object, Object Storage concatenates
|
||||
the contents of the segment objects and returns this in
|
||||
the response body of the request.</para>
|
||||
<para>This behavior extends to the response headers returned
|
||||
by &GET; and &HEAD; requests. The
|
||||
<literal>Content-Length</literal> response header
|
||||
value is the total size of all segment objects. Object
|
||||
Storage calculates the <literal>ETag</literal> response
|
||||
header value by taking the <literal>ETag</literal> value
|
||||
of each segment, concatenating them together, and
|
||||
returning the MD5 checksum of the result.</para>
|
||||
<note>
|
||||
<para>If you make a © request by using a manifest
|
||||
object as the source, the new object is a normal, and
|
||||
not a segment, object. If the total size of the source
|
||||
segment objects exceeds 5 GB, the © request
|
||||
fails. However, you can make a duplicate of the
|
||||
manifest object and this new object can be larger than
|
||||
5 GB.</para>
|
||||
</note>
|
||||
<para>The manifest object types are:</para>
|
||||
5 GB. However, you can use a number of smaller objects to
|
||||
construct a large object. The large object is comprised of two
|
||||
types of objects:</para>
|
||||
<itemizedlist>
|
||||
<listitem>
|
||||
<para><emphasis role="bold">Segment objects</emphasis>
|
||||
store the object content. You can divide your content
|
||||
into segments, and upload each segment into its own
|
||||
segment object. Segment objects do not have any
|
||||
special features. You create, update, download, and
|
||||
delete segment objects just as you would normal
|
||||
objects.</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para>A <emphasis role="bold">manifest object</emphasis>
|
||||
links the segment objects into one logical large
|
||||
object. When you download a manifest object, Object
|
||||
Storage concatenates and returns the contents of the
|
||||
segment objects in the response body of the request.
|
||||
This behavior extends to the response headers returned
|
||||
by &GET; and &HEAD; requests. The
|
||||
<literal>Content-Length</literal> response header
|
||||
value is the total size of all segment objects. Object
|
||||
Storage calculates the <literal>ETag</literal>
|
||||
response header value by taking the
|
||||
<literal>ETag</literal> value of each segment,
|
||||
concatenating them together, and returning the MD5
|
||||
checksum of the result. The manifest object types
|
||||
are:</para>
|
||||
<variablelist wordsize="10">
|
||||
<varlistentry>
|
||||
<term><emphasis role="bold">Static large
|
||||
objects</emphasis></term>
|
||||
<listitem>
|
||||
<para>The manifest object content is an
|
||||
ordered list of the names of the segment
|
||||
objects in JSON format. See <xref
|
||||
linkend="static-large-objects"
|
||||
/>.</para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
<varlistentry>
|
||||
<term><emphasis role="bold">Dynamic large
|
||||
objects</emphasis></term>
|
||||
<listitem>
|
||||
<para>The manifest object has no content but
|
||||
it has a
|
||||
<literal>X-Object-Manifest</literal>
|
||||
metadata header. The value of this header
|
||||
is
|
||||
<literal>{container}/{prefix}</literal>,
|
||||
where <literal>{container}</literal> is
|
||||
the name of the container where the
|
||||
segment objects are stored, and
|
||||
<literal>{prefix}</literal> is a
|
||||
string that all segment objects have in
|
||||
common. See <xref
|
||||
linkend="dynamic-large-object-creation"
|
||||
/>.</para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
</variablelist>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
<note>
|
||||
<para>If you make a © request by using a manifest object
|
||||
as the source, the new object is a normal, and not a
|
||||
segment, object. If the total size of the source segment
|
||||
objects exceeds 5 GB, the © request fails.
|
||||
However, you can make a duplicate of the manifest object
|
||||
and this new object can be larger than 5 GB.</para>
|
||||
</note>
|
||||
<section xml:id="static-large-objects">
|
||||
<title>Static large objects</title>
|
||||
<para>To create a static large object, divide your content
|
||||
into pieces and create (upload) a segment object to
|
||||
contain each piece.</para>
|
||||
<para>You must record the <literal>ETag</literal> response
|
||||
header that the &PUT; operation returns. Alternatively,
|
||||
you can calculate the MD5 checksum of the segment prior to
|
||||
uploading and include this in the <literal>ETag</literal>
|
||||
request header. This ensures that the upload cannot
|
||||
corrupt your data.</para>
|
||||
<para>List the name of each segment object along with its size
|
||||
and MD5 checksum in order.</para>
|
||||
<para>Create a manifest object. Include the
|
||||
<parameter>?multipart-manifest=put</parameter> query
|
||||
string at the end of the manifest object name to indicate
|
||||
that this is a manifest object.</para>
|
||||
<para>The body of the &PUT; request on the manifest object
|
||||
comprises a json list, where each element contains the
|
||||
following attributes:</para>
|
||||
<itemizedlist>
|
||||
<listitem>
|
||||
<para><emphasis role="bold">Static large
|
||||
objects</emphasis>. The manifest object
|
||||
content is an ordered list of the names of the
|
||||
segment objects in JSON format.</para>
|
||||
<para><code>path</code>. The container and object name
|
||||
in the format:
|
||||
<code>{container-name}/{object-name}</code></para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para><emphasis role="bold">Dynamic large
|
||||
objects</emphasis>. The manifest object has no
|
||||
content.</para>
|
||||
<para>However, it has
|
||||
<literal>X-Object-Manifest</literal> metadata
|
||||
header. The value of this header is
|
||||
<literal><container>/<prefix></literal>,
|
||||
where <literal><container></literal> is the
|
||||
name of the container where the segment objects
|
||||
are stored, and <literal><prefix></literal> is
|
||||
a string that all segment objects have in
|
||||
common.</para>
|
||||
<para><code>etag</code>. The MD5 checksum of the
|
||||
content of the segment object. This value must
|
||||
match the <literal>ETag</literal> of that
|
||||
object.</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para><code>size_bytes</code>. The size of the segment
|
||||
object. This value must match the
|
||||
<literal>Content-Length</literal> of that
|
||||
object.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
<para>While both types of manifest objects have similar
|
||||
behavior, the following table describes their
|
||||
differences:</para>
|
||||
<example>
|
||||
<title>Static large object manifest list</title>
|
||||
<para>This example shows three segment objects. You can
|
||||
use several containers and the object names do not
|
||||
have to conform to a specific pattern, in contrast to
|
||||
dynamic large objects.</para>
|
||||
<literallayout class="monospaced"><xi:include href="samples/slo-manifest-example.txt" parse="text"/></literallayout>
|
||||
</example>
|
||||
<para>The <literal>Content-Length</literal> request header
|
||||
must contain the length of the json content—not the
|
||||
length of the segment objects. However, after the &PUT;
|
||||
operation completes, the <literal>Content-Length</literal>
|
||||
metadata is set to the total length of all the object
|
||||
segments. A similar situation applies to the
|
||||
<literal>ETag</literal>. If used in the &PUT;
|
||||
operation, it must contain the MD5 checksum of the json
|
||||
content. The <literal>ETag</literal> metadata value is
|
||||
then set to be the MD5 checksum of the concatenated
|
||||
<literal>ETag</literal> values of the object segments.
|
||||
You can also set the <literal>Content-Type</literal>
|
||||
request header and custom object metadata.</para>
|
||||
<para>When the &PUT; operation sees the
|
||||
<parameter>?multipart-manifest=put</parameter> query
|
||||
parameter, it reads the request body and verifies that
|
||||
each segment object exists and that the sizes and ETags
|
||||
match. If there is a mismatch, the &PUT;operation
|
||||
fails.</para>
|
||||
<para>If everything matches, the manifest object is created.
|
||||
The <literal>X-Static-Large-Object</literal> metadata is
|
||||
set to <literal>true</literal> indicating that this is a
|
||||
static object manifest.</para>
|
||||
<para>Normally when you perform a &GET; operation on the
|
||||
manifest object, the response body contains the
|
||||
concatenated content of the segment objects. To download
|
||||
the manifest list, use the
|
||||
<parameter>?multipart-manifest=get</parameter> query
|
||||
parameter. The resulting list is not formatted the same as
|
||||
the manifest you originally used in the &PUT;
|
||||
operation.</para>
|
||||
<para>If you use the &DELETE; operation on a manifest object,
|
||||
the manifest object is deleted. The segment objects are
|
||||
not affected. However, if you add the
|
||||
<parameter>?multipart-manifest=delete</parameter>
|
||||
query parameter, the segment objects are deleted and if
|
||||
all are successfully deleted, the manifest object is also
|
||||
deleted.</para>
|
||||
<para>To change the manifest, use a &PUT; operation with the
|
||||
<parameter>?multipart-manifest=put</parameter> query
|
||||
parameter. This request creates a manifest object. You can
|
||||
also update the object metadata in the usual way.</para>
|
||||
</section>
|
||||
<section xml:id="dynamic-large-object-creation">
|
||||
<title>Dynamic large objects</title>
|
||||
<para>You must segment objects that are larger than 5 GB
|
||||
before you can upload them. You then upload the segment
|
||||
objects like you would any other object and create a
|
||||
dynamic large manifest object. The manifest object tells
|
||||
Object Storage how to find the segment objects that
|
||||
comprise the large object. The segments remain
|
||||
individually addressable, but retrieving the manifest
|
||||
object streams all the segments concatenated. There is no
|
||||
limit to the number of segments that can be a part of a
|
||||
single large object.</para>
|
||||
<para>To ensure the download works correctly, you must upload
|
||||
all the object segments to the same container and ensure
|
||||
that each object name is prefixed in such a way that it
|
||||
sorts in the order in which it should be concatenated. You
|
||||
also create and upload a manifest file. The manifest file
|
||||
is a zero-byte file with the extra
|
||||
<literal>X-Object-Manifest</literal>
|
||||
<code>{container}/{prefix}</code> header, where
|
||||
<code>{container}</code> is the container the object
|
||||
segments are in and <code>{prefix}</code> is the common
|
||||
prefix for all the segments. You must UTF-8-encode and
|
||||
then URL-encode the container and common prefix in the
|
||||
<literal>X-Object-Manifest</literal> header.</para>
|
||||
<para>It is best to upload all the segments first and then
|
||||
create or update the manifest. With this method, the full
|
||||
object is not available for downloading until the upload
|
||||
is complete. Also, you can upload a new set of segments to
|
||||
a second location and update the manifest to point to this
|
||||
new location. During the upload of the new segments, the
|
||||
original manifest is still available to download the first
|
||||
set of segments.</para>
|
||||
<example>
|
||||
<title>Upload segment of large object request:
|
||||
HTTP</title>
|
||||
<literallayout class="monospaced"><xi:include href="samples/large-object-upload-segment-req.txt" parse="text"/></literallayout>
|
||||
</example>
|
||||
<para>No response body is returned. A status code of
|
||||
<returnvalue>2<replaceable>nn</replaceable></returnvalue>
|
||||
(between 200 and 299, inclusive) indicates a successful
|
||||
write; status <errorcode>411</errorcode>
|
||||
<errortext>Length Required</errortext> denotes a missing
|
||||
<literal>Content-Length</literal> or
|
||||
<literal>Content-Type</literal> header in the request.
|
||||
If the MD5 checksum of the data written to the storage
|
||||
system does NOT match the (optionally) supplied ETag
|
||||
value, a <errorcode>422</errorcode>
|
||||
<errortext>Unprocessable Entity</errortext> response is
|
||||
returned.</para>
|
||||
<para>You can continue uploading segments like this example
|
||||
shows, prior to uploading the manifest.</para>
|
||||
<example>
|
||||
<title>Upload next segment of large object request:
|
||||
HTTP</title>
|
||||
<literallayout class="monospaced"><xi:include href="samples/large-object-upload-next-segment-req.txt" parse="text"/></literallayout>
|
||||
</example>
|
||||
<para>Next, upload the manifest you created that indicates the
|
||||
container the object segments reside within. Note that
|
||||
uploading additional segments after the manifest is
|
||||
created causes the concatenated object to be that much
|
||||
larger but you do not need to recreate the manifest file
|
||||
for subsequent additional segments.</para>
|
||||
<example>
|
||||
<title>Upload manifest request: HTTP</title>
|
||||
<literallayout class="monospaced"><xi:include href="samples/upload-manifest-req.txt" parse="text"/></literallayout>
|
||||
</example>
|
||||
<example>
|
||||
<title>Upload manifest response: HTTP</title>
|
||||
<literallayout class="monospaced"><xi:include href="samples/upload-manifest-resp.txt" parse="text"/></literallayout>
|
||||
</example>
|
||||
<para>The <literal>Content-Type</literal> in the response for
|
||||
a &GET; or &HEAD; on the manifest is the same as the
|
||||
<literal>Content-Type</literal> set during the &PUT;
|
||||
request that created the manifest. You can easily change
|
||||
the <literal>Content-Type</literal> by reissuing the &PUT;
|
||||
request.</para>
|
||||
</section>
|
||||
<section xml:id="comparison_dynamic_static_objects">
|
||||
<title>Comparison of static and dynamic large objects</title>
|
||||
<para>While static and dynamic objects have similar behavior,
|
||||
this table describes their differences:</para>
|
||||
<table rules="all">
|
||||
<caption>Static and dynamic large objects</caption>
|
||||
<col width="20%"/>
|
||||
<col width="40%"/>
|
||||
<col width="40%"/>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Object type</th>
|
||||
<th>End-to-end integrity</th>
|
||||
<th>Upload order</th>
|
||||
<th>Removal or addition of segment objects</th>
|
||||
<th>Segment object size and number</th>
|
||||
<th>Segment object container name</th>
|
||||
<th>Manifest Object Metadata</th>
|
||||
<th>Making a copy of the manifest object</th>
|
||||
<td/>
|
||||
<th>Static large object</th>
|
||||
<th>Dynamic large object</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><para>Static large object</para></td>
|
||||
<th>End-to-end integrity</th>
|
||||
<td><para>Assured. The list of segments includes
|
||||
the MD5 checksum (<literal>ETag</literal>)
|
||||
of each segment. You cannot upload the
|
||||
manifest object if the
|
||||
<literal>ETag</literal> in the list
|
||||
differs from the segment object already
|
||||
uploaded. If a segment is somehow lost, an
|
||||
attempt to download the manifest object
|
||||
results in an error.</para></td>
|
||||
<td><para>The segment objects must be uploaded
|
||||
before the manifest object.</para></td>
|
||||
<td><para>You cannot add or remove segment objects
|
||||
from the manifest. However, you can create
|
||||
a completely new manifest object of the
|
||||
same name with a different manifest
|
||||
list.</para></td>
|
||||
<td><para>Segment objects must be at least 1 MB in
|
||||
size (by default). The final segment
|
||||
object can be any size. At most 1000
|
||||
segments are supported (by
|
||||
default).</para></td>
|
||||
<td><para>The manifest list includes the container
|
||||
name of each object. Segment objects can
|
||||
be in different containers.</para></td>
|
||||
<td><para>The object has
|
||||
<literal>X-Static-Large-Object</literal>
|
||||
set to <literal>true</literal>. You do not
|
||||
set this metadata directly. Instead the
|
||||
system sets it when you &PUT; a static
|
||||
manifest object.</para></td>
|
||||
<td><para/></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><para>Dynamic large object</para></td>
|
||||
differs from the uploaded segment object.
|
||||
If a segment is somehow lost, an attempt
|
||||
to download the manifest object results in
|
||||
an error.</para></td>
|
||||
<td><para>Not guaranteed. The eventual consistency
|
||||
model means that although you have
|
||||
uploaded a segment object, it might not
|
||||
|
|
@ -136,28 +294,77 @@
|
|||
it appears in the container, it does not
|
||||
form part of the content returned in
|
||||
response to a &GET; request.</para></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Upload order</th>
|
||||
<td><para>You must upload the segment objects
|
||||
before you upload the manifest
|
||||
object.</para></td>
|
||||
<td><para>You can upload manifest and segment
|
||||
objects in any order. You are recommended
|
||||
to upload the manifest object after the
|
||||
segments in case a premature download of
|
||||
the manifest occurs. However, this is not
|
||||
enforced.</para></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Removal or addition of segment objects</th>
|
||||
<td><para>You cannot add or remove segment objects
|
||||
from the manifest. However, you can create
|
||||
a completely new manifest object of the
|
||||
same name with a different manifest
|
||||
list.</para></td>
|
||||
<td><para>You can upload new segment objects or
|
||||
remove existing segments. The names must
|
||||
simply match the
|
||||
<literal><prefix></literal>
|
||||
supplied in
|
||||
<literal>X-Object-Manifest</literal>.</para></td>
|
||||
<td><para>Segment objects can be of any
|
||||
size.</para></td>
|
||||
<literal>{prefix}</literal> supplied
|
||||
in
|
||||
<literal>X-Object-Manifest</literal>.</para></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Segment object size and number</th>
|
||||
<td><para>Segment objects must be at least
|
||||
1 MB in size (by default). The final
|
||||
segment object can be any size. At most,
|
||||
1000 segments are supported (by
|
||||
default).</para></td>
|
||||
<td><para>Segment objects can be any
|
||||
size.</para></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Segment object container name</th>
|
||||
<td><para>The manifest list includes the container
|
||||
name of each object. Segment objects can
|
||||
be in different containers.</para></td>
|
||||
<td><para>All segment objects must be in the same
|
||||
container</para></td>
|
||||
container.</para></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Manifest object metadata</th>
|
||||
<td><para>The object has
|
||||
<literal>X-Static-Large-Object</literal>
|
||||
set to <literal>true</literal>. You do not
|
||||
set this metadata directly. Instead the
|
||||
system sets it when you &PUT; a static
|
||||
manifest object.</para></td>
|
||||
<td><para>The <literal>X-Object-Manifest</literal>
|
||||
value is the
|
||||
<literal><container>/<prefix></literal>
|
||||
indicating where the segment objects are
|
||||
located. You supply this request header in
|
||||
the &PUT; operation</para></td>
|
||||
<literal>{container}/{prefix}</literal>,
|
||||
which indicates where the segment objects
|
||||
are located. You supply this request
|
||||
header in the &PUT; operation.</para></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Copying the manifest object</th>
|
||||
<td><para>Include the
|
||||
<parameter>?multipart-manifest=get</parameter>
|
||||
query string in the © request. The
|
||||
new object contains the same manifest as
|
||||
the original. The segment objects are not
|
||||
copied. Instead, both the original and new
|
||||
manifest objects share the same set of
|
||||
segment objects.</para>
|
||||
</td>
|
||||
<td><para>The © operation does not create a
|
||||
manifest object. To duplicate a manifest
|
||||
object, use the &GET; operation to read
|
||||
|
|
@ -172,198 +379,5 @@
|
|||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<section xml:id="dynamic-large-object-creation">
|
||||
<title>Dynamic large objects</title>
|
||||
<para>You must segment objects that are larger than 5 GB
|
||||
before you can upload them. You then upload the
|
||||
segment objects like you would any other object and
|
||||
create a dynamic large manifest object. The manifest
|
||||
object tells Object Storage how to find the segment
|
||||
objects that comprise the large object. The segments
|
||||
remain individually addressable, but retrieving the
|
||||
manifest object streams all the segments concatenated.
|
||||
There is no limit to the number of segments that can
|
||||
be a part of a single large object.</para>
|
||||
<para>To ensure the download works correctly, you must
|
||||
upload all the object segments to the same container
|
||||
and ensure that each object name is prefixed in such a
|
||||
way that it sorts in the order in which it should be
|
||||
concatenated. You also create and upload a manifest
|
||||
file. The manifest file is a zero-byte file with the
|
||||
extra <literal>X-Object-Manifest</literal>
|
||||
<code><container>/<prefix></code> header,
|
||||
where <code><container></code> is the container
|
||||
the object segments are in and
|
||||
<code><prefix></code> is the common prefix
|
||||
for all the segments. You must UTF-8-encode and then
|
||||
URL-encode the container and common prefix in the
|
||||
<literal>X-Object-Manifest</literal>
|
||||
header.</para>
|
||||
<para>It is best to upload all the segments first and then
|
||||
create or update the manifest. With this method, the
|
||||
full object is not available for downloading until the
|
||||
upload is complete. Also, you can upload a new set of
|
||||
segments to a second location and then update the
|
||||
manifest to point to this new location. During the
|
||||
upload of the new segments, the original manifest is
|
||||
still available to download the first set of
|
||||
segments.</para>
|
||||
<example>
|
||||
<title>Upload segment of large object request:
|
||||
HTTP</title>
|
||||
<literallayout class="monospaced"><xi:include href="samples/large-object-upload-segment-req.txt" parse="text"/></literallayout>
|
||||
</example>
|
||||
<example>
|
||||
<title>Upload segment of large object response:
|
||||
HTTP</title>
|
||||
<literallayout class="monospaced">s</literallayout>
|
||||
</example>
|
||||
<para>No response body is returned. A status code of
|
||||
<returnvalue>2<replaceable>nn</replaceable></returnvalue>
|
||||
(between 200 and 299, inclusive) indicates a
|
||||
successful write; status <errorcode>411</errorcode>
|
||||
<errortext>Length Required</errortext> denotes a
|
||||
missing <literal>Content-Length</literal> or
|
||||
<literal>Content-Type</literal> header in the request.
|
||||
If the MD5 checksum of the data written to the storage
|
||||
system does NOT match the (optionally) supplied ETag
|
||||
value, a <errorcode>422</errorcode>
|
||||
<errortext>Unprocessable Entity</errortext> response
|
||||
is returned.</para>
|
||||
<para>You can continue uploading segments like this
|
||||
example shows, prior to uploading the manifest.</para>
|
||||
<example>
|
||||
<title>Upload next segment of large object request:
|
||||
HTTP</title>
|
||||
<literallayout class="monospaced"><xi:include href="samples/large-object-upload-next-segment-req.txt" parse="text"/></literallayout>
|
||||
</example>
|
||||
<example>
|
||||
<title>Upload next segment of large object response:
|
||||
HTTP</title>
|
||||
<literallayout class="monospaced">w</literallayout>
|
||||
</example>
|
||||
<para>Next, upload the manifest you created that indicates
|
||||
the container the object segments reside within. Note
|
||||
that uploading additional segments after the manifest
|
||||
is created causes the concatenated object to be that
|
||||
much larger but you do not need to recreate the
|
||||
manifest file for subsequent additional
|
||||
segments.</para>
|
||||
<example>
|
||||
<title>Upload manifest request: HTTP</title>
|
||||
<literallayout class="monospaced"><xi:include href="samples/upload-manifest-req.txt" parse="text"/></literallayout>
|
||||
</example>
|
||||
<example>
|
||||
<title>Upload manifest response: HTTP</title>
|
||||
<literallayout class="monospaced"><xi:include href="samples/upload-manifest-resp.txt" parse="text"/></literallayout>
|
||||
</example>
|
||||
<para>The response's <literal>Content-Type</literal> for a
|
||||
&GET; or &HEAD; on the manifest is the same as the
|
||||
<literal>Content-Type</literal> set during the
|
||||
&PUT; request that created the manifest. You can
|
||||
easily change the <literal>Content-Type</literal> by
|
||||
reissuing the &PUT; request.</para>
|
||||
</section>
|
||||
<section xml:id="static-large-objects">
|
||||
<title>Static large objects</title>
|
||||
<procedure>
|
||||
<title>To create a static large object</title>
|
||||
<step>
|
||||
<para>Divide your content into pieces and create
|
||||
(upload) a segment object to contain each
|
||||
piece. You must record the <literal>ETag</literal>
|
||||
response header returned by the&PUT;
|
||||
operation. Alternatively, you can calculate
|
||||
the MD5 checksum of the segment prior to
|
||||
uploading and include this in the
|
||||
<literal>ETag</literal> request header. This
|
||||
ensures that the upload cannot corrupt your
|
||||
data.</para>
|
||||
</step>
|
||||
<step>
|
||||
<para>List the name of each segment object along
|
||||
with its size and MD5 checksum in order.
|
||||
Create a manifest object. You indicate that
|
||||
this is a manifest object by including the
|
||||
<parameter>?multipart-manifest=put</parameter> query
|
||||
string at the end of the manifest object
|
||||
name.</para>
|
||||
</step>
|
||||
</procedure>
|
||||
<para>The body of the &PUT; request on the manifest object
|
||||
comprises a json list, where each element contains the
|
||||
following attributes:</para>
|
||||
<itemizedlist>
|
||||
<listitem>
|
||||
<para><code>path</code>. The container and object
|
||||
name in this format:
|
||||
<code><container-name>/<object-name></code></para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para><code>etag</code>. The MD5 checksum of the
|
||||
content of the segment object. This value must
|
||||
match the <literal>ETag</literal> of that
|
||||
object.</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para><code>size_bytes</code>. The size of the
|
||||
segment object. This value must match the
|
||||
<literal>Content-Length</literal> of that
|
||||
object.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
<example>
|
||||
<title>Static large object manifest list</title>
|
||||
<para>This example shows three segment objects. You
|
||||
can use several containers and the object names do
|
||||
not have to conform to a specific pattern, in
|
||||
contrast to dynamic large objects.</para>
|
||||
<literallayout class="monospaced"><xi:include href="samples/slo-manifest-example.txt" parse="text"/></literallayout>
|
||||
</example>
|
||||
<para>The <literal>Content-Length</literal> request header must
|
||||
contain the length of the json content. Not the length
|
||||
of the segment objects. However, after the &PUT;
|
||||
operation completes, the <literal>Content-Length</literal>
|
||||
metadata is set to the total length of all the object
|
||||
segments. A similar situation applies to the
|
||||
<literal>ETag</literal>. If used in the &PUT; operation,
|
||||
it must contain the MD5 checksum of the json content.
|
||||
The <literal>ETag</literal> metadata value is then set to be
|
||||
the MD5 checksum of the concatenated <literal>ETag</literal>
|
||||
values of the object segments. You can also set the
|
||||
<literal>Content-Type</literal> request header and
|
||||
custom object metadata.</para>
|
||||
<para>When the &PUT; operation sees the
|
||||
<parameter>?multipart-manifest=put</parameter>
|
||||
query parameter, it reads the request body and verifies
|
||||
that each segment object exists and that the sizes and
|
||||
ETags match. If there is a mismatch, the
|
||||
&PUT;operation fails.</para>
|
||||
<para>If everything matches, the manifest object is
|
||||
created. The <literal>X-Static-Large-Object</literal>
|
||||
metadata is set to <literal>true</literal> indicating
|
||||
that this is a static object manifest.</para>
|
||||
<para>Normally when you perform a &GET; operation on the
|
||||
manifest object, the response body contains the
|
||||
concatenated content of the segment objects. To
|
||||
download the manifest list, use the
|
||||
<parameter>?multipart-manifest=get</parameter> query parameter.
|
||||
The resulting list is not formatted the same as the
|
||||
manifest you originally used in the &PUT;
|
||||
operation.</para>
|
||||
<para>If you use the &DELETE; operation on a manifest
|
||||
object, the manifest object is deleted. The segment
|
||||
objects are not affected. However, if you add the
|
||||
<parameter>?multipart-manifest=delete</parameter> query parameter,
|
||||
the segment objects are deleted and if all are
|
||||
successfully deleted, the manifest object is also
|
||||
deleted.</para>
|
||||
<para>To change the manifest, use a &PUT; operation with
|
||||
the <parameter>?multipart-manifest=put</parameter>
|
||||
query parameter. This request creates a
|
||||
new manifest object. You can
|
||||
also update the object metadata in the usual
|
||||
way.</para>
|
||||
</section>
|
||||
</section>
|
||||
</section>
|
||||
|
|
|
|||
|
|
@ -4,9 +4,9 @@
|
|||
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0"
|
||||
xml:id="static-website">
|
||||
<title>Create static website</title>
|
||||
<para>To discover whether your Object Storage system supports
|
||||
this feature, see <xref linkend="discoverability"
|
||||
/>. Alternatively, check with your service provider.</para>
|
||||
<para>To discover whether your Object Storage system supports this
|
||||
feature, see <xref linkend="discoverability"/>. Alternatively,
|
||||
check with your service provider.</para>
|
||||
<para>You can use your Object Storage account to create a static
|
||||
website. This mode is normally only active for anonymous
|
||||
requests, which provide no authentication token. To use it
|
||||
|
|
@ -21,7 +21,7 @@
|
|||
<literal>X-Container-Meta-Web-Error</literal>. (The latter
|
||||
header is discussed below, under <link
|
||||
linkend="Set_Error_Pages_for_Static_Website-dle4005">Set
|
||||
Error Pages for Static Website</link>.) With
|
||||
error pages for static website</link>.) With
|
||||
<literal>X-Container-Meta-Web-Index</literal>, you
|
||||
determine the index file (or default page served, such as
|
||||
<literal>index.html</literal>) displays your website. When
|
||||
|
|
@ -41,9 +41,9 @@
|
|||
style sheet (for example,
|
||||
<literal>lists.css</literal>).</para>
|
||||
<section xml:id="Examples_for_static_web-dle4025">
|
||||
<title>Static Web Middleware through swift</title>
|
||||
<title>Static web middleware through swift</title>
|
||||
<example>
|
||||
<title>Make Container Publicly Readable</title>
|
||||
<title>Make container publicly readable</title>
|
||||
<para>Make the container publicly readable. Once the
|
||||
container is publicly readable, you may access your
|
||||
objects directly, but you must set the index file to
|
||||
|
|
@ -52,27 +52,27 @@
|
|||
<screen><prompt>$</prompt> <userinput>swift post -r '.r:*' container</userinput></screen>
|
||||
</example>
|
||||
<example>
|
||||
<title>Set Site Index File</title>
|
||||
<title>Set site index file</title>
|
||||
<para>Set the index file. In this case,
|
||||
<literal>index.html</literal> is the default file
|
||||
displayed when the site displays.</para>
|
||||
<screen><prompt>$</prompt> <userinput>swift post -m 'web-index:index.html' container</userinput></screen>
|
||||
</example>
|
||||
<example>
|
||||
<title>Enable File Listing</title>
|
||||
<title>Enable file listing</title>
|
||||
<para>Turn on file listing. If you do not set the index
|
||||
file, list the objects in the container. Instructions
|
||||
on styling the list with the CSS follow.</para>
|
||||
<screen><prompt>$</prompt> <userinput>swift post -m 'web-listings: true' container</userinput></screen>
|
||||
</example>
|
||||
<example>
|
||||
<title>Enable CSS for File Listing</title>
|
||||
<title>Enable CSS for file listing</title>
|
||||
<para>Style the file listing.</para>
|
||||
<programlisting language="ini"><xi:include href="samples/file-listings-css-set-req.txt" parse="text"/></programlisting>
|
||||
</example>
|
||||
</section>
|
||||
<section xml:id="Set_Error_Pages_for_Static_Website-dle4005">
|
||||
<title>Set Error Pages for Static Website</title>
|
||||
<title>Set error pages for static website</title>
|
||||
<para>You can create and set custom error pages for visitors
|
||||
to your website; currently, only 401 (Unauthorized) and
|
||||
404 (Not Found) errors are supported. To do this, set the
|
||||
|
|
@ -93,10 +93,10 @@
|
|||
<para>Set the <literal>X-Container-Meta-Web-Error</literal>
|
||||
metadata once for your entire static website.</para>
|
||||
<example>
|
||||
<title>Set Error Pages for Static Website Request</title>
|
||||
<title>Set error pages for static website request</title>
|
||||
<programlisting language="ini"><xi:include href="samples/error-page-set-req.txt" parse="text"/></programlisting>
|
||||
</example>
|
||||
<para>Any 2<varname>nn</varname> response indicates success.
|
||||
</para>
|
||||
<para>Any 2<varname>nn</varname> response indicates
|
||||
success.</para>
|
||||
</section>
|
||||
</section>
|
||||
|
|
|
|||
Loading…
Reference in New Issue