openstack
/
python-swiftclient
Code Issues Proposed changes

Merge "docs: Clean up formatting"

This commit is contained in:
Zuul 2019-05-13 19:48:14 +00:00 committed by Gerrit Code Review
parent af4bac31fa 7563d9cb56
commit 2e43285f67
4 changed files with 95 additions and 101 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

40
doc/source/cli/index.rst
View File

@ -245,13 +245,10 @@ storage URL options shown below:
--os-storage-url https://10.1.5.2:8080/v1/AUTH_ced809b6a4baea7aeab61a \ --os-storage-url https://10.1.5.2:8080/v1/AUTH_ced809b6a4baea7aeab61a \
list list
.. We need the backslash below in order to indent the note .. note::
\
.. note:: Leftover environment variables are a common source of confusion when
authorization fails.
Leftover environment variables are a common source of confusion when
authorization fails.
CLI commands CLI commands
~~~~~~~~~~~~ ~~~~~~~~~~~~
@ -739,15 +736,15 @@ is passed, the Unix timestamp when the temporary URL will expire.
But beyond that, ``time`` can also be specified as an ISO 8601 timestamp But beyond that, ``time`` can also be specified as an ISO 8601 timestamp
in one of following formats: in one of following formats:
i) Complete date: YYYY-MM-DD (eg 1997-07-16) i) Complete date: YYYY-MM-DD (e.g. 1997-07-16)
ii) Complete date plus hours, minutes and seconds: ii) Complete date plus hours, minutes and seconds:
YYYY-MM-DDThh:mm:ss YYYY-MM-DDThh:mm:ss
(eg 1997-07-16T19:20:30) (e.g. 1997-07-16T19:20:30)
iii) Complete date plus hours, minutes and seconds with UTC designator: iii) Complete date plus hours, minutes and seconds with UTC designator:
YYYY-MM-DDThh:mm:ssZ YYYY-MM-DDThh:mm:ssZ
(eg 1997-07-16T19:20:30Z) (e.g. 1997-07-16T19:20:30Z)
Please be aware that if you don't provide the UTC designator (i.e., Z) Please be aware that if you don't provide the UTC designator (i.e., Z)
the timestamp is generated using your local timezone. If only a date is the timestamp is generated using your local timezone. If only a date is
@ -881,17 +878,14 @@ Download an object from a container:
testSwift.txt [auth 0.028s, headers 0.045s, total 0.045s, 0.002 MB/s] testSwift.txt [auth 0.028s, headers 0.045s, total 0.045s, 0.002 MB/s]
.. We need the backslash below in order to indent the note .. note::
\
.. note:: To upload an object to a container, your current working directory must be
where the file is located or you must provide the complete path to the file.
To upload an object to a container, your current working directory must be In other words, the --object-name <object-name> is an option that will upload
where the file is located or you must provide the complete path to the file. file and name object to <object-name> or upload directory and use <object-name> as
In other words, the --object-name <object-name> is an option that will upload object prefix. In the case that you provide the complete path of the file,
file and name object to <object-name> or upload directory and use <object-name> as that complete path will be the name of the uploaded object.
object prefix. In the case that you provide the complete path of the file,
that complete path will be the name of the uploaded object.
For example: For example:

20
doc/source/index.rst
View File

@ -39,17 +39,17 @@ Indices and tables
License License
~~~~~~~ ~~~~~~~
Copyright 2013 OpenStack, LLC. Copyright 2013 OpenStack, LLC.
Licensed under the Apache License, Version 2.0 (the "License"); Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License. you may not use this file except in compliance with the License.
You may obtain a copy of the License at You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0 * http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and See the License for the specific language governing permissions and
limitations under the License. limitations under the License.

82
doc/source/introduction.rst
View File

@ -16,41 +16,41 @@ Use Cases
Alongside the command line tool, the ``python-swiftclient`` includes two Alongside the command line tool, the ``python-swiftclient`` includes two
levels of API: levels of API:
* A low level client API that provides simple Python wrappers around the * A low level client API that provides simple Python wrappers around the
various authentication mechanisms and the individual HTTP requests. various authentication mechanisms and the individual HTTP requests.
* A high level service API that provides methods for performing common * A high level service API that provides methods for performing common
operations in parallel on a thread pool. operations in parallel on a thread pool.
Example use cases: Example use cases:
* Uploading and retrieving data * Uploading and retrieving data
Use the command line tool if you are simply uploading and downloading Use the command line tool if you are simply uploading and downloading
files and directories to and from your filesystem. The command line tool files and directories to and from your filesystem. The command line tool
can be integrated into a shell script to automate tasks. can be integrated into a shell script to automate tasks.
* Integrating into an automated Python workflow * Integrating into an automated Python workflow
Use the ``SwiftService`` API to perform operations offered by the CLI Use the ``SwiftService`` API to perform operations offered by the CLI
if your use case requires integration with a Python-based workflow. if your use case requires integration with a Python-based workflow.
This method offers greater control and flexibility over individual object This method offers greater control and flexibility over individual object
operations, such as the metadata set on each object. The ``SwiftService`` operations, such as the metadata set on each object. The ``SwiftService``
class provides methods to perform multiple sets of operations against a class provides methods to perform multiple sets of operations against a
swift object store using a configurable shared thread pool. A single swift object store using a configurable shared thread pool. A single
instance of the ``SwiftService`` class can be shared between multiple instance of the ``SwiftService`` class can be shared between multiple
threads in your own code. threads in your own code.
* Developing an application in Python to access a swift object store * Developing an application in Python to access a swift object store
Use the ``SwiftService`` API to develop Python applications that use Use the ``SwiftService`` API to develop Python applications that use
swift to store and retrieve objects. A ``SwiftService`` instance provides swift to store and retrieve objects. A ``SwiftService`` instance provides
a configurable thread pool for performing all operations supported by the a configurable thread pool for performing all operations supported by the
CLI. CLI.
* Fine-grained control over threading or the requests being performed * Fine-grained control over threading or the requests being performed
Use the ``Connection`` API if your use case requires fine grained control Use the ``Connection`` API if your use case requires fine grained control
over advanced features or you wish to use your own existing threading over advanced features or you wish to use your own existing threading
model. Examples of advanced features requiring the use of the model. Examples of advanced features requiring the use of the
``Connection`` API include creating an SLO manifest that references ``Connection`` API include creating an SLO manifest that references
already existing objects, or fine grained control over the query strings already existing objects, or fine grained control over the query strings
supplied with each HTTP request. supplied with each HTTP request.
Important considerations Important considerations
~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~
@ -66,19 +66,19 @@ the proper use case, and not treat the storage like a traditional filesystem.
There are two main restrictions to bear in mind when designing an application There are two main restrictions to bear in mind when designing an application
that uses an object store: that uses an object store:
* You cannot rename objects. Due to fact that the name of an object is one * You cannot rename objects. Due to fact that the name of an object is one
of the factors that determines where the object and its replicas are stored, of the factors that determines where the object and its replicas are stored,
renaming would require multiple copies of the data to be moved between renaming would require multiple copies of the data to be moved between
physical storage devices. If you want to rename an object you must upload physical storage devices. If you want to rename an object you must upload
to the new location, or make a server side copy request to the new location, to the new location, or make a server side copy request to the new location,
and then delete the original. and then delete the original.
* You cannot modify objects. Objects are stored in multiple locations and * You cannot modify objects. Objects are stored in multiple locations and
are checked for integrity based on the MD5 sum calculated during are checked for integrity based on the MD5 sum calculated during
upload. In order to modify the contents of an object, the entire desired upload. In order to modify the contents of an object, the entire desired
contents must be re-uploaded. In certain special cases it is possible to contents must be re-uploaded. In certain special cases it is possible to
work around this restriction using large objects, but no general work around this restriction using large objects, but no general
file-like access is available to modify a stored object. file-like access is available to modify a stored object.
Objects cannot be locked Objects cannot be locked
------------------------ ------------------------

54
doc/source/service-api.rst
View File

@ -26,10 +26,10 @@ the auth version based on the combination of options specified, but
supplying options from multiple different auth versions can cause unexpected supplying options from multiple different auth versions can cause unexpected
behaviour. behaviour.
.. note:: .. note::
Leftover environment variables are a common source of confusion when Leftover environment variables are a common source of confusion when
authorization fails. authorization fails.
Keystone V3 Keystone V3
~~~~~~~~~~~ ~~~~~~~~~~~
@ -109,17 +109,17 @@ in this dictionary are described below, along with their defaults:
Options Options
~~~~~~~ ~~~~~~~
``retries``: ``5`` ``retries``: ``5``
The number of times that the library should attempt to retry HTTP The number of times that the library should attempt to retry HTTP
actions before giving up and reporting a failure. actions before giving up and reporting a failure.
``container_threads``: ``10`` ``container_threads``: ``10``
``object_dd_threads``: ``10`` ``object_dd_threads``: ``10``
``object_uu_threads``: ``10`` ``object_uu_threads``: ``10``
``segment_threads``: ``10`` ``segment_threads``: ``10``
The above options determine the size of the available thread pools for The above options determine the size of the available thread pools for
performing swift operations. Container operations (such as listing a performing swift operations. Container operations (such as listing a
container) operate in the container threads, and a similar pattern container) operate in the container threads, and a similar pattern
@ -131,86 +131,86 @@ Options
``uu`` and ``dd``. This stands for "upload/update" and "download/delete", ``uu`` and ``dd``. This stands for "upload/update" and "download/delete",
and the corresponding actions will be run on separate threads pools. and the corresponding actions will be run on separate threads pools.
``segment_size``: ``None`` ``segment_size``: ``None``
If specified, this option enables uploading of large objects. Should the If specified, this option enables uploading of large objects. Should the
object being uploaded be larger than 5G in size, this option is object being uploaded be larger than 5G in size, this option is
mandatory otherwise the upload will fail. This option should be mandatory otherwise the upload will fail. This option should be
specified as a size in bytes. specified as a size in bytes.
``use_slo``: ``False`` ``use_slo``: ``False``
Used in combination with the above option, ``use_slo`` will upload large Used in combination with the above option, ``use_slo`` will upload large
objects as static rather than dynamic. Only static large objects provide objects as static rather than dynamic. Only static large objects provide
error checking for the downloaded object, so we recommend this option. error checking for the downloaded object, so we recommend this option.
``segment_container``: ``None`` ``segment_container``: ``None``
Allows the user to select the container into which large object segments Allows the user to select the container into which large object segments
will be uploaded. We do not recommend changing this value as it could make will be uploaded. We do not recommend changing this value as it could make
locating orphaned segments more difficult in the case of errors. locating orphaned segments more difficult in the case of errors.
``leave_segments``: ``False`` ``leave_segments``: ``False``
Setting this option to true means that when deleting or overwriting a large Setting this option to true means that when deleting or overwriting a large
object, its segments will be left in the object store and must be cleaned object, its segments will be left in the object store and must be cleaned
up manually. This option can be useful when sharing large object segments up manually. This option can be useful when sharing large object segments
between multiple objects in more advanced scenarios, but must be treated between multiple objects in more advanced scenarios, but must be treated
with care, as it could lead to ever increasing storage usage. with care, as it could lead to ever increasing storage usage.
``changed``: ``None`` ``changed``: ``None``
This option affects uploads and simply means that those objects which This option affects uploads and simply means that those objects which
already exist in the object store will not be overwritten if the ``mtime`` already exist in the object store will not be overwritten if the ``mtime``
and size of the source is the same as the existing object. and size of the source is the same as the existing object.
``skip_identical``: ``False`` ``skip_identical``: ``False``
A slightly more thorough case of the above, but rather than ``mtime`` and size A slightly more thorough case of the above, but rather than ``mtime`` and size
uses an object's ``MD5 sum``. uses an object's ``MD5 sum``.
``yes_all``: ``False`` ``yes_all``: ``False``
This options affects only download and delete, and in each case must be This options affects only download and delete, and in each case must be
specified in order to download/delete the entire contents of an account. specified in order to download/delete the entire contents of an account.
This option has no effect on any other calls. This option has no effect on any other calls.
``no_download``: ``False`` ``no_download``: ``False``
This option only affects download and means that all operations proceed as This option only affects download and means that all operations proceed as
normal with the exception that no data is written to disk. normal with the exception that no data is written to disk.
``header``: ``[]`` ``header``: ``[]``
Used with upload and post operations to set headers on objects. Headers Used with upload and post operations to set headers on objects. Headers
are specified as colon separated strings, e.g. "content-type:text/plain". are specified as colon separated strings, e.g. "content-type:text/plain".
``meta``: ``[]`` ``meta``: ``[]``
Used to set metadata on an object similarly to headers. Used to set metadata on an object similarly to headers.
.. note:: .. note::
Setting metadata is a destructive operation, so when updating one Setting metadata is a destructive operation, so when updating one
of many metadata values all desired metadata for an object must be re-applied. of many metadata values all desired metadata for an object must be re-applied.
``long``: ``False`` ``long``: ``False``
Affects only list operations, and results in more metrics being made Affects only list operations, and results in more metrics being made
available in the results at the expense of lower performance. available in the results at the expense of lower performance.
``fail_fast``: ``False`` ``fail_fast``: ``False``
Applies to delete and upload operations, and attempts to abort queued Applies to delete and upload operations, and attempts to abort queued
tasks in the event of errors. tasks in the event of errors.
``prefix``: ``None`` ``prefix``: ``None``
Affects list operations; only objects with the given prefix will be Affects list operations; only objects with the given prefix will be
returned/affected. It is not advisable to set at the service level, as returned/affected. It is not advisable to set at the service level, as
those operations that call list to discover objects on which they should those operations that call list to discover objects on which they should
operate will also be affected. operate will also be affected.
``delimiter``: ``None`` ``delimiter``: ``None``
Affects list operations, and means that listings only contain results up Affects list operations, and means that listings only contain results up
to the first instance of the delimiter in the object name. This is useful to the first instance of the delimiter in the object name. This is useful
for working with objects containing '/' in their names to simulate folder for working with objects containing '/' in their names to simulate folder
structures. structures.
``dir_marker``: ``False`` ``dir_marker``: ``False``
Affects uploads, and allows empty 'pseudofolder' objects to be created Affects uploads, and allows empty 'pseudofolder' objects to be created
when the source of an upload is ``None``. when the source of an upload is ``None``.
``checksum``: ``True`` ``checksum``: ``True``
Affects uploads and downloads. If set check md5 sum for the transfer. Affects uploads and downloads. If set check md5 sum for the transfer.
``shuffle``: ``False`` ``shuffle``: ``False``
When downloading objects, the default behaviour of the CLI is to shuffle When downloading objects, the default behaviour of the CLI is to shuffle
lists of objects in order to spread the load on storage drives when multiple lists of objects in order to spread the load on storage drives when multiple
clients are downloading the same files to multiple locations (e.g. in the clients are downloading the same files to multiple locations (e.g. in the
@ -220,12 +220,12 @@ Options
are downloaded in lexically-sorted order. Setting this option to ``True`` are downloaded in lexically-sorted order. Setting this option to ``True``
gives the same shuffling behaviour as the CLI. gives the same shuffling behaviour as the CLI.
``destination``: ``None`` ``destination``: ``None``
When copying objects, this specifies the destination where the object When copying objects, this specifies the destination where the object
will be copied to. The default of None means copy will be the same as will be copied to. The default of None means copy will be the same as
source. source.
``fresh_metadata``: ``None`` ``fresh_metadata``: ``None``
When copying objects, this specifies that the object metadata on the When copying objects, this specifies that the object metadata on the
source will *not* be applied to the destination object - the source will *not* be applied to the destination object - the
destination object will have a new fresh set of metadata that includes destination object will have a new fresh set of metadata that includes