Previously, if a Range request came back 200 OK (rather than 206 Partial
Content), we would mangle the response body. This could happen if there
was a middleware that would silently drop Range headers, for example.
Now, if the response does not include a Content-Range header, we will
log a warning and seek to our previous position in the stream. If the
Content-Range header has an unexpected value, we will raise an exception.
Change-Id: I94d4536cc1489968d45a2b6ba7edd70c85800275
As a result of the Hackathon we have produced a new
documentation structure for the python-swiftclient.
This patch introduces the new structure and adds the
required content.
The intention is to document the CLI, the SwiftService
and Connection API. Importantly, we also provide
guidance on important considerations when using a swift
object store, such as which aspect of the python-swiftclient
to use for various use cases, common authentication patterns
and some useful examples.
Co-Authored-By: Alexandra Settle <alexandra.settle@rackspace.com>
Co-Authored-By: Mohit Motiani <mohit.motiani@intel.com>
Co-Authored-By: Hisashi Osanai <osanai.hisashi@jp.fujitsu.com>
Change-Id: I9eb41f8e9137efa66cead67dc264a76a3c03fbda
The documentation of "swift download" hints that "marker" option
is supported, but in reality we forgot to patch it through, so
all downloads were always done with the default, empty marker.
Closes-Bug: #1565393
Change-Id: I38bd29d2baa9188b61397dec75ce1d864041653c
...using a new "application/swiftclient-segment" content-type.
Segments uploaded by swiftclient are expected to have a many-to-one
relationship to large objects, rather than the more-general many-to-many
relationship that SLO and DLO generally allow. Later, we may use this
information to make more intelligent decisions, such as when to
automatically clean up segments.
Change-Id: Ie56a3aa10065db754ac572cc37d93f2c901aac60
Why now?
* argparse was introduced in Python 3.2 and back-ported to Python 2.7.
Until we dropped Python 2.6 support, we were stuck on optparse.
* keystoneauth.loading.cli provides register_argparse_arguments and
load_from_argparse_arguments helper methods. Now that we're moving
toward Keystone Session support, argparse seems required.
Closes-Bug: 1553030
Change-Id: I5139fb64a8631a3010680090fd04345f95c55c7b
SwiftService().post(cont, [SwiftPostObject(obj, options]) currently
ignores options['header'], raises exception when options['headers']
is set and make malformed metadata when options['meta'] is set.
Fix tipos in code, add unittest for SwiftService().post
Closes-Bug: #1560052
Change-Id: Ie460f753492e9b73836b4adfc7c9c0f2130a8a91
Now, client applications can get to things like transaction IDs for
failures without needing to turn on all of logging.
While we're at it, add a from_response factory method for
ClientException.
Co-Authored-By: Alexander Corwin <ancorwin@gmail.com>
Change-Id: Ib46d5f8fc7f36f651f5908bb9d900316fdaebce3
in python-swiftclient/swiftclient/service.py,
there is a method
def download(self, container=None, objects=None, options=None):
if container is specified but objects not, it download all
objects in specified container.
if both container and objects are specified, it download all
specified objects in the container.
when it comes to the case that, objects argument is specified,
but it turned out to be empty array [ ], the download method
download all the objects under specified container.
this may be not reasonable.
for example,
the objects was not empty when it came from command line,
but it's filtered, maybe by --prefix argument.
at last, it turned out to be empty array.
when calling download method with objects arguments
being empty array, we should download nothing instead of
all the objects under the specified container.
Change-Id: I81aab935533a50b40679c8b3575f298c285233a8
Closes-bug: #1549881
Previously, parse_header_string was only called with data coming out of
requests, which would be either bytes or unicode. Now that we're sending
it request headers as well (see related change), we need to be more
defensive.
If the value given is neither bytes nor unicode, convert it to a native
string. This will allow developers using the client API to continue
sending header dicts like
{'X-Delete-After': 2}
...as in Swift's test/probe/test_object_expirer.py
Change-Id: Ie57a93274507b184af5cad4260f244359a585f09
Related-Change: I43dd7254f7281d4db59b286aa2145643c64e1705
The test should have included utf8 encoded unicode data to
test that encoded unicode data stored in headers was parsed
correctly.
Also fixes the docstring for swiftclient.safe_value()
Change-Id: Id0def0b3af7a364f1257cc22f67b71c0cc5d8479
* Improve some formatting
* Be more explicit about how much will be revealed when
* Rename redact_sensitive_tokens to redact_sensitive_headers, as it
affects more than tokens.
Change-Id: I02b375d914e9f0a210d038ecb31188d09a8ffce3
Add checking object-threads, container-threads and segment-threads options validation
The values should be a positive integer
Change-Id: Iaf98e33dae4b9a7f82e33f7cc2e5a0b293a1c76f
Close-Bug: #1546973
Currently the swiftclient logs sensitive info in headers when logging
HTTP requests. This patch hides sensitive info in headers such as
'X-Auth-Token' in a similar way to swift itself (we add a
'reveal_sensitive_prefix' configuration to the client).
With this patch, tokens are truncated by removing the specified number
of characters, after which '...' is appended to the logged token to
indicate that it has been redacted.
Co-Authored-By: Li Cheng <shcli@cn.ibm.com>
Co-Authored-By: Zack M. Davis <zdavis@swiftstack.com>
Change-Id: I43dd7254f7281d4db59b286aa2145643c64e1705
Closes-bug: #1516692
In client get_container(), when full_listing is true,
the calls back to get_container() pass service_token
as a positional arg which maps its value to the
full_listing arg. It should use a keyword.
Change-Id: Iac2af45df124ff33fcb7fbaf1ba959ef06c96378
Closes-Bug: #1496093
"# flake8: noqa" option disables all checks for the whole file.
To disable one line we should use "# noqa".
Change-Id: I7859eab30563d0eb91c5f055d1b523173b562e54
Closes-bug: #1540254
Add support for the auth-version to be specified using
--os-identity-api-version or OS_IDENTITY_API_VERSION
for compatibility with other openstack client command
line options.
The auth version used will be selected as follows:
- if either --auth-version or --os-identity-api-version
is set, use that value
- otherwise use the value of ST_AUTH_VERSION, if set
- otherwise use the value of OS_AUTH_VERSION, if set
- otherwise (new behaviour) use the value of
OS_IDENTITY_API_VERSION, if set
- otherwise default to 1.0
Note that before this change the auth version might have
defaulted to 1.0 despite OS_IDENTITY_API_VERSION being set,
but with this change OS_IDENTITY_API_VERSION is preferred.
Change-Id: Ifba4c4e43560ede3013337b8cdbc77dc2de6e8ff
Closes-Bug: #1541273
This fix ensures creation and use of the correct default segment
container when pseudo-folder paths are passed via <container> arg.
Change-Id: I90356b041dc9dfbd55eb341271975621759476b9
Closes-Bug: 1532981
Related-Bug: 1478210
Allow swiftclient to authenticate against keystone using tenant
name/id and token only. Without this patch, the password is
required, which may not always be available. Authentication
against keystone is required to get the service catalog,
which includes the endpoints for swift.
Change-Id: I4477af445474c5fa97ff864c4942f1330b59e5d6
Closes-Bug: #1476002
When issuing `delete` commands that would require three or more
individual deletes, check whether the cluster supports bulk deletes
and use that if it's available.
Additionally, a new option is added to the `delete` command:
* --prefix <prefix>
Delete all objects that start with <prefix>. This is similar to the
--prefix option for the `list` command.
Example:
$ swift delete c --prefix obj_prefix/
...will delete from container "c" all objects whose name begins with
"obj_prefix/", such as "obj_prefix/foo" and "obj_prefix/bar".
Change-Id: I6b9504848d6ef562cf4f570bbcd17db4e3da8264
