Previously, parse_header_string was only called with data coming out of
requests, which would be either bytes or unicode. Now that we're sending
it request headers as well (see related change), we need to be more
defensive.
If the value given is neither bytes nor unicode, convert it to a native
string. This will allow developers using the client API to continue
sending header dicts like
{'X-Delete-After': 2}
...as in Swift's test/probe/test_object_expirer.py
Change-Id: Ie57a93274507b184af5cad4260f244359a585f09
Related-Change: I43dd7254f7281d4db59b286aa2145643c64e1705
The test should have included utf8 encoded unicode data to
test that encoded unicode data stored in headers was parsed
correctly.
Also fixes the docstring for swiftclient.safe_value()
Change-Id: Id0def0b3af7a364f1257cc22f67b71c0cc5d8479
* Improve some formatting
* Be more explicit about how much will be revealed when
* Rename redact_sensitive_tokens to redact_sensitive_headers, as it
affects more than tokens.
Change-Id: I02b375d914e9f0a210d038ecb31188d09a8ffce3
Currently the swiftclient logs sensitive info in headers when logging
HTTP requests. This patch hides sensitive info in headers such as
'X-Auth-Token' in a similar way to swift itself (we add a
'reveal_sensitive_prefix' configuration to the client).
With this patch, tokens are truncated by removing the specified number
of characters, after which '...' is appended to the logged token to
indicate that it has been redacted.
Co-Authored-By: Li Cheng <shcli@cn.ibm.com>
Co-Authored-By: Zack M. Davis <zdavis@swiftstack.com>
Change-Id: I43dd7254f7281d4db59b286aa2145643c64e1705
Closes-bug: #1516692
In client get_container(), when full_listing is true,
the calls back to get_container() pass service_token
as a positional arg which maps its value to the
full_listing arg. It should use a keyword.
Change-Id: Iac2af45df124ff33fcb7fbaf1ba959ef06c96378
Closes-Bug: #1496093
"# flake8: noqa" option disables all checks for the whole file.
To disable one line we should use "# noqa".
Change-Id: I7859eab30563d0eb91c5f055d1b523173b562e54
Closes-bug: #1540254
Add support for the auth-version to be specified using
--os-identity-api-version or OS_IDENTITY_API_VERSION
for compatibility with other openstack client command
line options.
The auth version used will be selected as follows:
- if either --auth-version or --os-identity-api-version
is set, use that value
- otherwise use the value of ST_AUTH_VERSION, if set
- otherwise use the value of OS_AUTH_VERSION, if set
- otherwise (new behaviour) use the value of
OS_IDENTITY_API_VERSION, if set
- otherwise default to 1.0
Note that before this change the auth version might have
defaulted to 1.0 despite OS_IDENTITY_API_VERSION being set,
but with this change OS_IDENTITY_API_VERSION is preferred.
Change-Id: Ifba4c4e43560ede3013337b8cdbc77dc2de6e8ff
Closes-Bug: #1541273
This fix ensures creation and use of the correct default segment
container when pseudo-folder paths are passed via <container> arg.
Change-Id: I90356b041dc9dfbd55eb341271975621759476b9
Closes-Bug: 1532981
Related-Bug: 1478210
Allow swiftclient to authenticate against keystone using tenant
name/id and token only. Without this patch, the password is
required, which may not always be available. Authentication
against keystone is required to get the service catalog,
which includes the endpoints for swift.
Change-Id: I4477af445474c5fa97ff864c4942f1330b59e5d6
Closes-Bug: #1476002
When issuing `delete` commands that would require three or more
individual deletes, check whether the cluster supports bulk deletes
and use that if it's available.
Additionally, a new option is added to the `delete` command:
* --prefix <prefix>
Delete all objects that start with <prefix>. This is similar to the
--prefix option for the `list` command.
Example:
$ swift delete c --prefix obj_prefix/
...will delete from container "c" all objects whose name begins with
"obj_prefix/", such as "obj_prefix/foo" and "obj_prefix/bar".
Change-Id: I6b9504848d6ef562cf4f570bbcd17db4e3da8264
Also, don't try to do int(None) for chunk-encoded responses (like DLOs
that are longer than a single container listing).
Change-Id: Ibacd75d5ee46135d62388786903c895fda8ed3ba
When we introduced LengthWrapper, we neglected to make it resettable. As
a result, upload failures result in errors like:
put_object(...) failure and no ability to reset contents for reupload.
Now, LengthWrappers will be resettable if their _readable has seek/tell.
Related-Change: I6c8bc1366dfb591a26d934a30cd21c9e6b9a04ce
Change-Id: I21f43f06e8c78b24d1fc081efedf2687942e042f
Currently the swift client retries establishing a connection to the server
(by default up to 5 times). However, when downloading an object, once
the connection has been established and the inital headers have been
returned, no attempt is made to retry.
So, for example, if 99MB of a 100MB object have been downloaded and the
connection is then lost, the download will fail. This patch changes the
behaviour to re-establish the connection and fetch the remaining bytes
using the 'Range' header to offset.
Data retry is not yet supported if the original request is for a subset
of the object data (ie uses the 'Range' header), or if resp_chunk_size
has not been set.
The object's etag is checked using If-Match to make sure the object data
hasn't changed since the start of the download.
Change-Id: Iab47f10081ff39f6d344dbc2479cbc3bfd1c5b29
This fix makes it possible to upload objects to pseudo-folders by
passing the upload paths via <container> arg regardless of whether the
container or folder path exist or not.
Change-Id: I575e58aa12adcf71cdaa70d025a0ea5c63f46903
Closes-Bug: #1478210
Partial-Bug: #1432734
Related-Bug: #1432734
The debug and info options need to be set before a subcommand method is called,
otherwise they are simply ignored. This is kind of irritating - other options
(for example -U, -A, -K) are usable after a positional command.
This patch fixes this, and commands like these are no longer ignoring --debug or --info:
swift stat --debug
swift list container --info
Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
Change-Id: Ib19b05deef7a015881f1eed4a3946025e16bf922
Currently, if you attempt to stream an upload from an iterator, as in
def data():
yield 'foo'
yield ''
yield 'bar'
conn.put_object('c', 'o', data())
... requests will faithfully emit a zero-length chunk, ending the
transfer. Swift will then close the connection, possibly (if Connection:
keep-alive was set) after attempting to parse the next chunk as a new
request.
Now, Swift will receive all of the bytes from the iterable, and any
zero-byte chunks will be ignored.
This will be fixed in requests [1], but not until an eventual 3.0.0
release.
[1] https://github.com/kennethreitz/requests/pull/2631
Change-Id: I19579ed7a0181ac3f488433e7c1839f7f7a040b8
Wrong usage of "an" in the messages:
"the optional os_options paramater includes an non-empty"
"We are allowing to have an tenant_name argument"
Should be:
"the optional os_options paramater includes a non-empty"
"We are allowing to have a tenant_name argument"
Totally 2 occurrences in python-swiftclient base code.
Change-Id: I2f2f7e07432fedfee5ccb418d9505250b3fed597
It was dropping warnings like "UserWarning: Providing attr without
filter_value to get_urls() is deprecated as of the 1.7.0 release and may
be removed in the 2.0.0 release. Either both should be provided or
neither should be provided."
Change-Id: Iead0bcf36b4a46bf465a55a33a21fd7f14f0ac40
New documentation for python-swiftclient that introduces
the APIs available and gives some opinionated advice about
when to use the shell, the client API and the service API.
Change-Id: I19020f041fab2e72469979f712ffe3951c431d24
According to help `swift upload -h` you can add a customized request header 'Content-Type'.
But actually it is ignored (cleared and default is used) if subcommand is upload.
Subcommand post works as expected in help.
Bug fix: Use 'Content-Type' from the customized request headers also if
uploading.
Change-Id: If0d1354b6214b909527341078fe1769aa6587457
