If "-" is passed in for the source, python-swiftclient will upload
the object by reading the contents of the standard input. The object
name option must be set, as well, and this cannot be used in
conjunction with other files.
This approach stores the entire contents as one object. A follow on
patch will change this behavior to upload from standard input as SLO,
unless the segment size is larger than the content size.
Change-Id: I1a8be6377de06f702e0f336a5a593408ed49be02
Currently, the swiftclient upload command passes a custom metadata
header for each object (called object-meta-mtime), whose value is
the current UNIX timestamp. When downloading such an object with the
swiftclient, the mtime header is parsed and passed as the atime and
mtime for the newly created file.
There are use-cases where this is not desired, for example when using
tmp or scratch directories in which files older than a specific date
are deleted. This commit provides a boolean option for ignoring the
mtime header.
Change-Id: If60b389aa910c6f1969b999b5d3b6d0940375686
Previously, python-swiftclient worked around a requests issue where
Content-Type could be set to application/x-www-form-urlencoded when
using python3. This issue has been resolved and a fix released in
requests 2.4 (fixed in subsequent releases as well). The patch makes
the workaround conditional on the requests version, so that with
sufficiently new requests libraries, the Content-Type is not set.
For reference, requests 2.4 was released August 29th, 2014. The
specific issue filed in the requests tracker is:
https://github.com/requests/requests/issues/2071.
Related-Change: I035f8b4b9c9ccdc79820b907770a48f86d0343b4
Closes-Bug: #1433767
Change-Id: Ieb2243d2ff5326920a27ce8c3c6f0f5c396701ed
Since time immemorial, Swift has returned unquoted ETags for plain-old
Swift objects -- I hear tell that we once tried to change this, but
quickly backed it out when some clients broke.
However, some proxies (such as nginx) apparently may force the ETag to
adhere to the RFC, which states [1]:
An entity-tag consists of an opaque *quoted* string
(emphasis mine). See the related bug for an instance of this happening.
Since we can still get the original ETag easily, we should tolerate the
more-compliant format.
[1] https://tools.ietf.org/html/rfc2616.html#section-3.11 or, if you
prefer the new ones, https://tools.ietf.org/html/rfc7232#section-2.3
Change-Id: I7cfacab3f250a9443af4b67111ef8088d37d9171
Closes-Bug: 1681529
Related-Bug: 1678976
Previously, using SwiftService to delete "many" objects would use
bulk delete if available, but it would not respect the bulk delete
page size. If the number of objects to delete exceeded the bulk delete
page size, SwiftService would ignore the error and nothing would be
deleted.
This patch changes _should_bulk_delete() to be _bulk_delete_page_size();
instead of returning a simple True/False, it returns the page size for
the bulk deleter, or 1 if objects should be deleted one at a time.
Delete SDK calls are then spread across multiple bulk DELETEs if the
requested number of objects to delete exceeds the returned page size.
Fixed the logic in _should_bulk_delete() so that if the object list
is exactly 2x the thread count, it will not bulk delete. This is the
natural conclusion following the logic that existed previously: if
the delete request can be satisfied by every worker thread doing one
or two tasks, don't bulk delete. But if it requires a worker thread
to do three or more tasks, do a bulk delete instead. Previously, the
logic would mean that if every worker thread did exactly two tasks, it
would bulk delete. This patch changes a "<" to a "<=".
Closes-Bug: 1679851
Change-Id: I3c18f89bac1170dc62187114ef06dbe721afcc2e
If we were to include this in a normal PUT, it would 400, but only if
slo is actually in the pipeline. If it's *not*, we'll create a normal
Swift object and the header sticks.
- This is really confusing for users; see the related bug.
- If slo is later enabled in the cluster, Swift starts responding 500
with a KeyError because the client and on-disk formats don't match!
Change-Id: I1d80c76af02f2ca847123349224ddc36d2a6996b
Related-Change: I986c1656658f874172860469624118cc63bff9bc
Related-Bug: #1680083
Client-side implementation for ISO 8601 timestamp
support of tempurl middleware. Please see
https://review.openstack.org/#/c/422679/
Change-Id: I76da28b48948475ec1bae5258e0b39a316553fb7
Probably the most common format for documenting arguments is
reST field lists [1]. This change updates some docstrings to
comply with the field lists syntax.
[1] http://sphinx-doc.org/domains.html#info-field-lists
Change-Id: Ic011fd3e3a8c5bafa24a3438a6ed5bb126b50e95
The opened file for upload is not closed.
This fix prevents possible file handle leak.
Closes-Bug: #1559079
Change-Id: Ibc58667789e8f54c74ae2bbd32717a45f7b30550
This change pulls out that relatively new [1] little string to pull at
in the MockHttpResponse that I think is sorta ugly. And replaces it
with the correct behavior that's representative of the Real for which
it's standing in (which is sadly our wrapper to make a requests response
feel like a httplib.HTTPResponse).
It's not clear (to me) the history which allowed this difference in the
behavior of the Real and Fake to persist - it seems to have always been
this way [2].
I also reworded a relatively new test [1] to cover more code, and make
assertions on the desired behavior of the client instead of "just" the
http_log method.
FWIW, I don't think there was necessarily anything wrong with the scope
of the new test [1] - and it certainly makes sense to see new tests copy
nearby existing tests. But I subjectively think this smaller test is
more demonstrative of the desired behavior.
1. Related-Change-Id: I6d7ccbf4ef9b46e890ecec58842c5cdd2804c7a9
2. Related-Change-Id: If07af46cb377f3f3d70f6c4284037241d360a8b7
Change-Id: Ib99a029c1bd1ea1efa8060fe8a11cb01deea41c6
The SwiftService and shell support the ability to limit
deletions to only those objects that match a specified
prefix, so let's expose that (really useful) behaviour
in the command line help as well :)
Change-Id: I9ef177aa96e4829196b5200dd8e9d0d2f7f89b63
Adding constraints support to libraries is slightly more complex than
services as the libraries themselves are listed in upper-constraints.txt
which leads to errors that you can't install a specific version and a
constrained version.
This change adds constraints support by also adding a helper script to
edit the constraints to remove python-swiftclient.
Change-Id: I3947a6165eaa9f5cb62a7df4f5a2c16065da2f1d
In python swiftclient: swift <sub_command> —help will now
display st_<sub_command>_options + st_<sub_command>_help texts
e.g. http://paste.openstack.org/show/589752/
Change-Id: I34e4b2ac29ef395f8ca474ce7a82f59a1fd8c7f4
Closes-Bug: #1621415
Previously, we only accepted iterables of strings like 'Header: Value'.
Now, we'll also accept lists of tuples like ('Header', 'Value') as well
as dictionaries like {'Header': 'Value'}.
This should be more intuitive for application developers, who are
already used to being able to pass dicts or lists of tuples to libraries
like requests.
Change-Id: I93ed2f1e8305f0168b7a4bd90c205b04730da836
[H203] Use assertIs(Not)None to check for None (off by default) Unit
test assertions tend to give better messages for more specific
assertions. As a result, assertIsNone(...) is preferred over
assertEqual(None, ...) and assertIs(None, ...), and assertIsNotNone(...)
is preferred over assertNotEqual(None, ...) and assertIsNot(None,
...). Off by default.
More details, see:
http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises
Trivial fix.
Change-Id: Icd268b96dea5e5bb9bd344f597dfcd9cc82253f0
