Previously, if a Range request came back 200 OK (rather than 206 Partial
Content), we would mangle the response body. This could happen if there
was a middleware that would silently drop Range headers, for example.
Now, if the response does not include a Content-Range header, we will
log a warning and seek to our previous position in the stream. If the
Content-Range header has an unexpected value, we will raise an exception.
Change-Id: I94d4536cc1489968d45a2b6ba7edd70c85800275
Also, don't try to do int(None) for chunk-encoded responses (like DLOs
that are longer than a single container listing).
Change-Id: Ibacd75d5ee46135d62388786903c895fda8ed3ba
Currently the swift client retries establishing a connection to the server
(by default up to 5 times). However, when downloading an object, once
the connection has been established and the inital headers have been
returned, no attempt is made to retry.
So, for example, if 99MB of a 100MB object have been downloaded and the
connection is then lost, the download will fail. This patch changes the
behaviour to re-establish the connection and fetch the remaining bytes
using the 'Range' header to offset.
Data retry is not yet supported if the original request is for a subset
of the object data (ie uses the 'Range' header), or if resp_chunk_size
has not been set.
The object's etag is checked using If-Match to make sure the object data
hasn't changed since the start of the download.
Change-Id: Iab47f10081ff39f6d344dbc2479cbc3bfd1c5b29
Adds a functional test to verify change made in [1]
[1] change id I19579ed7a0181ac3f488433e7c1839f7f7a040b8
Change-Id: I45dbf66edab645e6339e67906aee5faa4fb7efbd
Currently, if you attempt to stream an upload from an iterator, as in
def data():
yield 'foo'
yield ''
yield 'bar'
conn.put_object('c', 'o', data())
... requests will faithfully emit a zero-length chunk, ending the
transfer. Swift will then close the connection, possibly (if Connection:
keep-alive was set) after attempting to parse the next chunk as a new
request.
Now, Swift will receive all of the bytes from the iterable, and any
zero-byte chunks will be ignored.
This will be fixed in requests [1], but not until an eventual 3.0.0
release.
[1] https://github.com/kennethreitz/requests/pull/2631
Change-Id: I19579ed7a0181ac3f488433e7c1839f7f7a040b8
In python-swiftclient some test cases using asserEqual(arg, None)
instead of assertIsNone(arg).assertIsNone method provides clear error message.
Change-Id: I4d673ede0965408344325c9c234c5c4b1ae4146a
Closes-Bug: #1527556
Wrong usage of "an" in the messages:
"the optional os_options paramater includes an non-empty"
"We are allowing to have an tenant_name argument"
Should be:
"the optional os_options paramater includes a non-empty"
"We are allowing to have a tenant_name argument"
Totally 2 occurrences in python-swiftclient base code.
Change-Id: I2f2f7e07432fedfee5ccb418d9505250b3fed597
Add a unit test to test the 'string' like behaviour of
get_object when it is called without resp_chunk_size set.
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: I496032a76036141d027c30b076c810b34bc6bef0
Because python creates pyc|pyo files and __pycache__
directories during tox runs, certain changes in the tree,
like deletes of files, or switching branches, can create
spurious errors.
Change-Id: Ibaac514521bab11bbf552e0310d1203230c0d984
Closes-Bug: #1368661
It was dropping warnings like "UserWarning: Providing attr without
filter_value to get_urls() is deprecated as of the 1.7.0 release and may
be removed in the 2.0.0 release. Either both should be provided or
neither should be provided."
Change-Id: Iead0bcf36b4a46bf465a55a33a21fd7f14f0ac40
As of mitaka, the infra team won't have the resources available to
reasonably test py26, also the oslo team is dropping py26 support
from their libraries. sine we rely on oslo for a lot of our work,
and depend on infra for our CI, we should drop py26 support too.
Closes-Bug: 1519510
Depends-On: I37116731db11449d0c374a6a83a3a43789a19d5f
Change-Id: I776847ce77dfe82880f34d0b7804514e5aed3f8d
New documentation for python-swiftclient that introduces
the APIs available and gives some opinionated advice about
when to use the shell, the client API and the service API.
Change-Id: I19020f041fab2e72469979f712ffe3951c431d24
According to help `swift upload -h` you can add a customized request header 'Content-Type'.
But actually it is ignored (cleared and default is used) if subcommand is upload.
Subcommand post works as expected in help.
Bug fix: Use 'Content-Type' from the customized request headers also if
uploading.
Change-Id: If0d1354b6214b909527341078fe1769aa6587457
* Always use testtools.TestCase, since we're relying on testtools
* Always use mock (as opposed to unittest.mock) since we're relying on
mock
* Add note about when a missing logging handler was added
* Stop %-formatting the giant usage string that doesn't actually need
any formatting
* Prefer assertIs, assertIn, assertIsInstance over assertTrue
* Use else-self.fail instead of sentinel values
* Check resp.get('error') is None before checking resp['success'] is
True, so test failures actually tell you something useful
* Tighten some isinstance assertions
* Import MockHttpTest from correct location
* Only populate clean_os_environ once
* Use setUp for setup, not __init__
* Replace assertIn(key, dict) and assertEqual(foo, dict[key]) with
assertEqual(foo, dict.get(key)) when key is a literal and foo is not
None
* Use mock.patch.object instead of manually patching for tests
* Use six.binary_type instead of type(''.encode('utf-8'))
* Stop shadowing builtin bytes
* Reclaim some margin
* Stop checking the return-type of encode_utf8; we already know it's
bytes
Change-Id: I2138ea553378ce88810b7353147c8645a8f8c90e
Cleanups for change I35c3b266b3c733f6b1629de4c683ea7d40128032
Add missing param definitions to client get_container
and head_object docstrings.
For consistency, add headers parameter to the Connection class
head_object and head_container wrapper methods.
Add tests to verify that the headers parameter of Connection
get_container, head_container and head_object methods is passed to the
module functions.
Change-Id: Ib40d5b626b2793840727c58cffbf725bea55651f
Some tests rely on the fake connection checking expected request
parameters, but that assumes that the fake ocnnection is even called,
which is not being checked. Add more explicit assertions that requests
are in fact made.
Change-Id: Id1c48235d7d97fd1b0feec6c19ed59a87bebdf89
Headers parameter is required when passing client key for encryption.
It is missing for get_container and head_object.
Change-Id: I35c3b266b3c733f6b1629de4c683ea7d40128032
This commit adds the utils module to those for which Sphinx
automatically generates documentation from docstrings. (Many of the
functions here may be of little interest to users, but
`generate_temp_url`, at least, definitely deserves to be in the
documentation; in this way, this commit can be seen as a spiritual
companion to ca70dd9e.)
Also, a few markup errors and perceived infelicities in existing
docstrings are amended.
Change-Id: I8c66a23cb359d7dd9302a16459fad9825fedb690
