This patch adds a small test to ensure a connection
is released after all chunks have been consumed.
It's a follow up to commit 8756591b and added to ensure
there will be no regression in the future (this test
fails also with that patch not applied).
Change-Id: I6a6fcd26879eb2070f418c8770a395ff6c30aa51
Setting timeout to a default of False in get_auth()
results in a requests timeout of 0.0 in keystoneclient,
which causes a connection failure.
This bug will cause func tests to not run when using
keystoneauth.
Added unit tests to verify correct default timeout is set
in get_auth().
Drive-by: remove what seems like a stale TODO comment
Change-Id: I17c781ce160a682b1768d315422ade0cdd2df198
In addition to removing an unnecessary dependency, this closes another
hole that was allowing raw bytes to appear in user-facing messages.
Change-Id: Ia0b76426a38e5a5c368c4c7e7ba2aef286758aca
Extends existing unit test for timeout being passed to get_auth
to cover v2.0 auth when keystone client should get the timeout
kwarg.
Related-Bug: 1447847
Change-Id: Ie9cfc86fa2156b94b45d290ac12e3f71b20d6c4f
A previous change added socket timeouts, but they weren't plumbed
through to the get_auth code path. Make sure we're passing them along
everywhere.
Original-Author: Monty Taylor <mordred@inaugust.com>
Change-Id: I398241898248e66d1f8c436c8ed2ec7a0e9387f6
Closes-bug: 1447847
The underlying requests library supports a timeout input parameter. The
other python-*client libraries support passing it in, so while working
on shade, it seemed like a great idea to be able to pass in timeout to
swiftclient too. For reference, there's a change in shade:
I095c1240693abf024bda2315dd77f4400b24a45b that shows interaction with
the other libs, and a tentative patch that would consume this one.
Change-Id: I699ebb1e092aa010af678de7ba15712da6ed5315
Currently, get_object returns a generator. This allows access to the
object's data in multiples of 'resp_chunk_size'.
This patch adds a read function to also allow accessing the object data
using read(size).
This allows, for example, the consumer of an object (where no byte range
has been specified) to read up to certain boundaries while streaming to
a new Large Object with segments of a specified size.
Reading and chunking can be safely mixed.
Related-Bug: 1367925
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: I1cfb67f45afc7015fd896f1a89bebae048871769
With MD5Sum checking being added, a concern was brought up that It was
a change with no possibility of reverting to the old behaviour.
This change adds the flag '--ignore-checksum' to the upload subcommand
allowing the checks to be turned off.
Changed occurrences of the magic string for a null md5 to use a descriptive
constant instead.
Updated Error messages generated when validation fails. They should now be more descriptive
and not output a literal newline sequence.
Change-Id: Id1756cbb6700bb7e38f0ee0e75bc535e37f777ed
Changed existing code to calculate the MD5 of the object
during the upload stream. Checks this MD5 against the etag
returned in the response.
An exception is raised if they do not match.
Closes-Bug: 1379263
Change-Id: I6c8bc1366dfb591a26d934a30cd21c9e6b9a04ce
If you specify a token and storage url when creating a Connection,
regardless of the auth api version the first request will be made
directly to swift. You can either provide a preauthurl and preauthtoken
or fall back to os_options' object_storage_url and auth_token keys
(exposed as --os-storage-url and --os-auth-token on the command line or
OS_STORAGE_URL and OS_AUTH_TOKEN in the environment).
If a _retry wrapped request on a Connection fails because of invalid
authentication (401) the Connection's cached token and url will be
invalidated. If the Connection's retries attribute is > 0 the
subsequent attempt will call get_auth to refresh the token, but the
pre-configured storage_url will always be re-used. This is consistent
with current auth v2 behavior and less surprising for auth v1.
The pre-existing, but previously undocumented behavior/interface of
get_auth would override the storage_url returned by the auth service if
the 'os_storage_url' option was provided in the os_options dict. To
ensure that this behavior is consistent across auth v1 and v2 from the
command line and when using the Connection class as a library - the
preauthurl is stashed in the os_options dict when provided.
Improved Connection.get_capabilities storage_url handling to better
support the consistent behavior of a preauthurl/object_storage_url on
the connection regardless of auth version.
Fixed up some test infrastructure to enable setting up and testing
multiple requests/responses.
Change-Id: I6950fb73f3e28fdddb62760cae9320e2f4336776
Some client.Connection methods will raise a KeyError
if a response_dict argument is passed and an error
occurs during authentication or making the request.
The fix is straightforward: add a test for existence
of a response_dict before attempting to get it from
kwargs.
The bulk of this patch is adding unit tests for the
response_dict feature.
Closes-Bug: 1381304
Change-Id: Ic7e1b3dfae33909533931c52ac97355867a08a07
Fixes a regression that is preventing swiftclient from
authenticating using keystone v3 options.
swiftclient/shell.py processes the dict of command
line options to set up a child dict ('os_options')
of keystone auth specific options (line 960). The
processing includes stripping --os- prefixes from
the command line options before adding keys to
os_options.
A recent patch https://review.openstack.org/#/c/85453/
introduced a duplication of this option processing
in service.py (line 79) which replaces the os_options
created in shell.py, but omits keystone v3 related
options. Consequently the keystone v3 options are
not being passed to the client get_auth() method.
This patch adds the keystone v3 related options to
the option processing in service.py.
For pragmatic reasons (i.e. fixing the bug quickly)
the option processing code has not been removed from
parse_args in shell.py. It is likely that the code in
parse_args is now redundant, but all code paths between
parse_args and process_options should be inspected and
test coverage added before removing that code.
Unit tests have been added in test_shell.py to verify
that command line options are correctly passed to the
client get_auth method.
The MockHttpTest class is re-used in test_shell.py,
so it is moved from test_swiftclient.py
to tests/unit/utils.py
Closes-bug: #1372465
Change-Id: I4fed013cdb8936509609d06093337cc147ade0d6
Enables swiftclient to authenticate using
the keystone v3 API, allowing user id's, user
domains and tenant/project domains to be
specified.
Since swiftclient imports keystoneclient, the
main changes in swiftclient/client.py are to
selectively import the correct keystoneclient
library version and pass a number of new
options to it via the get_auth() function. In
addition the get_keystoneclient_2_0 method
has been renamed get_auth_keystone to better
reflect its purpose since it now deals with
both v2 and v3 use cases.
In swiftclient/shell.py the new options are
added to the parser. To make the default help
message shorter, help for all the --os-*
options (including the existing v2 options)
is only displayed when explicitly requested
usng a new --os-help option.
A new set of unit tests is added to
test_shell.py to verify the parser. A comment
in tests/sample.conf explains how to
configure the existing functional tests to
run using keystone v3 API.
Note that to use keystone v3
with swift you will need to set
auth_version = v3.0 in the auth_token
middleware config section of
proxy-server.conf.
Change-Id: Ifda0b3263eb919a8c6a1b204ba0a1215ed6f642f
get_auth() in client.py raises an exception if
tenant_name is not included in the os_options
dict. This is overly constrained since tenant_id
is equally sufficient. This patch modifies
get_auth to require either tenant_name
or tenant_id.
Change-Id: Ibbcda1704637eb887efa5895579d260a1e072327
A couple of assertTrue in the test_swiftclient.py
unit tests should be assertEqual. Also, the expected
values now need to be bytes literals.
Change-Id: I7cc1bd60d9ba82d1a28fbae2e1243d3c799451bd
This patch fixes three issues that were found running
functional tests with Python 3:
1. python-requests sets 'application/x-www-form-urlencoded' as
content-type if the input is not a string object and no
content-type is given.
2. Encoding of the headers is now only done if required. This
prevents comparisons between unencoded headers and encoded
prefixes and avoids unnecessary forloop-iterations.
One unittest was extended to ensure it works for unencoded
and encoded headers with or without the prefix.
3. Functional tests recently switched to using byte data for
testing, thus the comparison needs to be a byte object as well.
Change-Id: I035f8b4b9c9ccdc79820b907770a48f86d0343b4
There is no testtools.main() at all (this was a
unittest.main() before). Let's remove this unused
and non-working code.
The following code can be used if someone needs
to start tests manually:
python -m unittest <test_file.py>
Change-Id: Id5162ac73825584df6c23dbe68786ff3355fc6ae
Don't encode standard headers keys but just the x-*-meta- ones. That
fixes py3 to not have them casted as bytes and not properly passed to
the requests kwargs.
Other trivial py3 fixes along the way.
Change-Id: I91b95f32fb2aec9b20892a5bb95fd1bc65d002f3
Coverage for swiftclient.client is 71% with these tests.
Unit tests have been moved into another subdirectory
to separate them from functional tests.
Change-Id: Ib8c4d78f7169cee893f82906f6388a5b06c45602
