This patch fixes downloading files to stdout and modifies
_SwiftReader to operate as an iterator that performs file
checks at the end of iteration as well as a context manager.
File verification checks have been removed from __exit__
and added to __iter__.
Change-Id: I3250bdeeef8484a9122c4b5b854756a7c8f8731e
Closes-Bug: 1395922
Closes-Bug: 1387376
I'm not sure how test_process_options_defaults ever passed for anyone that had
all the ST_(AUTH|USER|KEY) set in their environment when they ran it. Despite
our efforts to patch os.environ swiftclient.servce was acctually importing
environ from os and then sticking the results into some global state. The
tests seemed to work fine on infra, but for me locally until I `unset ST_AUTH`
the test would always fail. Seems to work now and looks fairly reasonable on
the surface.
Change-Id: Id68590d6af16040a2877e719bc38d1e126603e42
Removes an account stat from the object upload path.
This stat fails when user is not account admin even
though the user may have container ACL permission to
write objects.
Reduces the severity of the CLI output message when
upload fails to create the given container (this is
not an error since the container may exist - the user
just does not have permission to PUT or POST the
container).
Changes the 'swift upload' exit return code from 1 to
0 if container PUT fails but object PUT succeeds.
For segment uploads, makes the attempt to create the
segment container conditional on it not being the same
as the manifest container. This avoids an unnecessary
container PUT.
Fixes another bug that became apparent: with segmented
upload a container HEAD may be attempted to determine
the policy to be used for the segment container. When
this failed the result dict has headers=None which was
causing an exception in the shell result handler.
Add unit tests for object upload/download and container
list with --os-storage-url option.
Closes-Bug: #1371650
Change-Id: If1f8a02ee7459ea2158ffa6e958f67d299ec529e
Check for None was being done after a method call which
caused an attribute error if the value was None.
The Method call was also a mistake and has been corrected to the
hasattr function like intended.
Added Tests.
Closes-Bug: 1392651
Change-Id: Ifb1c84e26533bccbaddcce5738f434f36feca74e
The response dictionary in _delete_segment attach's any error
it encounters to the dict key 'exception', all other response
dict's use 'error' to store the exception.
Changed to make it consistent and added tests
Also added a third branch in st_delete as messages
created in _delete_container where being silently dropped.
Change-Id: Ifbc3b1fae78910fbc6acf4a86cfb0f60bb1aa336
actual_md5 was being used instead of _actual_md5 only when assigning
the md5 object. This resulted in all checks and updates of the md5
to be skipped.
When fixed it revealed another bug that the md5 of the read content
is always checked even if there is no _expected_etag.
Added tests
Change-Id: Iaf1e21324ce592049d02cd5be123604b99833e86
When downloading an object, the content_type header is inspected
to determine if a directory needs to be created. This header is
assumed to always be in the response; if it isn't an Exception is
raised. swiftclient should not assume content_type will always be
set.
Change-Id: I156195c02b6e0bc398fa962eb1f78c4dbddd1596
swiftclient.shell.st_post was ignoring the result dict
returned from SwiftService.post and therefore giving no
indication when posts failed e.g. due to invalid auth
credentials.
This patch ensures that SwiftService always returns a
result_dict from post() and then checks the result dict.
On failure the shell now outputs the ClientException
message and exits with error code 1.
Also adds unit tests and cleans up some unnecessary mocked
return values in existing tests.
Closes-Bug: 1383243
Change-Id: I111150eb3d026c8d01c2cac13d3613ca7304e5b9
Changed the message shown when a user doesn't enter project/tenant
id/name to be more informative.
When attempting to stat a container without supplying project/tenant
name or id, an empty response was being returned instead of an error
being raised.
Changed the error raised in swiftclient.client when no tenant or project
is specified to be more specific.
Add tests for basic regression checking.
Closes-Bug: #1372589
Change-Id: I4eb6964d9f1702db119cc0294edc02841b1ecd5f
During a segmented object upload, attempts are made
to create the container for the manifest and the
segment container. These jobs are currently placed
on separate threads and can occur out of order which
causes the unit test failure seen in the bug report.
The container jobs should be ordered because the
segment container job attempts to HEAD the manifest
container to determine correct storage policy.
This patch enforces ordering of the two jobs and
modifies the unit test to assert that both container
PUTs occur in the expected order.
Change-Id: If90bec126867a4be2df34067ccefee660202f59f
Closes-Bug: #1376878
This patch removes a debugging print statement that slipped into
service.py in patch https://review.openstack.org/#/c/85453/
Change-Id: I80eee49eaa74b52e9c36f00669206aa7b2cdec9b
Fixes a regression that is preventing swiftclient from
authenticating using keystone v3 options.
swiftclient/shell.py processes the dict of command
line options to set up a child dict ('os_options')
of keystone auth specific options (line 960). The
processing includes stripping --os- prefixes from
the command line options before adding keys to
os_options.
A recent patch https://review.openstack.org/#/c/85453/
introduced a duplication of this option processing
in service.py (line 79) which replaces the os_options
created in shell.py, but omits keystone v3 related
options. Consequently the keystone v3 options are
not being passed to the client get_auth() method.
This patch adds the keystone v3 related options to
the option processing in service.py.
For pragmatic reasons (i.e. fixing the bug quickly)
the option processing code has not been removed from
parse_args in shell.py. It is likely that the code in
parse_args is now redundant, but all code paths between
parse_args and process_options should be inspected and
test coverage added before removing that code.
Unit tests have been added in test_shell.py to verify
that command line options are correctly passed to the
client get_auth method.
The MockHttpTest class is re-used in test_shell.py,
so it is moved from test_swiftclient.py
to tests/unit/utils.py
Closes-bug: #1372465
Change-Id: I4fed013cdb8936509609d06093337cc147ade0d6
Cannot help streamlining the code a bit, sorry. But the meat is that
we should not try to make directories if output path is explicit.
Previously we created directories using the URL path, which is
obviously wrong if explicit output file is supplied... unless
a crafty user supplied the same path with -o that is contained
in the URL path. If anyone was doing such tricks, it's not going
to work anymore (we are forcing a regression for the sake of
theoretical correctness here).
Fixes bug: 1369546
Change-Id: Ifce31f2ba233eb55550f3810348bf16bf2447d62
This patch adds a SwiftService class that incorporates the high
level logic from swiftclient/shell.py. It also ports shell.py to
use the new class, and updates the code in swiftclient/multithreading.py
to allow the SwiftService to be used for multiple operations whilst
using only one thread pool.
Currently, code that imports swiftclient has to have its own logic for
things like creating large objects, parallel uploads, and parallel
downloads. This patch adds a SwiftService class that makes that
functionality available in Python code as well as through the shell.
Change-Id: I08c5796b4c01001d79fd571651c3017c16462ffd
Implements: blueprint bin-swift-logic-as-importable-library
