This patch fixed help message for "<subcommand> --help" in
general help message. Previous help message was only
"this help message". It has not described for subcommand.
Change-Id: I2fe6413baf98ba3881425ee6bed29c618e29fff8
Closes-Bug: 1420253
This patch fixes a bug that results in SLO segments being left
behind regardless of the setting of --leave-segments.
Change-Id: I2b3c8ab4817df6facb4b5961dbe570645441a552
Closes-Bug: 1418007
This patch fixes downloading files to stdout and modifies
_SwiftReader to operate as an iterator that performs file
checks at the end of iteration as well as a context manager.
File verification checks have been removed from __exit__
and added to __iter__.
Change-Id: I3250bdeeef8484a9122c4b5b854756a7c8f8731e
Closes-Bug: 1395922
Closes-Bug: 1387376
I'm not sure how test_process_options_defaults ever passed for anyone that had
all the ST_(AUTH|USER|KEY) set in their environment when they ran it. Despite
our efforts to patch os.environ swiftclient.servce was acctually importing
environ from os and then sticking the results into some global state. The
tests seemed to work fine on infra, but for me locally until I `unset ST_AUTH`
the test would always fail. Seems to work now and looks fairly reasonable on
the surface.
Change-Id: Id68590d6af16040a2877e719bc38d1e126603e42
Removes an account stat from the object upload path.
This stat fails when user is not account admin even
though the user may have container ACL permission to
write objects.
Reduces the severity of the CLI output message when
upload fails to create the given container (this is
not an error since the container may exist - the user
just does not have permission to PUT or POST the
container).
Changes the 'swift upload' exit return code from 1 to
0 if container PUT fails but object PUT succeeds.
For segment uploads, makes the attempt to create the
segment container conditional on it not being the same
as the manifest container. This avoids an unnecessary
container PUT.
Fixes another bug that became apparent: with segmented
upload a container HEAD may be attempted to determine
the policy to be used for the segment container. When
this failed the result dict has headers=None which was
causing an exception in the shell result handler.
Add unit tests for object upload/download and container
list with --os-storage-url option.
Closes-Bug: #1371650
Change-Id: If1f8a02ee7459ea2158ffa6e958f67d299ec529e
Modify two tests to use the CaptureOutput class. These
tests were added after the comprehensive transition to using
CaptureOutput made in change [1], so this is just bringing
them in line with that test pattern.
Also deletes an unused mock.
[1] change id Ib59bbbe88256f215eed0a8ebc8282e02181d4377
Change-Id: Ic524311ffb3b0d6566addec0676633ddb8075e25
If you specify a token and storage url when creating a Connection,
regardless of the auth api version the first request will be made
directly to swift. You can either provide a preauthurl and preauthtoken
or fall back to os_options' object_storage_url and auth_token keys
(exposed as --os-storage-url and --os-auth-token on the command line or
OS_STORAGE_URL and OS_AUTH_TOKEN in the environment).
If a _retry wrapped request on a Connection fails because of invalid
authentication (401) the Connection's cached token and url will be
invalidated. If the Connection's retries attribute is > 0 the
subsequent attempt will call get_auth to refresh the token, but the
pre-configured storage_url will always be re-used. This is consistent
with current auth v2 behavior and less surprising for auth v1.
The pre-existing, but previously undocumented behavior/interface of
get_auth would override the storage_url returned by the auth service if
the 'os_storage_url' option was provided in the os_options dict. To
ensure that this behavior is consistent across auth v1 and v2 from the
command line and when using the Connection class as a library - the
preauthurl is stashed in the os_options dict when provided.
Improved Connection.get_capabilities storage_url handling to better
support the consistent behavior of a preauthurl/object_storage_url on
the connection regardless of auth version.
Fixed up some test infrastructure to enable setting up and testing
multiple requests/responses.
Change-Id: I6950fb73f3e28fdddb62760cae9320e2f4336776
Check for None was being done after a method call which
caused an attribute error if the value was None.
The Method call was also a mistake and has been corrected to the
hasattr function like intended.
Added Tests.
Closes-Bug: 1392651
Change-Id: Ifb1c84e26533bccbaddcce5738f434f36feca74e
The response dictionary in _delete_segment attach's any error
it encounters to the dict key 'exception', all other response
dict's use 'error' to store the exception.
Changed to make it consistent and added tests
Also added a third branch in st_delete as messages
created in _delete_container where being silently dropped.
Change-Id: Ifbc3b1fae78910fbc6acf4a86cfb0f60bb1aa336
Added the ability to clear the buffers in the
CaptureOutput class so it can be easily used multiple
times in the same context manager.
Also added a option to suppress the SystemExit
associated with printing an error.
Change-Id: Ib59bbbe88256f215eed0a8ebc8282e02181d4377
Replace URLs for workflow documentation to appropriate parts of the
OpenStack Project Infrastructure Manual.
Change-Id: I7513a141c95f3126ad348ed679b2baa335bdf45f
Swiftclient hangs forever without any reasonable hint if someone
uses a recent version of swiftclient together with auth version
2 or later. The reason for this is the sys.exit() call in a thread
- the main thread never exists, and swift just waits forever.
Raising a ClientException instead of calling sys.exit() fixes this
and returns the desired information back to the user.
Change-Id: I1fdff85da2dd4b883d337d6098b1695e448d6f2b
This change adds some new classes to test.unit.utils which wrap up the
collection and mocking of the std err and std out. This will make the
testing code easier to write and maintain.
Change-Id: I50ad2a736b2bb550ab83f6a43fb5a0fb5393573e
keystoneclient isn't adding a NullHandler to its logger,
so we see 'No handlers could be found for logger...'
messages when using the swift cli with keystone auth.
Since keystoneclient follows the usual pattern of
naming loggers from module name, we can add a
NullHandler to the 'keystoneclient' logger to handle
any logging from the library.
Note that this makes no change to keystoneclient
logging being emitted when using 'swift --debug'
or 'swift --info'.
No test because I am not sure how to usefully
test this without ending up just testing a mock
of keystoneclient.
Change-Id: Ic81cf8e5a590ae9224d35e3a5b1c5f2a8c33835a
actual_md5 was being used instead of _actual_md5 only when assigning
the md5 object. This resulted in all checks and updates of the md5
to be skipped.
When fixed it revealed another bug that the md5 of the read content
is always checked even if there is no _expected_etag.
Added tests
Change-Id: Iaf1e21324ce592049d02cd5be123604b99833e86
When downloading an object, the content_type header is inspected
to determine if a directory needs to be created. This header is
assumed to always be in the response; if it isn't an Exception is
raised. swiftclient should not assume content_type will always be
set.
Change-Id: I156195c02b6e0bc398fa962eb1f78c4dbddd1596
swiftclient.shell.st_post was ignoring the result dict
returned from SwiftService.post and therefore giving no
indication when posts failed e.g. due to invalid auth
credentials.
This patch ensures that SwiftService always returns a
result_dict from post() and then checks the result dict.
On failure the shell now outputs the ClientException
message and exits with error code 1.
Also adds unit tests and cleans up some unnecessary mocked
return values in existing tests.
Closes-Bug: 1383243
Change-Id: I111150eb3d026c8d01c2cac13d3613ca7304e5b9
Changed the message shown when a user doesn't enter project/tenant
id/name to be more informative.
When attempting to stat a container without supplying project/tenant
name or id, an empty response was being returned instead of an error
being raised.
Changed the error raised in swiftclient.client when no tenant or project
is specified to be more specific.
Add tests for basic regression checking.
Closes-Bug: #1372589
Change-Id: I4eb6964d9f1702db119cc0294edc02841b1ecd5f
Some client.Connection methods will raise a KeyError
if a response_dict argument is passed and an error
occurs during authentication or making the request.
The fix is straightforward: add a test for existence
of a response_dict before attempting to get it from
kwargs.
The bulk of this patch is adding unit tests for the
response_dict feature.
Closes-Bug: 1381304
Change-Id: Ic7e1b3dfae33909533931c52ac97355867a08a07
SysExit is raised if the OutputManager has non-zero
error_count. However, error_count is incremented in
a print thread, but tested on the main thread.
Currently error_count is tested before the OutputManager
print threads have exited, which results in a race. This race
means that SysExit is not always raised even when an error
has occured.
This change moves the error_count test after the
OutputManager context manager has exited, which ensures that
the OutputManager threads have exited.
Change-Id: I5ef7d9f10057fe2b41f48ab95066a7265b86a3ac
Closes-Bug: #1379229
During a segmented object upload, attempts are made
to create the container for the manifest and the
segment container. These jobs are currently placed
on separate threads and can occur out of order which
causes the unit test failure seen in the bug report.
The container jobs should be ordered because the
segment container job attempts to HEAD the manifest
container to determine correct storage policy.
This patch enforces ordering of the two jobs and
modifies the unit test to assert that both container
PUTs occur in the expected order.
Change-Id: If90bec126867a4be2df34067ccefee660202f59f
Closes-Bug: #1376878
We forgot to propagate options into http_connection() for the info
aka capabilities command. Fix is to use the proper helper.
Fixes bug: 1381866
Change-Id: I9515203c969a1bba38dd909412355080383905f9
Instead of always specifying segment size in bytes the user
can now use B,K,M or G as suffixes for the corresponding
size.
Conversion is done with Binary units (1024) rather than SI
units (1000).
e.g
swift upload test_container -S 1073741824 large_file
can now be written
swift upload test_container -S 1G large_file
The change is backwards compatible as it ignores arguments to -S
that don't have a valid suffix.
Updated unit tests and help message.
Change-Id: I6314b4e45cf2fbffde2fe57a02df77a25e911e84
SkipTest is raised if there is no func_test section in the Swift test
config file. However, the currently raised Exception will result in a
failed test, not in a test marked as skipped.
Since we already use testtools we can use the included skipTest easily.
Change-Id: I1bbb1f9dbe31fb0698d774550708d1196b266625
If 'x-timestamp' header is not present in 'swift list' result,
date string cannot be correctly generated. Check for correct
exception to catch this case and return a special string value.
This is just a workaround for faulty RadosGW behavior.
Change-Id: I172843a2668e33efac00924906ff0dca3e790050
Closes-Bug: 1349528
This patch removes a debugging print statement that slipped into
service.py in patch https://review.openstack.org/#/c/85453/
Change-Id: I80eee49eaa74b52e9c36f00669206aa7b2cdec9b
tests/unit/test_shell.py:TestParsing tests can fail if
there are OS_* variables set in the environment. There
is already code in the test setUp to remove ST_* variables,
so we should do the same for OS_* variables.
This patch also changes the mechanism used to remove and
then restore any unwanted variables found in os.environ.
The existing setUp() takes a copy of os.environ and then
deletes any ST_* variables in the original. In tearDown() it
sets os.environ as the copy. However, the environ imported
into shell.py remains pointing to the original os.environ
object. So after the first call to tearDown, subsequent
mocking of os.environ has no effect on shell.environ.
This renders some of the tests ineffective e.g.
test_insufficient_env_vars_v3 is not actually setting any
vars in shell.environ.
The issue can be provoked by repeating a test:
nosetests -w tests/unit/ test_shell.py:TestParsing.test_args_v3 \
test_shell.py:TestParsing.test_args_v3
The test will pass first time and fail second time.
Change-Id: I5d100f81115e74878d510326acb5777e6a3626c8
