Now, client applications can get to things like transaction IDs for
failures without needing to turn on all of logging.
While we're at it, add a from_response factory method for
ClientException.
Co-Authored-By: Alexander Corwin <ancorwin@gmail.com>
Change-Id: Ib46d5f8fc7f36f651f5908bb9d900316fdaebce3
My understanding is that it was mainly being used so we could have sane
testing on py26. With py26 support being dropped, we no longer need it.
Also drop discover from test-requirements.txt, as we don't seem to
actually use it.
Change-Id: Iee04c42890596d3b483c1473169480a3ae19aac8
Related-Change: I37116731db11449d0c374a6a83a3a43789a19d5f
Add support for the auth-version to be specified using
--os-identity-api-version or OS_IDENTITY_API_VERSION
for compatibility with other openstack client command
line options.
The auth version used will be selected as follows:
- if either --auth-version or --os-identity-api-version
is set, use that value
- otherwise use the value of ST_AUTH_VERSION, if set
- otherwise use the value of OS_AUTH_VERSION, if set
- otherwise (new behaviour) use the value of
OS_IDENTITY_API_VERSION, if set
- otherwise default to 1.0
Note that before this change the auth version might have
defaulted to 1.0 despite OS_IDENTITY_API_VERSION being set,
but with this change OS_IDENTITY_API_VERSION is preferred.
Change-Id: Ifba4c4e43560ede3013337b8cdbc77dc2de6e8ff
Closes-Bug: #1541273
This fix ensures creation and use of the correct default segment
container when pseudo-folder paths are passed via <container> arg.
Change-Id: I90356b041dc9dfbd55eb341271975621759476b9
Closes-Bug: 1532981
Related-Bug: 1478210
The test asserts calls made in specific order,
but they are made from threads so may be in
different order.
Change-Id: I1b6e7303fe0e6fb2afc7da3462b891feab90bc17
Closes-Bug: #1539536
The test asserts calls made in specific order,
but they are made from threads so may be in
different order.
Change-Id: I857ad3b909c3b635927fb1a39682d66d20c6fd59
Previously, the following empty directories would be created:
* container
* container/pseudo
* pseudo
Change-Id: I002e2da8d28a873728e0b5c2d33f94f21132d058
When issuing `delete` commands that would require three or more
individual deletes, check whether the cluster supports bulk deletes
and use that if it's available.
Additionally, a new option is added to the `delete` command:
* --prefix <prefix>
Delete all objects that start with <prefix>. This is similar to the
--prefix option for the `list` command.
Example:
$ swift delete c --prefix obj_prefix/
...will delete from container "c" all objects whose name begins with
"obj_prefix/", such as "obj_prefix/foo" and "obj_prefix/bar".
Change-Id: I6b9504848d6ef562cf4f570bbcd17db4e3da8264
This fix makes it possible to upload objects to pseudo-folders by
passing the upload paths via <container> arg regardless of whether the
container or folder path exist or not.
Change-Id: I575e58aa12adcf71cdaa70d025a0ea5c63f46903
Closes-Bug: #1478210
Partial-Bug: #1432734
Related-Bug: #1432734
The debug and info options need to be set before a subcommand method is called,
otherwise they are simply ignored. This is kind of irritating - other options
(for example -U, -A, -K) are usable after a positional command.
This patch fixes this, and commands like these are no longer ignoring --debug or --info:
swift stat --debug
swift list container --info
Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
Change-Id: Ib19b05deef7a015881f1eed4a3946025e16bf922
It was dropping warnings like "UserWarning: Providing attr without
filter_value to get_urls() is deprecated as of the 1.7.0 release and may
be removed in the 2.0.0 release. Either both should be provided or
neither should be provided."
Change-Id: Iead0bcf36b4a46bf465a55a33a21fd7f14f0ac40
* Always use testtools.TestCase, since we're relying on testtools
* Always use mock (as opposed to unittest.mock) since we're relying on
mock
* Add note about when a missing logging handler was added
* Stop %-formatting the giant usage string that doesn't actually need
any formatting
* Prefer assertIs, assertIn, assertIsInstance over assertTrue
* Use else-self.fail instead of sentinel values
* Check resp.get('error') is None before checking resp['success'] is
True, so test failures actually tell you something useful
* Tighten some isinstance assertions
* Import MockHttpTest from correct location
* Only populate clean_os_environ once
* Use setUp for setup, not __init__
* Replace assertIn(key, dict) and assertEqual(foo, dict[key]) with
assertEqual(foo, dict.get(key)) when key is a literal and foo is not
None
* Use mock.patch.object instead of manually patching for tests
* Use six.binary_type instead of type(''.encode('utf-8'))
* Stop shadowing builtin bytes
* Reclaim some margin
* Stop checking the return-type of encode_utf8; we already know it's
bytes
Change-Id: I2138ea553378ce88810b7353147c8645a8f8c90e
Attempt to fix the linked bug by initialising the mock instance for
Connection.delete_object before calling the SwiftService upload
method, so that the delete_object mock already exists before the
delete_segments jobs that run in multiple threads call it. Otherwise
there is a risk that the delete_segment job threads could race while
creating either the delete_object mock or the
delete_object.return_value mock, resulting in each thread getting a
different instance. That would explain the intermittent test failures
reported in the bug.
Change-Id: Ia82697c093529076b0bbcc6bccd577afdf0839e1
Partial-Bug: #1480223
The `tempurl` subcommand's second positional argument is called
`seconds` and has heretofore interpreted as the number of seconds for
which the tempURL should be valid, counting from the moment of running
the command. This is indeed a common, if not the most common,
use-case. But some users, occasionally, might want to generate a tempURL
that expires at some particular ("absolute") time, rather than a
particular amount of time relative to the moment of happening to run the
command. (One might make an analogy to the way in which Swift's expiring
object support supports an `X-Delete-At` header in addition to
`X-Delete-After`—and it's the former that must be regarded as
ontologically prior.) Thus, this commit adds an `--absolute` optional
argument to the `tempurl` subcommand; if present, the `seconds` argument
will be interpreted as a Unix timestamp of when the tempURL should be
expire, rather than a duration for which the tempURL should be valid
starting from "now".
Change-Id: If9ded96f2799800958d5063127f3de812f50ef06
Some improvements to the tests for staggered download
that were added in [1].
[1] Ie737cbb7f8b1fa8a79bbb88914730b05aa7f2906
Change-Id: Ib999bc7bd198c1d9c217c57501f751e854d4c6ad
fix against H234: assertEquals() logs a DeprecationWarning
in Python3.x. use assertEqual() instead.
Closes-bug: #1480776
Change-Id: Iffda6bb5f2616d4af4567eeea37bb26531e34371
Signed-off-by: Hiroshi Miura <miurahr@nttdata.co.jp>
The current code builds a full object listing before performing either a multiple
download or delete operation (and also shuffles this complete list in the case of
a download). This patch removes the creation of the full object list and adds the
ability to turn off shuffle for files when downloading. Also added is a limit on
the number of list results that can be queued by a single call to service.list
without consuming results (reduces memory overhead for large listings).
Some tests added for service.py download and list.
Change-Id: Ie737cbb7f8b1fa8a79bbb88914730b05aa7f2906
There was always an implicit assumption that they were UTF-8 before, and
by converting them to unicode we close another hole allowing raw bytes
to appear in user-facing messages.
Closes-Bug: #1431866
Change-Id: If2e41d9a592c3ad02818e9c6f0959fd4b73cd0e0
Currently SwiftClient populate response dictionary before
removing "./" or "/" at begining of object name. This patch
fixes that by changing that order.
Closes-bug: #1412425
Change-Id: I80222754caba5d42a468f4677ac539e46682dd31
In addition to removing an unnecessary dependency, this closes another
hole that was allowing raw bytes to appear in user-facing messages.
Change-Id: Ia0b76426a38e5a5c368c4c7e7ba2aef286758aca
Extends existing unit test for timeout being passed to get_auth
to cover v2.0 auth when keystone client should get the timeout
kwarg.
Related-Bug: 1447847
Change-Id: Ie9cfc86fa2156b94b45d290ac12e3f71b20d6c4f
Previously, only the first container-listing's worth of segments was
deleted, which would leave behind orphaned segments when the object was
very large with small segments or the server's container_listing_limit
was small.
In addition, process DLO and SLO deletions on the segment thread pool,
rather than the object thread pool.
Change-Id: I1587375261a6237fa55a9cb96bda8dae918cc795
Related-Bug: #1418007
With MD5Sum checking being added, a concern was brought up that It was
a change with no possibility of reverting to the old behaviour.
This change adds the flag '--ignore-checksum' to the upload subcommand
allowing the checks to be turned off.
Changed occurrences of the magic string for a null md5 to use a descriptive
constant instead.
Updated Error messages generated when validation fails. They should now be more descriptive
and not output a literal newline sequence.
Change-Id: Id1756cbb6700bb7e38f0ee0e75bc535e37f777ed
This patch adds unit tests to verify that
uploads overwriting an existing DLO manifest
will delete any segments pointed to by the
manifest, unless --leave-segments is specified.
Also test the --leave-segments option with an
existing SLO manifest.
Related-Bug: 1418007
Change-Id: I54e8eb138976269bb36392a811089c33c1ace4d3
