The "unicode" type was renamed to "str" in Python 3.
Use six.text_type to make swiftclient compatible with Python 3.
For more information about changes needed for py3
compatibility, see:
https://wiki.openstack.org/wiki/Python3
Change-Id: Ic65607a69935652a1874340928f626fbcc35c014
Previously, we compared the ETag from Swift against the MD5 of the
entire large object. However, the ETag for large objects is generally
the MD5 of the concatenation of the ETags for each segment, unless the
object is a DLO whose segments span more than one page of a container
listing. Rather than worry about ETags, just compare each chunk of the
segmented file. This allows the use of --skip-identical when uploading
SLOs and DLOs.
Additionally, there are several test-related improvements:
* The default arguments for OutputManager are now evaluated on
construction, rather than on definition, so that
TestOutputManager.test_instantiation will succeed when using nosetest
as a test runner. (See also: bug 1251507)
* An account_username option is now available in the functional tests
config file for auth systems that do not follow the account:username
format.
* CaptureOutput no longer writes to the captured stream, and
MockHttpTest now captures output. These were polluting test output
unnecessarily. (See also: bug 1201376)
Change-Id: Ic484e9a0c186c9283c4012c6a2fa77b96b8edf8a
Closes-Bug: #1201376
Closes-Bug: #1379252
Related-Bug: #1251507
This patch fixes a crash and stack trace in stat when an object
name contains non-ascii characters.
Change-Id: Ib6dc686771e593243de56cafc100b17e51d9d9d5
Closes-Bug: 1411665
This patch fixes downloading files to stdout and modifies
_SwiftReader to operate as an iterator that performs file
checks at the end of iteration as well as a context manager.
File verification checks have been removed from __exit__
and added to __iter__.
Change-Id: I3250bdeeef8484a9122c4b5b854756a7c8f8731e
Closes-Bug: 1395922
Closes-Bug: 1387376
Removes an account stat from the object upload path.
This stat fails when user is not account admin even
though the user may have container ACL permission to
write objects.
Reduces the severity of the CLI output message when
upload fails to create the given container (this is
not an error since the container may exist - the user
just does not have permission to PUT or POST the
container).
Changes the 'swift upload' exit return code from 1 to
0 if container PUT fails but object PUT succeeds.
For segment uploads, makes the attempt to create the
segment container conditional on it not being the same
as the manifest container. This avoids an unnecessary
container PUT.
Fixes another bug that became apparent: with segmented
upload a container HEAD may be attempted to determine
the policy to be used for the segment container. When
this failed the result dict has headers=None which was
causing an exception in the shell result handler.
Add unit tests for object upload/download and container
list with --os-storage-url option.
Closes-Bug: #1371650
Change-Id: If1f8a02ee7459ea2158ffa6e958f67d299ec529e
Added the ability to clear the buffers in the
CaptureOutput class so it can be easily used multiple
times in the same context manager.
Also added a option to suppress the SystemExit
associated with printing an error.
Change-Id: Ib59bbbe88256f215eed0a8ebc8282e02181d4377
This patch adds a SwiftService class that incorporates the high
level logic from swiftclient/shell.py. It also ports shell.py to
use the new class, and updates the code in swiftclient/multithreading.py
to allow the SwiftService to be used for multiple operations whilst
using only one thread pool.
Currently, code that imports swiftclient has to have its own logic for
things like creating large objects, parallel uploads, and parallel
downloads. This patch adds a SwiftService class that makes that
functionality available in Python code as well as through the shell.
Change-Id: I08c5796b4c01001d79fd571651c3017c16462ffd
Implements: blueprint bin-swift-logic-as-importable-library
Storage policy stats was not well parsed in account stat. This
patch parses the stats and print out the stats in a format like below:
$swift -A http://swift_cluster/auth/v1.0 -U test:tester -K testing stat
Account: AUTH_test
Containers: 5
Objects: 1
Bytes: 2097152
Objects in policy "golden": 1
Bytess in policy "golden": 2097152
Objects in policy "silver": 0
Bytes in policy "silver": 0
X-Timestamp: 1404697760.88809
X-Trans-Id: txec519e24b44a413abb705-0053da2dcb
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
Change-Id: I7ad0ee6d88f8393e3a93e90cd52b9b592da7072d
* On Python 3, the printer doesn't encode Unicode to utf8 anymore, since
print() expects a Unicode string.
* Update unit tests for Python 3 since repr() doesn't escape non-ASCII
characters in Unicode strings anymore:
http://legacy.python.org/dev/peps/pep-3138/
Change-Id: I89471019d691a46651312d6a49964b719192148a
xrange is not supported in python 3, so replace it with range since no
large size lists involved.
Closes-Bug: #1237717
Change-Id: I4e5e0782153d32c8beee03f7d871722ed4352859
When you stat a container or object with the verbose flag the full path of the
reousrce will be displayed with the token similarlly to how an account stat
displays the auth url and token.
* move some logic out of bin/swift.st_stat to test it
* new module swiftclient.commnad_helpers for code you want to test
* moved prt_bytes into swiftclient.utils to test it
* fixed IndexError with prt_bytes on sizes >= 1024Y
Change-Id: Iaaa96e0308b08c554205b0055b8a04de581fefa4
This patch extracts the multi-threading code from bin/swift into
swiftclient/multithreading and adds tests. In particular, this new way
of doing it (with context managers) will prevent non-daemonic threads
from wedging the process when unexpected exceptions happen.
I enabled reporting of which lines, specifically, are not covered by
unit tests (added -m option to "coverage report" in .unittests).
This patch includes a drive-by fix for uploading a segmented file with
--use-slo when that object already exists. A key of "name" was used
instead of "path", raising KeyError.
There's also another drive-by fix for uploading segmented objects with
--use-slo. Commit 874e0e4427b80e1b15b74a1557b73ba9d61443ca regressed
this by removing the capturing of thread-worker results in
QueueFunctionThread.run(). This patch restores that functionality and
the feature (uploading SLO objects).
Change-Id: I0b4f677e4a734e83d1a25088d9a74f7d46384e53
