Allow the ability to download a subset of containers (--all with -p) or
a subset of objects within a container (container name with -p).
This patch also includes a drive-by fix for "download --all" which would
not actually download any objects (for me, at least) because the object
queue got filled with "stop" messages before the container workers had
run long enough to put work in the object queue. Doh!
I also closed up a few holes where an (unexpected, obviously) Exception
could cause the process to hang because non-daemon threads still
existed.
Change-Id: I71c6935c60282b5353badc2dfce8a935d47e3bb7
Allows optionally disabling SSL compression. This can significantly
improve HTTPS upload/download performance in some cases -- in particular
when the object is not compressible and you have very high network
bandwidth.
Implements blueprint ssl-compression.
Change-Id: I1260055f9c2e83cdabfeb51aed11b3899bed4d55
The current implementation uses busy-wait and checking flags to
communicate between threads. It wastes a lot of CPU powers. With
python 2.6 is required for Swift, the communication between
threads should now be using queue and signal mechanisms. This
patch removed the busy-wait loops and use queue and queue join
for threads coordination which will not consume CPU cycles if
a thread is blocked.
Change-Id: I648cd637a92a159d5c13baa83f357cee2dfe7937
Some mechancical replacement of the deprecated except x,y:
construct with except x as y, which works with any Python
version >= 2.6
Change-Id: Ic245049dc7b408a5c89b9e27dfd2bd7c08acc5b5
Add to the help strings to clarify the need for --os-auth-token to
also have --os-storage-url supplied.
Change-Id: I6a7259c38ce01813ae96f307efaf46e7863bd223
Also fixed bug with current large objects with segment listing prefixes.
Allow retry for object PUTs when possible.
Change-Id: I0edff127fd5d5c53da33aa7cb76a4f4dc85bf6e6
Currently swift client does not allow content type to be specified
when upload an object. It does not allow a user to change content
type neither. This patch will allow a user to specify content
type by using upload command or modify the content type by using
post command.
For example:
1. To upload an object with specific content type:
swift upload -H content-type:applicaiton/xml con1 file1
2. To modify an object to a specific content type:
swift post -H content-type:application/josn con1 file1
This patch also allow a client to specify other request headers.
Change-Id: I12db83a1d465285e0906889cc67dfaa44e059568
Fixes: bug #1154621
Fix bug 1110924
Add segment container option as part of 'swift upload -S'. If this
option is omitted, the segments will go to <container>_segments.
Example:
$ swift upload c1 obj1 -S 1 -C user_segments
Please check swift upload --help for more details
Change-Id: Ib71aae322485d8d3ac89916d37ebcac053f49e3b
Add env[SWIFTCLIENT_INSECURE] as default of --insecure option.
If set to 'true', allow to access insecure keystone server.
The name follows 'NOVACLIENT_INSECURE' in novaclient.
Change-Id: I322674eba9c07e6def97bce339815fa15191a92d
- Remove PYTHON_SWIFTCLIENT and use --debug making it more consistents
with other openstack clients libraries.
- Fix printing the curl command.
- Don't show the body of the GET objects it could get bad on large
objects and it's an iterator anyway.
- Use -I for showing HEADs (and not -X HEAD).
Change-Id: I954e7d3d795401fc3679725440dec36cdc80af87
Add support to specify a ca certificate bundle to verify
keystone TLS (https) certificates. This only verifies
certificates on the keystone connection, swift https
connections are unchanged.
Change-Id: I14351b405af4fd3d1970ba6656c1282a5d0a1082
If enable this option , swift CLI is allowed to access a keystone
server with self signed certificate.
Change-Id: I5e219fe875b246b68ac51a077e7ff15e95463adf
Adding a trailing slash to the authentication URL may interfere with
auth service implementations not using /FOO/ but /FOO as authentication
URL.
OpenStack Object Storage API 1.0 doesn't specify that the auth URL must
end in a slash and swiftclient.client doesn't force it either.
Change-Id: I4e6b6d758d5ccc884e253880513e3d8763ffc2ff
Fixes: bug #1085827
str(float) isn't a good way of converting a float to a string with all the precision
Eg
>>> f = os.path.getmtime("z")
>>> f
1347717491.343554
>>> str(f)
'1347717491.34'
>>> "%f" % f
'1347717491.343554'
Change-Id: I6eb02f7f6730eff814c348d1039eae4606642b52
Add --os-region-name (and OS_REGION_NAME env) to bin/swift
Add region_name to the os_options in Connection class.
bug 1019832
Change-Id: Id8515d97e5351638dce10581c7368f61518e1fa7
This patch adds a "--no-download" option to the "download" command.
When given, all writing to disk is bypassed, while still actually
downloading the data and validating etags.
This can be handy when you're using the swift command-line client to
test out a swift cluster and don't want client-side disk writing to be a
bottle-neck (but you still want to know about any etag validation
failures).
Change-Id: I0a511f473a64820161d1eb529b995900742794f2
This changes every command-line option with a '_' in its name
and changes them to '-'. The old option names are maintained
for backward compatibility but are no longer in the help text.
BP command-options
Change-Id: I79b3c03f59ce8e253aa0dcbf0c2ed5a56d71cd0c
Description: The swift command tool will set the auth version
to 2 if OS_AUTH_URL is set even use -V 1 option to set Version
to 1.So when use nova/glance client and swift client in the
same shell, and export environment OS_AUTH_URL, it will lead
swift client to raise 400 error if swift not use keystone
for auth.
Fixes bug 1034158
Change-Id: I8003ff2ad4ac25fd710f87c4dab1507f6040ed3d
When downloading the same containers or objects with multiple
invocations of the swift command-line client, you'll get better
throughput and avoid "hot spots" if each client randomizes its download
order.
Note that the marker must be picked *before* shuffling the containers or
objects.
Change-Id: I7240eda57a80e1708c2483827c6329fd57d5fc51
When using the swift command-line tool to evaluate a Swift cluster, it
can be very handy to get some insight into the download timing. This
patch adds timing data to verbose output for the download command. For
each downloaded file, the printed line will also contain:
- The time it took to send the request and receive the header
- The total time the request took (including writing the file out
locally)
- The average throughput of the download
Change-Id: Ib4a995623af973bb1eed4fb52c8c0e5da935964d
Fix race condition in _delete_container() where all elements of
object_queue have been removed, but the last one (per thread) may not
have actually been deleted yet when the container deletion thread calls
conn.delete_container(container). Fixes bug 1032879.
Improves container deletion throughput by immediately deleting
containers with no objects instead of waiting for all pending object
deletes to complete. Fixes bug 1032878.
Change-Id: I404229a4c608995294e0ada77724ac8afe8d6f3c
- This allows us to delegate all 2.0 authentication directly to the
library without reimplementing ourselves.
- Support reusing a token / storage-url without re-authenticating every
time via the switch os_storage_url os_auth_token.
- Allow auth via tenant_id instead of just tenant_name via the switch
os_tenant_id.
- Refactor a bit to make it easier in the future to add new OS features
(i.e: region).
- Implements blueprint use-keystoneclient-for-swiftclient.
- Fixes bug 1016641.
Change-Id: I532f38a68af884de25326aaac05a2050f5ffa1c7
The 'delete', 'download', and 'upload' commands use multiple threads
for concurrency. However, the number of threads was hardcoded at
10. This patch simply makes those configurable.
For example, if I'm downloading a lot of files but I don't want to
saturate the downstream on my Internet connection, I might choose to
use only 1 or 2 threads for object downloads. Conversely, if I'm
uploading a lot of small files across a fast network, I would want
lots of threads to speed things along.
The default number of threads is 10, so the default behavior is
unchanged.
Change-Id: I64c06741b24ca97fef5ded206d7e898bf5cab3b8
