83 Commits

Author SHA1 Message Date
Jenkins
20cd3402b2 Merge "Add --object-name" 2013-12-25 11:20:10 +00:00
Taurus Cheung
26ecec0f6d Add --object-name
Add parameter --object-name, which:
1) Sets target object name when upload single file
2) Sets object prefix when upload a directory

Change-Id: Idc4357c3355e66d31c100540b901e70db20b03c3
Closes-Bug: 1012979
2013-12-24 10:27:17 +08:00
Sahid Orentino Ferdjaoui
222f283b22 Fix help of some optional arguments
Some positional arguments are optional but the help indicate
they are mendatory and can be confuse for an user.

  swift delete <container> [object]
  swift post <container> [object]

Change-Id: I7be525905b51d3ef160b0f47194e817b5cba0f9b
2013-12-18 10:12:22 +00:00
jola-mirecka
9b7e24c449 Fix download bandwidth for swift command.
We noticed a difference in download bandwidth while using
authentication details and pre-auth information for download.
Download using authentication details included authentication
time into the download bandwidth at each call of the function.
This time should have been excluded from download bandwidth.

This patch set is adding a timer variable to the client library.
That allows to accurately measure and store the time of each action.
Then while printing the download bandwith in the swift command
authentication time is excluded from bandwidth.

Change-Id: I63df9023e169f637f120ae9e25dac9c90a4e75a0
Fixes: bug #1197443
2013-11-28 13:02:54 -05:00
Jenkins
a253b5cecd Merge "Allow custom headers when using swift download (CLI)" 2013-11-27 10:16:55 +00:00
Jenkins
04e0cb2783 Merge "Add verbose output to all stat commands" 2013-11-21 08:50:33 +00:00
Matthieu Huin
3cabda8c48 Allow custom headers when using swift download (CLI)
This fixes bug #1051046

A repeatable option, --header or -H, is added so a user can specify
custom headers such as Range or If-Modified-Since when downloading
an object with the swift CLI.

Change-Id: I1f7dcf64cf625f2e5a4488c210894cfe6e0d5974
2013-11-04 17:41:55 +01:00
Romain Hardouin
4b78b4d136 Fix a typo in help text: "downlad"
Change-Id: Icd6cf75f3c5758c13118060c19c9c000e28ef3ae
Closes-Bug: #1245082
2013-10-27 01:41:04 +02:00
Jenkins
775a24b1bc Merge "enhance swiftclient logging" 2013-10-18 21:43:05 +00:00
Leah Klearman
b682935055 enhance swiftclient logging
* log to debug for successes and info for errors
* remove dead code, neither body or raw_body are sent in kwargs
* include resp.reason and response headers in logging
* remove trailing \n in logging. users who want them can set logging.basicConfig(format=FORMAT)
* add --info option to swift cli

Change-Id: If07af46cb377f3f3d70f6c4284037241d360a8b7
2013-10-17 16:05:36 -07:00
Jenkins
1f2717e334 Merge "Extend usage message for swift download" 2013-10-17 17:42:42 +00:00
Jenkins
efde2378c2 Merge "user defined headers added to swift post queries" 2013-10-17 16:59:03 +00:00
Pete Zaitcev
e08ec3ab8a Clarify main help for post subcommand
Fixes bug 1231321

Change-Id: I7014aa160ebe5ba0bc5a91de05e6858fc58f69e0
2013-10-14 15:04:40 -06:00
Samuel Merritt
7d61c54399 Make pbr only a build-time dependency.
This lets you build python-swiftclient packages that don't require pbr
to be installed at all. You would need pbr on the machine running
rpmbuild / debuild, but not on the machines that install the packages.

Unfortunately, this does not make python-swiftclient able to be
installed via pip 0.3.1 on Lucid; you'll need to uninstall the system
python-pip package and install a new pip some other way. Given that
pip < 1.3 doesn't perform SSL certificate validation for pypi (trivial
MITM attack, anyone?), you'd probably want to get a new pip anyway.

Change-Id: I85d4d77aacf094e48d39e48e750594b95dbc7af0
2013-10-10 11:57:47 -07:00
Clay Gerrard
d687060a44 Add verbose output to all stat commands
When you stat a container or object with the verbose flag the full path of the
reousrce will be displayed with the token similarlly to how an account stat
displays the auth url and token.

 * move some logic out of bin/swift.st_stat to test it
 * new module swiftclient.commnad_helpers for code you want to test
 * moved prt_bytes into swiftclient.utils to test it
 * fixed IndexError with prt_bytes on sizes >= 1024Y

Change-Id: Iaaa96e0308b08c554205b0055b8a04de581fefa4
2013-10-09 14:31:47 -07:00
Matthieu Huin
afa1f9f76d user defined headers added to swift post queries
This commit fixes a problem where headers passed through the option
--header were not taken into account when doing "post" queries.

Change-Id: Ia34808c53a1816d9956a98e0803d8c4d20d2ad44
Closes-Bug: #1226256
2013-09-27 11:40:34 +02:00
David Goetz
266217e32c fix bug with replace old *LOs
Change-Id: I9e6278b62608cf36fa7e5e228f234cc8199c17fc
2013-09-10 13:12:29 -07:00
Alexis Lee
47c33cc4a3 Extend usage message for swift download
Make it clear that you do not have to provide either the container or
object positional parameters, and what swiftclient will do if you omit
them.

Change-Id: I3766fc1eabd51f197e2175771af579577a96c192
2013-09-06 14:57:34 +01:00
Stanislaw Pitucha
a47d3d6468 Remove redundant unit suffix
Transfer speed already has a "MB/s" unit. The "s" suffix was probably a
copy/paste mistake.

Change-Id: Iccd7894d5ca11aac8a9f98365834c5d3deef0720
2013-08-28 16:26:28 +01:00
Jakub Krajcovic
b0eb857b69 Reformat help outputs
Reformatted the help outputs in the swift client to bring them more
in line with the rest of the openstack clients. Changes were added to
all of the st_command_help variables and a new variable
st_command_options was added to help with correct formatting of
help outputs.

Change-Id: I0c549cf96292384ddb36b43ecec41a883d8b2e13
Implements: changes to help outputs
2013-08-19 20:45:00 +10:00
Darrell Bishop
9198e95468 Move multi-threading code to a library.
This patch extracts the multi-threading code from bin/swift into
swiftclient/multithreading and adds tests.  In particular, this new way
of doing it (with context managers) will prevent non-daemonic threads
from wedging the process when unexpected exceptions happen.

I enabled reporting of which lines, specifically, are not covered by
unit tests (added -m option to "coverage report" in .unittests).

This patch includes a drive-by fix for uploading a segmented file with
--use-slo when that object already exists.  A key of "name" was used
instead of "path", raising KeyError.

There's also another drive-by fix for uploading segmented objects with
--use-slo.  Commit 874e0e4427b80e1b15b74a1557b73ba9d61443ca regressed
this by removing the capturing of thread-worker results in
QueueFunctionThread.run().  This patch restores that functionality and
the feature (uploading SLO objects).

Change-Id: I0b4f677e4a734e83d1a25088d9a74f7d46384e53
2013-07-28 22:08:17 -07:00
Jenkins
65e5ef7b81 Merge "Allow setting # of retries in the binary" 2013-07-12 15:17:03 +00:00
Jenkins
c460ebff0a Merge "Allow storage url override for both auth vers." 2013-07-10 07:06:08 +00:00
Ben McCann
188e835ead Allow setting # of retries in the binary
Change-Id: I7ef4f1f86cc6ea4937043ea7bf7d9526e23bd2c7
2013-07-08 17:09:44 -07:00
Kun Huang
94d7c23181 Note '-V 2' is necessary for auth 2.0
Here are some questions in Launchpad, just because they miss '-V 2' for
auth 2.0.
https://answers.launchpad.net/swift/+question/231938
https://answers.launchpad.net/swift/+question/231279
So I add somethins for noteing people add the '-V 2'

Change-Id: I2e0fef969be4eca316975de03fdbc516a1069acf
2013-07-07 16:22:01 +08:00
Darrell Bishop
1d4d51b218 Allow storage url override for both auth vers.
When --os-storage-url is specified on the command-line to bin/swift, it
will override the used storage URL regardless of authentication for both
authentication version 1 and version 2.  This can be used to bypass a
load-balancer to hit a specific proxy server for testing/debugging
purposes.

Within the client library, this feature is accessed by passing the
desired storage URL into swiftclient.client.Conection.__init__() via the
os_options keyword argument.  For example:

  conn = Connection(auth_url, user, key, os_options={
    'object_storage_url': 'http://overridden.storage.url/AUTH_foo'})

This patch also adds a dependency on mock>=0.8.0, which is the same as
openstack/swift.

Change-Id: Id2a36ed6abffd65e7762b6beea5bbfc6c036e848
2013-07-05 14:12:59 -05:00
Jenkins
6f7458a290 Merge "Add -p option to download command." 2013-07-05 17:16:34 +00:00
Darrell Bishop
f022aac0cf Add -p option to download command.
Allow the ability to download a subset of containers (--all with -p) or
a subset of objects within a container (container name with -p).

This patch also includes a drive-by fix for "download --all" which would
not actually download any objects (for me, at least) because the object
queue got filled with "stop" messages before the container workers had
run long enough to put work in the object queue.  Doh!

I also closed up a few holes where an (unexpected, obviously) Exception
could cause the process to hang because non-daemon threads still
existed.

Change-Id: I71c6935c60282b5353badc2dfce8a935d47e3bb7
2013-06-26 23:00:46 -07:00
Mark Seger
9f5b334f7a add -t for totals to list command and --lh to stat
Change-Id: Ieaba5e7762032bf9a2652ead39870d55e85b4672
2013-06-26 15:48:21 +00:00
Jenkins
473167ff31 Merge "Add -l and --lh switches to swift 'list' command" 2013-06-11 22:03:34 +00:00
Mark Seger
17bb3fa779 Add -l and --lh switches to swift 'list' command
Also updated man page and added an example in 'list'
command help message

Change-Id: Icf38de9bab6d387a9a1deed444a91f68b4389cfe
2013-06-10 16:49:44 +00:00
Stuart McLaren
790f087a67 Add option to disable SSL compression
Allows optionally disabling SSL compression. This can significantly
improve HTTPS upload/download performance in some cases -- in particular
when the object is not compressible and you have very high network
bandwidth.

Implements blueprint ssl-compression.

Change-Id: I1260055f9c2e83cdabfeb51aed11b3899bed4d55
2013-06-04 16:33:18 +00:00
Tong Li
874e0e4427 remove busy-wait so that swift client won't use up all CPU cycles
The current implementation uses busy-wait and checking flags to
communicate between threads. It wastes a lot of CPU powers. With
python 2.6 is required for Swift, the communication between
threads should now be using queue and signal mechanisms. This
patch removed the busy-wait loops and use queue and queue join
for threads coordination which will not consume CPU cycles if
a thread is blocked.

Change-Id: I648cd637a92a159d5c13baa83f357cee2dfe7937
2013-05-29 19:09:49 +00:00
David Goetz
adebab739d do not warn about etag for slo
Change-Id: I17fa1643f28aa1cd160c2d770243875a3c4a80ee
2013-05-13 07:37:15 -07:00
Monty Taylor
1eb2c29856 Switch to flake8.
Change-Id: Ib9ba1e7eed09c5a90c558a8365d0a87c3f4b5ee5
2013-05-01 12:23:01 -04:00
Dirk Mueller
000e33d021 Improve Python 3.x compatibility
Some mechancical replacement of the deprecated except x,y:
construct with except x as y, which works with any Python
version >= 2.6

Change-Id: Ic245049dc7b408a5c89b9e27dfd2bd7c08acc5b5
2013-04-29 11:08:47 +02:00
Jenkins
2d97609a52 Merge "Static large object support." 2013-04-04 23:08:25 +00:00
Dean Troyer
fab61c8275 Improve auth option help
Add to the help strings to clarify the need for --os-auth-token to
also have --os-storage-url supplied.

Change-Id: I6a7259c38ce01813ae96f307efaf46e7863bd223
2013-04-04 15:41:22 -05:00
David Goetz
2b3d171907 Static large object support.
Also fixed bug with current large objects with segment listing prefixes.

Allow retry for object PUTs when possible.

Change-Id: I0edff127fd5d5c53da33aa7cb76a4f4dc85bf6e6
2013-04-03 09:51:15 -07:00
Jenkins
42fa4568f1 Merge "Allow user to specify headers at the command line." 2013-03-21 11:11:50 +00:00
Jenkins
f31916051d Merge "Allow v2 to use storage_url/storage_token directly" 2013-03-19 18:36:04 +00:00
tong li
48f446a938 Allow user to specify headers at the command line.
Currently swift client does not allow content type to be specified
when upload an object. It does not allow a user to change content
type neither. This patch will allow a user to specify content
type by using upload command or modify the content type by using
post command.

For example:
1. To upload an object with specific content type:
swift upload -H content-type:applicaiton/xml con1 file1

2. To modify an object to a specific content type:
swift post -H content-type:application/josn con1 file1

This patch also allow a client to specify other request headers.

Change-Id: I12db83a1d465285e0906889cc67dfaa44e059568
Fixes: bug #1154621
2013-03-18 09:48:33 -04:00
Jenkins
999e1c0f02 Merge "Added "/" check when list containers." 2013-02-28 02:58:28 +00:00
Chmouel Boudjnah
45af829aa5 Allow v2 to use storage_url/storage_token directly
- Fixes bug 1134306.

Change-Id: Ie1737c57f57bcdad50b3096c921d46a1fe5293b8
2013-02-27 13:55:05 +01:00
yuan-zhou
9ac2c99fc6 Allow to specify segment container
Fix bug 1110924

Add segment container option as part of 'swift upload -S'. If this
option is omitted, the segments will go to <container>_segments.

Example:
$ swift upload c1 obj1 -S 1 -C user_segments

Please check swift upload --help for more details

Change-Id: Ib71aae322485d8d3ac89916d37ebcac053f49e3b
2013-02-26 11:09:34 +08:00
Jian Zhang
ec38637a35 Added "/" check when list containers.
If there is a "/" in container name, print usage.

Fixs bug 1121897.

Change-Id: I633b5b7ff3a8dbc249ddd3c65439dd46c7d93b4a
2013-02-24 19:10:53 -08:00
Chmouel Boudjnah
9a484dfd63 Fix reporting version.
- Fixes bug 1102691.

Change-Id: I72749a1c87a1f337b45d193e923b437d12270b05
2013-01-22 16:32:50 +01:00
Jenkins
3d04363ba4 Merge "Add env[SWIFTCLIENT_INSECURE]" 2013-01-10 20:10:30 +00:00
You Yamagata
5edb40b442 Add env[SWIFTCLIENT_INSECURE]
Add env[SWIFTCLIENT_INSECURE] as default of --insecure option.
If set to 'true', allow to access insecure keystone server.
The name follows 'NOVACLIENT_INSECURE' in novaclient.

Change-Id: I322674eba9c07e6def97bce339815fa15191a92d
2013-01-10 13:26:53 +09:00
Chmouel Boudjnah
ab4282ebb4 Fix debug feature and add --debug to swift.
- Remove PYTHON_SWIFTCLIENT and use --debug making it more consistents
  with other openstack clients libraries.
- Fix printing the curl command.
- Don't show the body of the GET objects it could get bad on large
  objects and it's an iterator anyway.
- Use -I for showing HEADs (and not -X HEAD).

Change-Id: I954e7d3d795401fc3679725440dec36cdc80af87
2013-01-07 17:59:12 +01:00