776 Commits

Author SHA1 Message Date
Jenkins
a6f171437d Merge "Add test for --debug taking precedence over --info" 2016-02-11 20:22:24 +00:00
Jenkins
2b391d47f0 Merge "Fix debug and info option parsing" 2016-02-11 20:22:09 +00:00
Alistair Coles
28558c7e0a Add test for --debug taking precedence over --info
Change-Id: Ibf0903817852edb36389028383a35e0d65f88a26
2016-02-11 15:02:09 +00:00
Jenkins
d4090d555d Merge "Support --os-identity-api-version option" 2016-02-11 05:28:17 +00:00
Tim Burke
30ca247426 Display proper name when failing to create segments container
Previously, we displayed the base container's name.

Change-Id: I70f1949a44ba61158e31178e4536f229c37aab47
2016-02-10 13:25:38 -08:00
Jenkins
8918c292d7 Merge "more tests for pseudo/dir" 2016-02-10 19:56:27 +00:00
Jenkins
50f3396620 Merge "Fix segmented upload to pseudo-dir via <container>" 2016-02-10 19:55:30 +00:00
Jenkins
84d110c63e Merge "Accept token and tenant_id for authenticating against KS" 2016-02-10 19:39:53 +00:00
Alexandra
f82d26063a New python swiftclient doc
Updating with new ToC to be worked on at hackathon

Change-Id: I55ee83626dd88fcc3e6352b3854b758dd7090590
2016-02-10 18:33:34 +10:00
Clay Gerrard
9a97b51c0c more tests for pseudo/dir
Change-Id: Idab172aefd8e69ca8e4d623918eba1bc1da91a42
2016-02-09 13:15:22 -08:00
Chaozhe.Chen
9e6b12e6b2 Use "# noqa" instead of "#flake8: noqa"
"# flake8: noqa" option disables all checks for the whole file.
To disable one line we should use "# noqa".

Change-Id: I7859eab30563d0eb91c5f055d1b523173b562e54
Closes-bug: #1540254
2016-02-09 23:09:24 +08:00
Alistair Coles
d6ef83021e Support --os-identity-api-version option
Add support for the auth-version to be specified using
--os-identity-api-version or OS_IDENTITY_API_VERSION
for compatibility with other openstack client command
line options.

The auth version used will be selected as follows:

- if either --auth-version or --os-identity-api-version
  is set, use that value
- otherwise use the value of ST_AUTH_VERSION, if set
- otherwise use the value of OS_AUTH_VERSION, if set
- otherwise (new behaviour) use the value of
  OS_IDENTITY_API_VERSION, if set
- otherwise default to 1.0

Note that before this change the auth version might have
defaulted to 1.0 despite OS_IDENTITY_API_VERSION being set,
but with this change OS_IDENTITY_API_VERSION is preferred.

Change-Id: Ifba4c4e43560ede3013337b8cdbc77dc2de6e8ff
Closes-Bug: #1541273
2016-02-05 11:13:08 +00:00
Jenkins
d2ee01bfe4 Merge "Fix intermittent fail of test_delete_bulk_account" 2016-02-04 02:48:11 +00:00
James Nzomo
5ed02345d3 Fix segmented upload to pseudo-dir via <container>
This fix ensures creation and use of the correct default segment
container when pseudo-folder paths are passed via <container> arg.

Change-Id: I90356b041dc9dfbd55eb341271975621759476b9
Closes-Bug: 1532981
Related-Bug: 1478210
2016-02-04 05:02:17 +03:00
Jenkins
da27ad11cc Merge "Fix intermittent fail of test_delete_account" 2016-02-01 19:19:46 +00:00
Jenkins
fee68f7324 Merge "Don't trust X-Object-Meta-Mtime" 2016-02-01 17:43:03 +00:00
Alistair Coles
14a0447491 Fix intermittent fail of test_delete_account
The test asserts calls made in specific order,
but they are made from threads so may be in
different order.

Change-Id: I1b6e7303fe0e6fb2afc7da3462b891feab90bc17
Closes-Bug: #1539536
2016-02-01 10:09:30 +00:00
Jenkins
e5f058e27e Merge "Tighten up to unittests to expect rounding" 2016-02-01 10:09:07 +00:00
Alistair Coles
fa9251a1ee Fix intermittent fail of test_delete_bulk_account
The test asserts calls made in specific order,
but they are made from threads so may be in
different order.

Change-Id: I857ad3b909c3b635927fb1a39682d66d20c6fd59
2016-02-01 10:07:55 +00:00
Clay Gerrard
34ae109173 Tighten up to unittests to expect rounding
Change-Id: Ic453fd990ebbf17e0eeabc39b126d2bc14234e23
2016-01-29 19:27:11 -08:00
Tim Burke
337570a03a Don't trust X-Object-Meta-Mtime
Still use it if we can, but stop throwing ValueErrors if it's garbage.

Change-Id: I2cf25b535ad62cfacb7561954a92a4a73d91000a
2016-01-29 16:50:15 -08:00
Jenkins
6f431ee065 Merge "Prevent test runs from cluttering current directory" 2016-01-29 12:26:13 +00:00
Ondřej Nový
0fe02eb1c0 mock time in unit test
It's crashing now (sometimes):

_StringException: Traceback (most recent call last):
  File "/build/python-swiftclient-2.6.0/tests/unit/test_service.py", line 1085, in test_upload_object_job_stream
    self.assertAlmostEqual(mtime, expected_mtime, delta=0.5)
  File "/usr/lib/python3/dist-packages/unittest2/case.py", line 883, in assertAlmostEqual
    raise self.failureException(msg)
AssertionError: 1453224313.0 != 1453224312.4944572 within 0.5 delta

Change-Id: Ib2eeb13cd07febcb7c8b4e1435b885c4339093e4
2016-01-29 00:18:28 +01:00
Jenkins
6d52264c30 Merge "Error with uploading large object includes unicode path" 2016-01-28 00:22:35 +00:00
Jenkins
e65e01014e Merge "Convert http response(byte string) to string in python3." 2016-01-22 13:16:44 +00:00
Jenkins
88874ae441 Merge "Use bulk-delete middleware when available" 2016-01-22 05:49:31 +00:00
Tim Burke
38f9664167 Prevent test runs from cluttering current directory
Previously, the following empty directories would be created:
  * container
  * container/pseudo
  * pseudo

Change-Id: I002e2da8d28a873728e0b5c2d33f94f21132d058
2016-01-19 14:26:13 -08:00
Tim Burke
dcdd715215 Get rid of FakeConn cruft
Presumably, this was left over from before the httplib -> requests
transition?

Change-Id: I7f505514070bf9d8fefda77203bee78f0a5dd71d
2016-01-18 21:14:53 -08:00
Pratik Mallya
a175689418 Accept token and tenant_id for authenticating against KS
Allow swiftclient to authenticate against keystone using tenant
name/id and token only. Without this patch, the password is
required, which may not always be available. Authentication
against keystone is required to get the service catalog,
which includes the endpoints for swift.

Change-Id: I4477af445474c5fa97ff864c4942f1330b59e5d6
Closes-Bug: #1476002
2016-01-18 10:47:05 -08:00
Jude Job
47f673ed9f Error with uploading large object includes unicode path
This patch include a test case to test unicode path.

Change-Id: I7697679f0034ce65b068791d7d5145286d992bd1
Closes-Bug: #1532096
2016-01-18 12:56:27 +05:30
Jenkins
64c2c2eaaa Merge "Fix the http request headers being overwritten in logging" 2016-01-15 10:52:45 +00:00
Min Min Ren
61880c6f98 Fix the http request headers being overwritten in logging
Fix the http request headers in put_object being overwritten in logging

Change-Id: Id0d1e36561a61ed1ce30d93c801ec32f058a6fa4
Closes-bug: #1501292
2016-01-14 15:45:22 +00:00
Tim Burke
7a1e192803 Use bulk-delete middleware when available
When issuing `delete` commands that would require three or more
individual deletes, check whether the cluster supports bulk deletes
and use that if it's available.

Additionally, a new option is added to the `delete` command:

  * --prefix <prefix>

    Delete all objects that start with <prefix>. This is similar to the
    --prefix option for the `list` command.

Example:

$ swift delete c --prefix obj_prefix/

    ...will delete from container "c" all objects whose name begins with
    "obj_prefix/", such as "obj_prefix/foo" and "obj_prefix/bar".

Change-Id: I6b9504848d6ef562cf4f570bbcd17db4e3da8264
2016-01-12 15:40:57 -08:00
Jenkins
6ed6c3343f Merge "Retry file uploads via SwiftService" 2016-01-12 13:00:17 +00:00
Jenkins
c9ce80c460 Merge "Fix upload to pseudo-dir passed by <container> arg" 2016-01-11 23:58:41 +00:00
Tim Burke
5050027610 _RetryBody doesn't need to take explicit etag/content-length
Also, don't try to do int(None) for chunk-encoded responses (like DLOs
that are longer than a single container listing).

Change-Id: Ibacd75d5ee46135d62388786903c895fda8ed3ba
2016-01-11 15:36:37 -08:00
Tim Burke
d4157ce5b5 Retry file uploads via SwiftService
When we introduced LengthWrapper, we neglected to make it resettable. As
a result, upload failures result in errors like:

    put_object(...) failure and no ability to reset contents for reupload.

Now, LengthWrappers will be resettable if their _readable has seek/tell.

Related-Change: I6c8bc1366dfb591a26d934a30cd21c9e6b9a04ce
Change-Id: I21f43f06e8c78b24d1fc081efedf2687942e042f
2016-01-11 11:12:13 -08:00
Stuart McLaren
4af623bcf1 Retry download of object body
Currently the swift client retries establishing a connection to the server
(by default up to 5 times).  However, when downloading an object, once
the connection has been established and the inital headers have been
returned, no attempt is made to retry.

So, for example, if 99MB of a 100MB object have been downloaded and the
connection is then lost, the download will fail.  This patch changes the
behaviour to re-establish the connection and fetch the remaining bytes
using the 'Range' header to offset.

Data retry is not yet supported if the original request is for a subset
of the object data (ie uses the 'Range' header), or if resp_chunk_size
has not been set.

The object's etag is checked using If-Match to make sure the object data
hasn't changed since the start of the download.

Change-Id: Iab47f10081ff39f6d344dbc2479cbc3bfd1c5b29
2016-01-08 17:12:02 +00:00
Jenkins
d1e3109588 Merge "Replace assertEqual(arg, None) with assertIsNone(arg)" 2016-01-06 09:19:43 +00:00
Jenkins
633a491538 Merge "Add functional test for object PUT with raw iterator" 2016-01-06 04:01:02 +00:00
Jenkins
39676743bb Merge "Fixed few misspellings in comments" 2016-01-06 03:03:56 +00:00
Jenkins
81003b8d99 Merge "Wrap raw iterators to ensure we send entire contents to server" 2016-01-06 00:06:04 +00:00
Ondřej Nový
21a841a003 Fixed few misspellings in comments
Change-Id: I29d891e2dc900eb93e703f77f4e56f68710a8955
2016-01-05 20:28:47 +01:00
Alistair Coles
ab34609050 Add functional test for object PUT with raw iterator
Adds a functional test to verify change made in [1]

[1] change id I19579ed7a0181ac3f488433e7c1839f7f7a040b8

Change-Id: I45dbf66edab645e6339e67906aee5faa4fb7efbd
2016-01-05 18:25:19 +00:00
James Nzomo
2c6f367035 Fix upload to pseudo-dir passed by <container> arg
This fix makes it possible to upload objects to pseudo-folders by
passing the upload paths via <container> arg regardless of whether the
container or folder path exist or not.

Change-Id: I575e58aa12adcf71cdaa70d025a0ea5c63f46903
Closes-Bug: #1478210
Partial-Bug: #1432734
Related-Bug: #1432734
2016-01-04 22:48:07 +03:00
Christian Schwede
109e8f519f Fix debug and info option parsing
The debug and info options need to be set before a subcommand method is called,
otherwise they are simply ignored. This is kind of irritating - other options
(for example -U, -A, -K) are usable after a positional command.

This patch fixes this, and commands like these are no longer ignoring --debug or --info:

 swift stat --debug
 swift list container --info

Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>

Change-Id: Ib19b05deef7a015881f1eed4a3946025e16bf922
2015-12-30 14:22:01 -08:00
Tim Burke
39b1a31d8a Wrap raw iterators to ensure we send entire contents to server
Currently, if you attempt to stream an upload from an iterator, as in

    def data():
        yield 'foo'
        yield ''
        yield 'bar'
    conn.put_object('c', 'o', data())

... requests will faithfully emit a zero-length chunk, ending the
transfer. Swift will then close the connection, possibly (if Connection:
keep-alive was set) after attempting to parse the next chunk as a new
request.

Now, Swift will receive all of the bytes from the iterable, and any
zero-byte chunks will be ignored.

This will be fixed in requests [1], but not until an eventual 3.0.0
release.

[1] https://github.com/kennethreitz/requests/pull/2631

Change-Id: I19579ed7a0181ac3f488433e7c1839f7f7a040b8
2015-12-30 11:56:36 -08:00
Tim Burke
62bfe10f58 Fix some typos
Change-Id: Iaf7f30a7ae0c2ac76fc5cdcee31ea74c08ce601e
2015-12-30 11:01:33 -08:00
SaiKiran
6da38adb8d Replace assertEqual(arg, None) with assertIsNone(arg)
In python-swiftclient some test cases using asserEqual(arg, None)
instead of assertIsNone(arg).assertIsNone method provides clear error message.

Change-Id: I4d673ede0965408344325c9c234c5c4b1ae4146a
Closes-Bug: #1527556
2015-12-30 17:25:30 +05:30
hgangwx
ab65eef4ce Wrong usage of "an"
Wrong usage of "an" in the messages:
"the optional os_options paramater includes an non-empty"
"We are allowing to have an tenant_name argument"

Should be:
"the optional os_options paramater includes a non-empty"
"We are allowing to have a tenant_name argument"

Totally 2 occurrences in python-swiftclient base code.

Change-Id: I2f2f7e07432fedfee5ccb418d9505250b3fed597
2015-12-30 14:27:29 +08:00