740 Commits

Author SHA1 Message Date
Tim Burke
c3f0641704 Follow-up to patch 282363
* Improve some formatting
* Be more explicit about how much will be revealed when
* Rename redact_sensitive_tokens to redact_sensitive_headers, as it
  affects more than tokens.

Change-Id: I02b375d914e9f0a210d038ecb31188d09a8ffce3
2016-02-25 10:06:48 -08:00
Jenkins
0f60cf05f4 Merge "Do not reveal auth token in swiftclient log messages by default" 2016-02-25 01:49:06 +00:00
Paul Belanger
adc3177d1f Have tox manage LANG environmental var
We need to do this because some of the py34 testing that
python-swiftclient uses depends on this variable.  The reason we don't
see the issue in the gate, is because current bare-trusty images have
this set on the jenkins shell user.

When we move to just using DIBs, the variable won't be setup by
default and python3 tests will fail.

For more information:
  https://review.openstack.org/282898

Change-Id: Id9017f31b0543bccac9c07b83237b909e2bd2b0c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-02-24 14:45:16 -05:00
Min Min Ren
00de6cfa63 Check threads number options validation
Add checking object-threads, container-threads and segment-threads options validation
The values should be a positive integer

Change-Id: Iaf98e33dae4b9a7f82e33f7cc2e5a0b293a1c76f
Close-Bug: #1546973
2016-02-24 20:19:43 +08:00
Jenkins
4244b99b02 Merge "Update api docs title to make ToC better" 2016-02-24 03:26:39 +00:00
shu-mutou
f95de16875 Drop py33 support
"Python 3.3 support is being dropped since OpenStack Liberty."
written in following URL.
https://wiki.openstack.org/wiki/Python3

And already the infra team and the oslo team are dropping py33
support from their projects.

Since we rely on oslo for a lot of our work, and depend on infra
for our CI, we should drop py33 support too.

Change-Id: Ia8f2b26e446175b0d892a11952ef3dc11dcdc73c
Closes-Bug: #1526170
2016-02-23 00:19:56 +00:00
Jenkins
689ce4738d Merge "New python swiftclient doc" 2016-02-22 20:31:15 +00:00
John Dickinson
011d730c9b Update api docs title to make ToC better
Change-Id: Ie8eeb3dd8eddf1868f0fa99911c459ae9f7b0091
2016-02-22 10:32:28 -08:00
Joel Wright
4d44dcf360 Do not reveal auth token in swiftclient log messages by default
Currently the swiftclient logs sensitive info in headers when logging
HTTP requests. This patch hides sensitive info in headers such as
'X-Auth-Token' in a similar way to swift itself (we add a
'reveal_sensitive_prefix' configuration to the client).

With this patch, tokens are truncated by removing the specified number
of characters, after which '...' is appended to the logged token to
indicate that it has been redacted.

Co-Authored-By: Li Cheng <shcli@cn.ibm.com>
Co-Authored-By: Zack M. Davis <zdavis@swiftstack.com>
Change-Id: I43dd7254f7281d4db59b286aa2145643c64e1705
Closes-bug: #1516692
2016-02-22 17:55:51 +00:00
Alistair Coles
67f5468ee4 Fix wrong args for get_container with full listing
In client get_container(), when full_listing is true,
the calls back to get_container() pass service_token
as a positional arg which maps its value to the
full_listing arg. It should use a keyword.

Change-Id: Iac2af45df124ff33fcb7fbaf1ba959ef06c96378
Closes-Bug: #1496093
2016-02-22 15:22:44 +00:00
Jenkins
da0aa24f28 Merge "_RetryBody doesn't need to take explicit etag/content-length" 2016-02-18 23:40:51 +00:00
Jenkins
45583d6af7 Merge "Use "# noqa" instead of "#flake8: noqa"" 2016-02-18 00:14:49 +00:00
Jenkins
f95c03cc40 Merge "Display proper name when failing to create segments container" 2016-02-17 17:16:42 +00:00
Tim Burke
bed6bbd5ef Drop testtools from test-requirements.txt
My understanding is that it was mainly being used so we could have sane
testing on py26.  With py26 support being dropped, we no longer need it.

Also drop discover from test-requirements.txt, as we don't seem to
actually use it.

Change-Id: Iee04c42890596d3b483c1473169480a3ae19aac8
Related-Change: I37116731db11449d0c374a6a83a3a43789a19d5f
2016-02-12 09:57:58 -08:00
Jenkins
a6f171437d Merge "Add test for --debug taking precedence over --info" 2016-02-11 20:22:24 +00:00
Jenkins
2b391d47f0 Merge "Fix debug and info option parsing" 2016-02-11 20:22:09 +00:00
Alistair Coles
28558c7e0a Add test for --debug taking precedence over --info
Change-Id: Ibf0903817852edb36389028383a35e0d65f88a26
2016-02-11 15:02:09 +00:00
Jenkins
d4090d555d Merge "Support --os-identity-api-version option" 2016-02-11 05:28:17 +00:00
Tim Burke
30ca247426 Display proper name when failing to create segments container
Previously, we displayed the base container's name.

Change-Id: I70f1949a44ba61158e31178e4536f229c37aab47
2016-02-10 13:25:38 -08:00
Jenkins
8918c292d7 Merge "more tests for pseudo/dir" 2016-02-10 19:56:27 +00:00
Jenkins
50f3396620 Merge "Fix segmented upload to pseudo-dir via <container>" 2016-02-10 19:55:30 +00:00
Jenkins
84d110c63e Merge "Accept token and tenant_id for authenticating against KS" 2016-02-10 19:39:53 +00:00
Alexandra
f82d26063a New python swiftclient doc
Updating with new ToC to be worked on at hackathon

Change-Id: I55ee83626dd88fcc3e6352b3854b758dd7090590
2016-02-10 18:33:34 +10:00
Clay Gerrard
9a97b51c0c more tests for pseudo/dir
Change-Id: Idab172aefd8e69ca8e4d623918eba1bc1da91a42
2016-02-09 13:15:22 -08:00
Chaozhe.Chen
9e6b12e6b2 Use "# noqa" instead of "#flake8: noqa"
"# flake8: noqa" option disables all checks for the whole file.
To disable one line we should use "# noqa".

Change-Id: I7859eab30563d0eb91c5f055d1b523173b562e54
Closes-bug: #1540254
2016-02-09 23:09:24 +08:00
Alistair Coles
d6ef83021e Support --os-identity-api-version option
Add support for the auth-version to be specified using
--os-identity-api-version or OS_IDENTITY_API_VERSION
for compatibility with other openstack client command
line options.

The auth version used will be selected as follows:

- if either --auth-version or --os-identity-api-version
  is set, use that value
- otherwise use the value of ST_AUTH_VERSION, if set
- otherwise use the value of OS_AUTH_VERSION, if set
- otherwise (new behaviour) use the value of
  OS_IDENTITY_API_VERSION, if set
- otherwise default to 1.0

Note that before this change the auth version might have
defaulted to 1.0 despite OS_IDENTITY_API_VERSION being set,
but with this change OS_IDENTITY_API_VERSION is preferred.

Change-Id: Ifba4c4e43560ede3013337b8cdbc77dc2de6e8ff
Closes-Bug: #1541273
2016-02-05 11:13:08 +00:00
Jenkins
d2ee01bfe4 Merge "Fix intermittent fail of test_delete_bulk_account" 2016-02-04 02:48:11 +00:00
James Nzomo
5ed02345d3 Fix segmented upload to pseudo-dir via <container>
This fix ensures creation and use of the correct default segment
container when pseudo-folder paths are passed via <container> arg.

Change-Id: I90356b041dc9dfbd55eb341271975621759476b9
Closes-Bug: 1532981
Related-Bug: 1478210
2016-02-04 05:02:17 +03:00
Jenkins
da27ad11cc Merge "Fix intermittent fail of test_delete_account" 2016-02-01 19:19:46 +00:00
Jenkins
fee68f7324 Merge "Don't trust X-Object-Meta-Mtime" 2016-02-01 17:43:03 +00:00
Alistair Coles
14a0447491 Fix intermittent fail of test_delete_account
The test asserts calls made in specific order,
but they are made from threads so may be in
different order.

Change-Id: I1b6e7303fe0e6fb2afc7da3462b891feab90bc17
Closes-Bug: #1539536
2016-02-01 10:09:30 +00:00
Jenkins
e5f058e27e Merge "Tighten up to unittests to expect rounding" 2016-02-01 10:09:07 +00:00
Alistair Coles
fa9251a1ee Fix intermittent fail of test_delete_bulk_account
The test asserts calls made in specific order,
but they are made from threads so may be in
different order.

Change-Id: I857ad3b909c3b635927fb1a39682d66d20c6fd59
2016-02-01 10:07:55 +00:00
Clay Gerrard
34ae109173 Tighten up to unittests to expect rounding
Change-Id: Ic453fd990ebbf17e0eeabc39b126d2bc14234e23
2016-01-29 19:27:11 -08:00
Tim Burke
337570a03a Don't trust X-Object-Meta-Mtime
Still use it if we can, but stop throwing ValueErrors if it's garbage.

Change-Id: I2cf25b535ad62cfacb7561954a92a4a73d91000a
2016-01-29 16:50:15 -08:00
Jenkins
6f431ee065 Merge "Prevent test runs from cluttering current directory" 2016-01-29 12:26:13 +00:00
Ondřej Nový
0fe02eb1c0 mock time in unit test
It's crashing now (sometimes):

_StringException: Traceback (most recent call last):
  File "/build/python-swiftclient-2.6.0/tests/unit/test_service.py", line 1085, in test_upload_object_job_stream
    self.assertAlmostEqual(mtime, expected_mtime, delta=0.5)
  File "/usr/lib/python3/dist-packages/unittest2/case.py", line 883, in assertAlmostEqual
    raise self.failureException(msg)
AssertionError: 1453224313.0 != 1453224312.4944572 within 0.5 delta

Change-Id: Ib2eeb13cd07febcb7c8b4e1435b885c4339093e4
2016-01-29 00:18:28 +01:00
Jenkins
6d52264c30 Merge "Error with uploading large object includes unicode path" 2016-01-28 00:22:35 +00:00
Jenkins
e65e01014e Merge "Convert http response(byte string) to string in python3." 2016-01-22 13:16:44 +00:00
Jenkins
88874ae441 Merge "Use bulk-delete middleware when available" 2016-01-22 05:49:31 +00:00
Tim Burke
38f9664167 Prevent test runs from cluttering current directory
Previously, the following empty directories would be created:
  * container
  * container/pseudo
  * pseudo

Change-Id: I002e2da8d28a873728e0b5c2d33f94f21132d058
2016-01-19 14:26:13 -08:00
Tim Burke
dcdd715215 Get rid of FakeConn cruft
Presumably, this was left over from before the httplib -> requests
transition?

Change-Id: I7f505514070bf9d8fefda77203bee78f0a5dd71d
2016-01-18 21:14:53 -08:00
Pratik Mallya
a175689418 Accept token and tenant_id for authenticating against KS
Allow swiftclient to authenticate against keystone using tenant
name/id and token only. Without this patch, the password is
required, which may not always be available. Authentication
against keystone is required to get the service catalog,
which includes the endpoints for swift.

Change-Id: I4477af445474c5fa97ff864c4942f1330b59e5d6
Closes-Bug: #1476002
2016-01-18 10:47:05 -08:00
Jude Job
47f673ed9f Error with uploading large object includes unicode path
This patch include a test case to test unicode path.

Change-Id: I7697679f0034ce65b068791d7d5145286d992bd1
Closes-Bug: #1532096
2016-01-18 12:56:27 +05:30
Jenkins
64c2c2eaaa Merge "Fix the http request headers being overwritten in logging" 2016-01-15 10:52:45 +00:00
Min Min Ren
61880c6f98 Fix the http request headers being overwritten in logging
Fix the http request headers in put_object being overwritten in logging

Change-Id: Id0d1e36561a61ed1ce30d93c801ec32f058a6fa4
Closes-bug: #1501292
2016-01-14 15:45:22 +00:00
Tim Burke
7a1e192803 Use bulk-delete middleware when available
When issuing `delete` commands that would require three or more
individual deletes, check whether the cluster supports bulk deletes
and use that if it's available.

Additionally, a new option is added to the `delete` command:

  * --prefix <prefix>

    Delete all objects that start with <prefix>. This is similar to the
    --prefix option for the `list` command.

Example:

$ swift delete c --prefix obj_prefix/

    ...will delete from container "c" all objects whose name begins with
    "obj_prefix/", such as "obj_prefix/foo" and "obj_prefix/bar".

Change-Id: I6b9504848d6ef562cf4f570bbcd17db4e3da8264
2016-01-12 15:40:57 -08:00
Jenkins
6ed6c3343f Merge "Retry file uploads via SwiftService" 2016-01-12 13:00:17 +00:00
Jenkins
c9ce80c460 Merge "Fix upload to pseudo-dir passed by <container> arg" 2016-01-11 23:58:41 +00:00
Tim Burke
5050027610 _RetryBody doesn't need to take explicit etag/content-length
Also, don't try to do int(None) for chunk-encoded responses (like DLOs
that are longer than a single container listing).

Change-Id: Ibacd75d5ee46135d62388786903c895fda8ed3ba
2016-01-11 15:36:37 -08:00