keystoneclient isn't adding a NullHandler to its logger,
so we see 'No handlers could be found for logger...'
messages when using the swift cli with keystone auth.
Since keystoneclient follows the usual pattern of
naming loggers from module name, we can add a
NullHandler to the 'keystoneclient' logger to handle
any logging from the library.
Note that this makes no change to keystoneclient
logging being emitted when using 'swift --debug'
or 'swift --info'.
No test because I am not sure how to usefully
test this without ending up just testing a mock
of keystoneclient.
Change-Id: Ic81cf8e5a590ae9224d35e3a5b1c5f2a8c33835a
Changed the message shown when a user doesn't enter project/tenant
id/name to be more informative.
When attempting to stat a container without supplying project/tenant
name or id, an empty response was being returned instead of an error
being raised.
Changed the error raised in swiftclient.client when no tenant or project
is specified to be more specific.
Add tests for basic regression checking.
Closes-Bug: #1372589
Change-Id: I4eb6964d9f1702db119cc0294edc02841b1ecd5f
Some client.Connection methods will raise a KeyError
if a response_dict argument is passed and an error
occurs during authentication or making the request.
The fix is straightforward: add a test for existence
of a response_dict before attempting to get it from
kwargs.
The bulk of this patch is adding unit tests for the
response_dict feature.
Closes-Bug: 1381304
Change-Id: Ic7e1b3dfae33909533931c52ac97355867a08a07
We forgot to propagate options into http_connection() for the info
aka capabilities command. Fix is to use the proper helper.
Fixes bug: 1381866
Change-Id: I9515203c969a1bba38dd909412355080383905f9
Fixes a regression that is preventing swiftclient from
authenticating using keystone v3 options.
swiftclient/shell.py processes the dict of command
line options to set up a child dict ('os_options')
of keystone auth specific options (line 960). The
processing includes stripping --os- prefixes from
the command line options before adding keys to
os_options.
A recent patch https://review.openstack.org/#/c/85453/
introduced a duplication of this option processing
in service.py (line 79) which replaces the os_options
created in shell.py, but omits keystone v3 related
options. Consequently the keystone v3 options are
not being passed to the client get_auth() method.
This patch adds the keystone v3 related options to
the option processing in service.py.
For pragmatic reasons (i.e. fixing the bug quickly)
the option processing code has not been removed from
parse_args in shell.py. It is likely that the code in
parse_args is now redundant, but all code paths between
parse_args and process_options should be inspected and
test coverage added before removing that code.
Unit tests have been added in test_shell.py to verify
that command line options are correctly passed to the
client get_auth method.
The MockHttpTest class is re-used in test_shell.py,
so it is moved from test_swiftclient.py
to tests/unit/utils.py
Closes-bug: #1372465
Change-Id: I4fed013cdb8936509609d06093337cc147ade0d6
Enables swiftclient to authenticate using
the keystone v3 API, allowing user id's, user
domains and tenant/project domains to be
specified.
Since swiftclient imports keystoneclient, the
main changes in swiftclient/client.py are to
selectively import the correct keystoneclient
library version and pass a number of new
options to it via the get_auth() function. In
addition the get_keystoneclient_2_0 method
has been renamed get_auth_keystone to better
reflect its purpose since it now deals with
both v2 and v3 use cases.
In swiftclient/shell.py the new options are
added to the parser. To make the default help
message shorter, help for all the --os-*
options (including the existing v2 options)
is only displayed when explicitly requested
usng a new --os-help option.
A new set of unit tests is added to
test_shell.py to verify the parser. A comment
in tests/sample.conf explains how to
configure the existing functional tests to
run using keystone v3 API.
Note that to use keystone v3
with swift you will need to set
auth_version = v3.0 in the auth_token
middleware config section of
proxy-server.conf.
Change-Id: Ifda0b3263eb919a8c6a1b204ba0a1215ed6f642f
get_auth() in client.py raises an exception if
tenant_name is not included in the os_options
dict. This is overly constrained since tenant_id
is equally sufficient. This patch modifies
get_auth to require either tenant_name
or tenant_id.
Change-Id: Ibbcda1704637eb887efa5895579d260a1e072327
There are files containing string format arguments inside
logging messages. Using logging function parameters should
be preferred.
Change-Id: I90270908d27f0a57c184d5a547bb1af9b1381c95
* E111 indentation is not a multiple of four
* E128 continuation line under-indented for visual indent
* E265 block comment should start with '# '
* E713 test for membership should be 'not in'
Tested with pep8 version 1.5.6 (2014-04-14).
Change-Id: I7f85c143d463c501a0df20724362ad5c0f2d4dde
This patch fixes three issues that were found running
functional tests with Python 3:
1. python-requests sets 'application/x-www-form-urlencoded' as
content-type if the input is not a string object and no
content-type is given.
2. Encoding of the headers is now only done if required. This
prevents comparisons between unencoded headers and encoded
prefixes and avoids unnecessary forloop-iterations.
One unittest was extended to ensure it works for unencoded
and encoded headers with or without the prefix.
3. Functional tests recently switched to using byte data for
testing, thus the comparison needs to be a byte object as well.
Change-Id: I035f8b4b9c9ccdc79820b907770a48f86d0343b4
Don't encode standard headers keys but just the x-*-meta- ones. That
fixes py3 to not have them casted as bytes and not properly passed to
the requests kwargs.
Other trivial py3 fixes along the way.
Change-Id: I91b95f32fb2aec9b20892a5bb95fd1bc65d002f3
On Python 3, urllib.parse.quote() accepts bytes and str (unicode) types and
always return str (unicode).
Add also more tests with non-ASCII characters.
Change-Id: I8e0f19da7240e874392327d0da074ed4abb7d213
* Replace unicode with six.text_type
* Replace basestring with six.string_types
* The long type doesn't exist in Python 3 anymore: replace 1L with long(1) and
only test this type with Python 2
* Fix quote(): quote the URL if the string is a byte string. Use "bytes" type
instead of "str" to be Python 3 compatible.
Change-Id: I1df5aa85e4e7d07191fb5c654d52fc4bd8b9f440
dict.iteritems() has been removed in Python 3. In Python 2, dict.items()
creates a temporary list, but Samuel Merritt wrote:
"The size of the data is small enough that we don't need to worry about the
memory consumption; it's all just HTTP headers."
Change-Id: Iac427b8bbd032e4a1efb2bdc84a3968217eb6ddf
By default, the HTTPConnection class expose the raw.read urllib3
function which doesn't do any decoding on the http response.
We can fix this by passing decode_content=True to raw.read().
Change-Id: I7d42d31ae516705d1bde2769e642931308116c7a
urlparse netloc contains the TCP port if presents.
There is no need to provide conn.port when building
CLientException.
Change-Id: Id3e4fa8c4f4ab2ad693b0e8702ab69ed7c291830
The requests 'files' parameter adds this 'Content-Type: multipart/form-data'
HTTP header and the whole multipart body data get stored with the object.
This also create a memory hog issue because files are loaded in memory before
being actually sent. This patch removes this behavior and restores what was
done before, ie: direct uploading.
This patches also fixes an issue in requests, when used with glance
CooperativeReader it mis-calculates content-length leading to chunked encoding
for raw upload.
Change-Id: Ie5b0a1078bedd33f09c6157f48b5f88116c589fa
Closes-Bug: #1280072
Closes-Bug: #1280275
Currently, httplib implementation does not support SSL certificate
verification. This patch fixes this. Note that ssl compression parameter
and 100-continue thing is still missing from requests, though those are
lower priority.
Requests now takes care of:
* proxy configuration (get_environ_proxies),
* chunked encoding (with data generator),
* bulk uploading (with files dictionary),
* SSL certificate verification (with 'insecure' and 'cacert' parameter).
This patch have been tested with requests 1.1.0 (CentOS 6) and requests
2.2.1 (current version).
Change-Id: Ib5de962f4102d57c71ad85fd81a615362ef175dc
Closes-Bug: #1199783
DocImpact
SecurityImpact
This patch adds a capabilities option on swiftclient.
This option uses the new /info endpoint to request the
remote capabilities and nicely display it.
Change-Id: Ie34b454511d5527e402e66e1fdb72120f427f2fd
Added a retry_on_ratelimit parameter to the Connection
class so that ratelimited requests can be retried.
DocImpact
Change-Id: I2817a7ea0ed2d69a7659e80111fbd2c91a75d530
As far as proxies usage is concerned, keystone-client API and swift-client API behave differently because the former uses python Request library while the latter uses raw httplib. As a result, Keystone authentication honors environment variables http_proxy, https_proxy and no_proxy while Swift doesn't.
This patch, which code is mainly borrowed from Python Requests, makes Swift
data connections and Swift authentication connections behaving homogeneously.
Change-Id: Ic8a0089c35c458d7ed96e572e22429014298fe4c
Before commit 7d88d14d (http://git.io/hhJWdQ) swifclient used to accept
header value with the type None :
{'Headers': None}
It would just be happy with it with those None headers and not process
them, reinstate the old behavior.
Closes-Bug: 1256441
Change-Id: Ic7f80c38ef2ce9ef1687ed0d6d65521f2e754905
