Some proxy or gateway softwares, e.g. jumpgate [0], use url-pattern
based approach to match which hanlding logic needs to be triggered for
particular service calling when it received a http(s) call as a
middleman. The change fixed an issue which caused glanceclient send out
the request to a dis-normal url, which contains duplicated "/".
The change removed a wrong and duplicated code snippet from curl logging
function as well.
[0] http://goo.gl/yt52X1
Change-Id: Ic8b3920e11d400771ead7f9c44b615f10b4a5cef
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Setting stream=True with requests can lead to issues
with not closing the connection so the urllib3 connection
pool is not freed up, so only set stream=True if making
a request with application/octet-stream content-type.
See the body-content-workflow and keep-alive sections
in the requests docs here for more information:
http://docs.python-requests.org/en/latest/user/advanced/
Note that commit dbb242b changed the response body_iter
code to potentially return a six.StringIO object rather
than the old ResponseBodyIterator class and since the
images client code is not converting the body_iter into
a dict using json.loads, we have to do that directly
in the _request method where the body_iter is
constructed.
Co-authored-by: Flavio Percoco <fpercoco@redhat.com>
Change-Id: I86572b69c4511f933c9676108190271874346302
Partial-Bug: #1341777
This review implements blueprint python-request and replaces the old
http client implementation in favor of a new one based on
python-requests.
Major changes:
* raw_request and json_request removed since everything is now being
handled by the same method "_request"
* New methods that match HTTP's methods were added:
- get
- put
- post
- head
- patch
- delete
* Content-Type is now being "inferred" based on the data being sent:
- if it is file-like object it chunks the request
- if it is a python type not instance of basestring then it'll try
to serialize it to json
- Every other case will keep the incoming content-type and will send
the data as is.
* Glanceclient's HTTPSConnection implementation will be used if
no-compression flag is set to True.
Co-Author: Flavio Percoco<flaper87@gmail.com>
Change-Id: I09f70eee3e2777f52ce040296015d41649c2586a
To be able to create profiling traces for Glance, client should be
able to send special HTTP header that contains trace info.
This patch is as well important to be able to make cross project
traces. (Typical case nova calls glance via python client, if
profiler is initialized in nova, glance client will add extra
header, that will be parsed by special osprofiler middleware in glance
api)
Don't worry no security issue here, trace information is signed by HMAC
key that is setted in api-paste.ini. So only person that knows HMAC key
is able to send proper header.
Change-Id: Ib13084fbe9b33c2f3dee165f7d6c778546cce6ca
If --os-cacert was passed to the cli the following error was
triggered: "cafile must be None or a byte string". This is
because 'load_verify_locations' requires a byte string to
be passed in.
We fix this by explicitly converting the argument to a byte
string.
We do this in 'VerifiedHTTPSConnection' rather than sooner, eg
during arg handling, as it will no longer be required should we
move to a different http library (eg requests).
Fixes bug 1301849.
Change-Id: I9014f5d040cae9f0b6f03d8f13de8419597560cb
This debug line is causing tracebacks in the n-cpu logs for
tempest runs. Its because the logged data is sometimes unicode:
Traceback (most recent call last):
File "/usr/lib/python2.7/logging/__init__.py", line 846, in emit
msg = self.format(record)
File "/opt/stack/new/nova/nova/openstack/common/log.py", line 710, in format
return logging.StreamHandler.format(self, record)
File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
return fmt.format(record)
File "/opt/stack/new/nova/nova/openstack/common/log.py", line 674, in format
return logging.Formatter.format(self, record)
File "/usr/lib/python2.7/logging/__init__.py", line 467, in format
s = self._fmt % record.__dict__
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1234: ordinal not in range(128)
Logged from file http.py, line 153
The change used correct encoding error handling policy for the log
which may includes non-ascii char.
Closes-bug: 1320655
Change-Id: I97f5f14b9beddcceb7fbd371062caf5a38a62a20
Class `Managers` from `glanceclient.common.base` module is similar to
class `apiclient:ManagerWithFind` from common code.
In this patch:
- class glanceclient.common.base:Managers replaced by
apiclient:ManagerWithFind
- module glanceclient.common.base marked as 'deprecated'
Related to bp common-client-library-2
Change-Id: I41da4a9188e97ca2c07b6234fc2ac0a877553d3f
Switch to using network_utils for splitting the URL. The code
in oslo-incubator supports ipv6 urls
Change-Id: I76be6173b97eb000319d30b4e9232a5a7c4a5aba
Closes-Bug: #1298137
Some backend stores e.g. RBD, will fail if told to create an
image without a valid size (RBD will fail to write to a zero-size
image). Here we add support to allow the image size to be provided
when doing an upload. The result is that the upload content-length
will be set if available either from checking the supplied file
object or as provided by user.
Closes-Bug: 1220197
Change-Id: Ia1f2ea5680a139750d931591949b3e0058148b4b
the client should not log a server 50x response at ERROR level.
It already throws an exception, so the caller can care about what
it wants to do with this. This error will show up regularly in
nova that succeed, and there is no way to disable this message,
which isn't an error from Nova's perspective without turning off
*all* the logs.
Set this to debug instead, because from a caller perspective that's
what you want out of this.
Change-Id: I6d0efb53d1e81adf309f7fa580ec5a8073a811c5
Those modules have been combined in python 3. Therefore,
implementing the six module helps to have support to both versions.
Change-Id: I164a0f19790ff066d16d0cf4f0daa6f1097c848e
Closes-Bug: #1267181
six.iteritems() replaces dictionary.iteritems() on Python 2 and
dictionary.items() on Python 3.
six.string_types replaces basestring() in Python 2 and str
in Python 3.
Change-Id: Ia18510d167df35caec83626718010228e2140bc0
* Fix "TypeError: not all arguments converted during string formatting"
* Add a UT case to cover the path where the bug is in
Change-Id: I91a137c5c3a9a3cc603804bef5eaea14ae281c08
Closes-Bug: #1265730
This adds six to the requirements.txt file in order to make
some HTTP-related imports work across Python 2's httplib and Python 3's
http.client modules. Tests were updated, including one change to the
location of HTTPConnection - moving it from being accessed where it was
imported rather than its canonical location inside of
six.moves.http_client.
Change-Id: Ibc4932b37dfdf195cd5091066914513af1876955
Signed-off-by: Chuck Short <chuck.short@canonical.com>
range now behaves like xrange did in Python 2.
http://docs.python.org/3.1/whatsnew/3.0.html
Change-Id: I71bef224fb6b0e972373a024f8b86355b5970827
Signed-off-by: Chuck Short <chuck.short@canonical.com>
After removing posixpath.normpath(url) in http.py, the code has a
regression bug that the url like 'http://example.com:80/test' can
not work. The code urlparse.urljoin() can not work as '%s%s' %
(self.endpoint_path, url).
Fixes bug #1230032
Change-Id: Ie7266fc3a067b92dfeed169086b4bf6a87dedbd6
The code for processing an HTTP redirect included an incorrect method of
getting the Location header from an HTTPResponse. It needs to use the
getheader() method on the response, instead. This patch fixes that and
includes a unit test that covers this code path.
Change-Id: I0952fabad581b020dee07bdc4007b55b47c906aa
Closes-Bug: #1231524
This code specified None as the default value, but it would raise an
exception if that was ever returned. An empty string causes the code to
work as intended.
This came up while I was writing a unit test for another bug fix.
Change-Id: I658bb8a9b5124f281988fb60b645182ea0ccf99f
Related-bug: #1231524
Add a new module that contain generic wrapper for file and iterator, which
are used to wrap image to upload and the request body iterator in upload and
download cases repectively, to show and advance a pretty progress bar when this
laters are consumed, The progress bar is triggered by adding a --progress command
line argument to commands: image-create, image-download or image-update.
Change-Id: I2ba42fd0c58f4fa087adb568ec3f08246cae3759
bug fix: LP#1112309
blueprint: progressbar-when-uploading
Fixes bug 1179392
glanceclient.common.http.HTTPClient.get_connection_kwargs method
explicitly sets the timeout to be a float, while struc.pack is
still using LL as the format string which means 2 Long Integers.
Setting format string to fL which mean a float followed by Long
Integer.
Also, Bad file descriptor error is caused by socket being closed
to soon. Added a close() implementation to VerifiedHTTPSConnection
which will remove reference to socket before returning call to
base HTTPConnection.close(). This will avoid socket to be closed
before response body is read. Socket will close when response
close is called.
Change-Id: I3a973da3b962c7572ae0f61f6996bdd1f0048339
There is an upcoming patch in nova which passes identity
headers to glance client. We want to ensure that these get
passed to glance, which in turn with help the no auth
option in glance.
Resolves bug 1200761
Change-Id: Ifbef582aa4e64a2e7a46db43a9cc6cf8c3531dbd
Currently, accessing a host via ip address will pass SSL verification;
the CNAME is not checked as intended as part of verify_callback.
'preverify_ok is True' will always return false (int/bool comparison).
preverify_ok will be 1 if preverification has passed.
Fixes bug 1192229
Change-Id: Ib651548ab4289295a9b92ee039b2aff2d08aba5f
Instead of globally ignoring pyflakes and
hacking warnings, only blacklist those that trigger
very frequently so far, in order to clean them
up in followup commits. Fix and start gating
on the rest already.
Change-Id: Ied7c7250061e3bf379e8286e8ce3b9e4af817faf
Glanceclient implemented both functions before they landed into oslo.
Since both functions are already in oslo, it is now possible to pull
them in.
There's a small difference between glance's implementation and oslo's,
that is the later does not convert non-str objects - int, bool - to str
before trying to decode / encode them. This patch takes care of that
where necessary, more precisely, while encoding headers before doing a
new request.
Fixes bug: #1172253
Change-Id: I9a0dca31140bae28d8ec6aede515c5bb852b701b
image-update --debug message no longer adds '-d "None"' to the curl command that
is output. This keeps the curl output consistent with the client
request.
Fixes bug #1172702
Change-Id: I34ceeb6f4a67c753ca3a805ec11240a99ce38ec4
