As stated in the OpenStack Hacking Guidelines, it is prefered
that only modules should be imported.
Also updated tox.ini to ignore opestack/common among others.
Change-Id: I2f0a603c31052eadee581c11880c0ec6bd392829
To make this work we create a different HTTPClient that extends the
basic keystoneclient Adapter. The Adapter is a standard set of
parameters that all clients should know how to use like region_name and
user_agent. We extend this with the glance specific response
manipulation like loading and sending iterables.
Implements: bp session-objects
Change-Id: Ie8eb4bbf7d1a037099a6d4b272cab70525fbfc85
The oslo.utils libraries are moving away from namespace packages.
This requires oslo.utils>=1.2.0
bp drop-namespace-packages
Change-Id: I803df61e91eabb96329d859aef6bea03530fb84f
Some generated header values may in fact be None. Trying to encode None
causes the client to fail with an exception and cannot be worked around
by the user.
Change-Id: I638b1fba0ef9a07d726445d8c2cdd774140f5b83
Closes-bug: 1415935
This module now lives in oslo.utils, so import it from there instead.
Co-Authored-By: Ian Cordasco <ian.cordasco@rackspace.com>
Change-Id: Ib35dc840992433542490670781badd9529ec8947
If we don't explicitly close a response after streaming its download,
then we can run into HTTPConnectionPool full warnings. It also will hurt
performance if we have to continuously create new sockets for new
responses. Calling close will return the connection to the pool so it
can be reused. Note this is only necessary when streaming a response. If
we don't stream it, then requests will return the connection to the pool
for us.
Change-Id: I803bd4dd0e769c233501d5e5ff07a19705fbe233
Closes-bug: 1341777
The returns from requests' response.content is a bytes type. Under
python 3 this fails in error handling and string conversion. The
response.text variable should be used to treat a response body as a
string.
Closes-Bug: #1407531
Change-Id: Ifd588b5f6820ef21beb186d88d0b3f1a267695aa
When file size is an exact multiple of chunk_size, glance client
is processing EOF in image-data as a chunk and sends to
glance-api. The server treats this as the end of chunked
transmission and sends a http response.
When the actual last chunk is sent by the 'requests' library, the
server sends a 400 response and tracebacks with broken pipe
as the client has already closed the socket.
Closes-Bug: #1342080
Change-Id: Icdbff838450db1c252ddc919a230a7d3ca16765f
Change I09f70eee3e2777f52ce040296015d41649c2586a, introduced a bug where
the identity_headers are not added to the request headers anymore
causing the former to be completely ignored and useless.
This patch fixes that issue by restoring the previous code. A new test
has been added to avoid regressions.
Closes-bug: #1394965
Change-Id: I1b1633636448398cf3f41217f1d671b43ebd9946
python-glanceclient displays curl statements for debugging/troubleshooting
purposes. For IPv6 URLs, curl requires --globoff to be passed in the
arguments. Since glanceclient does not use curl directly, this patch
displays the curl commands with globoff option which works for both
IPv4 and IPv6 URLs.
Fix adapted from python-novaclient Ib7099e8e3bbc15f29bbaa1db37ef21e78a74e7bc
Closes-Bug: #1228744
Change-Id: Ie02c4e75ca1ab995102aa55bbff39b2161218b2d
Commit f980fc549247fa2deb87dfacebc6d8d13ccd45d1 changed how the
X-Auth-Token header was scrubbed when logging the request, but
effectively made the value required which can lead to an AttributeError
if the value for the header is None.
The value can be None if you're using Nova but don't have Nova
configured with auth_strategy='keystone' (see
nova.image.glance._create_glance_client for details).
This patch simply checks if the auth_token is set in the http client
object and if not, it doesn't set the X-Auth-Token key in the session
header.
Closes-Bug: #1381295
Change-Id: Ie285d5253df28a9f0f964147a53c99ceaa919c5c
In order to keep the support for `--ssl-nocompression` it was decided to
overwrite the https HTTPAdapter in `requests` poolmanager. Although this
seemed to work correctly, it was causing some issues when using
glanceclient from other services that rely on requests and that were
also configured to use TLS.
THis patch changes implements a different strategy by using
`glance+https` as the scheme to use when `no-compression` is requested.
Closes-bug: #1350251
Closes-bug: #1347150
Closes-bug: #1362766
Change-Id: Ib25237ba821ee20a561a163b79402d1375ebed0b
Similar to commit dec9c9f35 and log_curl_request,
this ignores decoding errors when logging response
headers by passing errors='ignore' to safe_encode.
Change-Id: Ic915a7d8334e9473f300c9db670a3a8f5cda8976
Closes-Bug: #1369756
The new version of requests (2.4.0) has updated underlying urllib3
to version 1.9. Unfortunately urllib3 introduced new exception
ProtocolError. Because of that unit tests in glance are failing:
ProtocolError: ('Connection aborted.', gaierror(-2, 'Name or service not known'))
To solve this problem new urllib3 exception is caught in the same place
that the old one was. Unfortunately both exception are still in use so
I couldn't remove the old one.
Change-Id: I55eef98e734c59b9b627f182768a633b2b701e43
Closes-Bug: #1364893
If no CA certificate is provided to be used for validation then
fall back to the system-wide CA bundle as presented by
python-requests.
Change-Id: I05206a868150d4b62b6f1b833310eb9b86b7c4f8
Closes-bug: #1362179
Python 3 changed the semantics of dict.items() [0], which now returns a
iterable 'view' instead of a list of tuples. This has the repercussion
that you can no longer check for membership of a key using:
key in dict.items()
This patch simply replaces that check with a test for the key existing
in the dict itself, rather than the items.
[0] http://legacy.python.org/dev/peps/pep-3106/
Closes-Bug: 1359880
Change-Id: I7c59b0432725b660c9fa7270cde2e07bf3ea77db
A bug was introduced which meant that the server SSL certificate was
not being verified. Here we make sure that it is checked (unless
the --insecure flag is used).
Helps guard against man-in-the-middle attack.
Change-Id: I08f30bf3906b6580c871729311343fa8eefda91b
Closes-bug: #1357430
To help end user uses profiling by glanceclient CLI directly, the
change added '--profile <HMAC_KEY>' argument to the shell.
This change also fixed a function regression, it was used to pass
necessary trace info to glance server via http header:
dbb242b776 (diff-740dd7d9149f46fe4c01ef0ab7eb3bfbL196)
In addition:
1. If client app want to enable profiling feature for glanceclient
object, please see: http://paste.openstack.org/show/85722/
2. Afer adding more trace point into Client object, as fundamental
requirement of osprofiler, what we need to notice caller is providing
correct messaging notifier in the context code.
The relevant change for glance server is ready at:
I45a19f5eb5304c2b78a9e12cbc0744941a807304
DocImpact
Change-Id: If42b69b2695a5f88536ecbc2c1b3592d9dbd880a
Signed-off-by: Zhi Yan Liu <lzy.dev@gmail.com>
Some proxy or gateway softwares, e.g. jumpgate [0], use url-pattern
based approach to match which hanlding logic needs to be triggered for
particular service calling when it received a http(s) call as a
middleman. The change fixed an issue which caused glanceclient send out
the request to a dis-normal url, which contains duplicated "/".
The change removed a wrong and duplicated code snippet from curl logging
function as well.
[0] http://goo.gl/yt52X1
Change-Id: Ic8b3920e11d400771ead7f9c44b615f10b4a5cef
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Downgrades the log message when an http request fails from error
to debug. The logging level changed in the http.py upgrade in
I09f70eee3e2777f52ce040296015d41649c2586a which effectively reverted
a similar fix commited in I6d0efb53d1e81adf309f7fa580ec5a8073a811c5.
We don't want to log at ERROR level because we already have an
exception raised and the caller can handle it if wants
to. However this error will appear regularly in the Nova logs
under normal behavior (say a client specifies an image which does
not exist) and there is no way of disabling only the glanceclient
error log. This results in a lot of noise in the nova log file.
Change-Id: Iec13bff439073a79cb24e9b22fd43603ae4e61b7
Setting stream=True with requests can lead to issues
with not closing the connection so the urllib3 connection
pool is not freed up, so only set stream=True if making
a request with application/octet-stream content-type.
See the body-content-workflow and keep-alive sections
in the requests docs here for more information:
http://docs.python-requests.org/en/latest/user/advanced/
Note that commit dbb242b changed the response body_iter
code to potentially return a six.StringIO object rather
than the old ResponseBodyIterator class and since the
images client code is not converting the body_iter into
a dict using json.loads, we have to do that directly
in the _request method where the body_iter is
constructed.
Co-authored-by: Flavio Percoco <fpercoco@redhat.com>
Change-Id: I86572b69c4511f933c9676108190271874346302
Partial-Bug: #1341777
This review implements blueprint python-request and replaces the old
http client implementation in favor of a new one based on
python-requests.
Major changes:
* raw_request and json_request removed since everything is now being
handled by the same method "_request"
* New methods that match HTTP's methods were added:
- get
- put
- post
- head
- patch
- delete
* Content-Type is now being "inferred" based on the data being sent:
- if it is file-like object it chunks the request
- if it is a python type not instance of basestring then it'll try
to serialize it to json
- Every other case will keep the incoming content-type and will send
the data as is.
* Glanceclient's HTTPSConnection implementation will be used if
no-compression flag is set to True.
Co-Author: Flavio Percoco<flaper87@gmail.com>
Change-Id: I09f70eee3e2777f52ce040296015d41649c2586a
To be able to create profiling traces for Glance, client should be
able to send special HTTP header that contains trace info.
This patch is as well important to be able to make cross project
traces. (Typical case nova calls glance via python client, if
profiler is initialized in nova, glance client will add extra
header, that will be parsed by special osprofiler middleware in glance
api)
Don't worry no security issue here, trace information is signed by HMAC
key that is setted in api-paste.ini. So only person that knows HMAC key
is able to send proper header.
Change-Id: Ib13084fbe9b33c2f3dee165f7d6c778546cce6ca
If --os-cacert was passed to the cli the following error was
triggered: "cafile must be None or a byte string". This is
because 'load_verify_locations' requires a byte string to
be passed in.
We fix this by explicitly converting the argument to a byte
string.
We do this in 'VerifiedHTTPSConnection' rather than sooner, eg
during arg handling, as it will no longer be required should we
move to a different http library (eg requests).
Fixes bug 1301849.
Change-Id: I9014f5d040cae9f0b6f03d8f13de8419597560cb
This debug line is causing tracebacks in the n-cpu logs for
tempest runs. Its because the logged data is sometimes unicode:
Traceback (most recent call last):
File "/usr/lib/python2.7/logging/__init__.py", line 846, in emit
msg = self.format(record)
File "/opt/stack/new/nova/nova/openstack/common/log.py", line 710, in format
return logging.StreamHandler.format(self, record)
File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
return fmt.format(record)
File "/opt/stack/new/nova/nova/openstack/common/log.py", line 674, in format
return logging.Formatter.format(self, record)
File "/usr/lib/python2.7/logging/__init__.py", line 467, in format
s = self._fmt % record.__dict__
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1234: ordinal not in range(128)
Logged from file http.py, line 153
The change used correct encoding error handling policy for the log
which may includes non-ascii char.
Closes-bug: 1320655
Change-Id: I97f5f14b9beddcceb7fbd371062caf5a38a62a20
Class `Managers` from `glanceclient.common.base` module is similar to
class `apiclient:ManagerWithFind` from common code.
In this patch:
- class glanceclient.common.base:Managers replaced by
apiclient:ManagerWithFind
- module glanceclient.common.base marked as 'deprecated'
Related to bp common-client-library-2
Change-Id: I41da4a9188e97ca2c07b6234fc2ac0a877553d3f
Switch to using network_utils for splitting the URL. The code
in oslo-incubator supports ipv6 urls
Change-Id: I76be6173b97eb000319d30b4e9232a5a7c4a5aba
Closes-Bug: #1298137
Some backend stores e.g. RBD, will fail if told to create an
image without a valid size (RBD will fail to write to a zero-size
image). Here we add support to allow the image size to be provided
when doing an upload. The result is that the upload content-length
will be set if available either from checking the supplied file
object or as provided by user.
Closes-Bug: 1220197
Change-Id: Ia1f2ea5680a139750d931591949b3e0058148b4b
