move glanceclient to keystoneauth as keystoneclient's auth session,
plugins and adapter code has been deprecated.
refer to [1] for more information.
1: 1a84e24fa4
implements bp: use-keystoneauth
Co-Authored-By: Itisha <ishadewan07@gmail.com>
Change-Id: I88fb327628e1bec48dc391f50d66b3deab4a8ab9
* exc.py: Encode body in response before calling replace over it.
* http.py: prepend the bytes literal to the empty string or else
we hit bug 1342080 again in python 3.
* utils.py: Use sys.stdin.buffer in python 3.
Change-Id: Ieefb8c633658e507486438e5518c5d53e819027d
As of now request-id is not logged if an excpetion
is raised.
Rearranged code so that request-id is logged even in case of
an exception.
Change-Id: Iee0398404ee752c0d880edf3054207c35862e71a
Closes-Bug: #1603863
String interpolation should be delayed to be handled by the logging
code, rather than being done at the point of the logging call.
See the oslo i18n guideline.
* http://docs.openstack.org/developer/oslo.i18n/guidelines.html
Change-Id: If06663076e4081c6268ba88c157513723b734b31
Closes-Bug: #1596829
Added support to log 'X-Openstack-Request-Id' for each api call.
If glanceclient is used from command line then following log will
be logged on console if --debug flag is used.
DEBUG:glanceclient.common.http:GET call to glance-api for
http://172.26.88.20:9292/v2/schemas/image used request id
req-e0c7c97a-8fc0-4ce3-a669-d0b1eb5d7aae
If python-glanceclient is used in applications (e.g. Nova) then
following log message will be logged in service logs.
DEBUG glanceclient.common.http
[req-be074f1e-1c17-4786-b703-2a221751c8f4 demo demo] GET call to
glance-api for
http://172.26.88.20:9292/v1/images/detail?is_public=none&limit=20
used request id req-9b1dd929-df30-46b2-a8f2-dfd6ffbad3fc
DocImpact:
To use this feature user need to set 'default_log_levels' in third
party application. For example nova uses glance then in nova.conf
'default_log_levels' should be set as below:
default_log_levels = glanceclient=DEBUG
Implements: blueprint log-request-id
Change-Id: Ib04a07bac41ad2a5e997348f3b0bccc640169dc9
This reverts commit 9e532db8b0f0ba537edef143a6f5380a2aaa1e4b.
If glanceclient is used in multi-threaded environment, then there is a
possibility of getting invalid/wrong last request-id. To avoid this,
need to use thread local storage to store last-request-id and add
public method to return this request-id to caller.
http://specs.openstack.org/openstack/openstack-specs/specs/return-request-id.html#alternatives
Change-Id: I08d8d87fc0cc291f1b930b2c0cfc110ec8394131
Headers were encoded in HTTPClient, but when glance client started
to use SessionClient this functionality was lost.
This commit replaces static method "encode_headers" from HTTPClient
and makes it a common function, that SessionClient can use when
converting image meta to headers.
Change-Id: If9f8020220d2a0431b4241b38b9c83c09c0d75cb
Closes-bug: #1574587
apiclient.base.Resource.get method requires manager.client to have
last_request_id member. Otherwise get operation fails with
AttributeError exception.
Change-Id: I0ece85e3f61f2a7f176520ddf3ebee7792e51993
Closes-bug: 1552533
Previously auth_token was initialized once in __init__ method.
After that we stored token in session headers. So to refresh token
users need to instantiate a new session inside http client or
re-create client itself.
In order to provide possibility to refresh token we need
to add token header before sending the request. So users can
just update auth_token attribute in the HTTPClient to refresh
user token.
Change-Id: Ifebe9011870bbddc46fc6d6a26563641d5559e97
Closes-Bug: #1563495
glance has a code specific for python2.5. We need to delete this
code cause glanceclient doesn't support neither python2.5 or
python2.6.
Change-Id: I17e4905b6e02fcfff033a6cde03324e2a47bfce2
Not only is this code broken on the requests we require on
distro-provided requests, it's not needed anymore. Remove it.
Closes-bug: 1526254
Change-Id: I47a07bf9910f118392785fc20e015f036a2e8a7c
We need all the headers to be safe strings so they can be joined
together and not become an unicode string in doing so.
This fixes a bug when creating an image with non-ascii characters in the
name.
This is required for python 2.6 compatibility.
Change-Id: I66ebc27edf4ccd8f903399da58705711c372536d
Closes-Bug: 1448080
We have a basic implementation for a fallback mechanism that will use v1
rather than v2 when downloading schema files from glance-api fails.
However, this is not sound. If the schemas are cached already, we won't
check if v2 is available and fail to fallback.
This patch fixes the aforementioned issue by getting the list of
available versions from the server only when the API versions was not
explicitly specified through the CLI. That is, for all commands that
don't pass `--os-image-api-version 2`, we'll check v2's availability and
we'll fallback to v1 if it isn't available.
This patch also changes how we handle `/versions` calls in the client.
The server has been, incorrectly, replying to requests to `/version`
with a 300 error, which ended up in the client re-raising such
exception. While I think 300 shouldn't raise an exception, I think we
should handle that in a spearate patch. Therefore, this patch just
avoids raising such exception when `/version` is explicitly called.
This fallback behaviour and the check on `/versions` will be removed in
future versions of the client. The later depends on this bug[0] being
fixed.
[0] https://bugs.launchpad.net/glance/+bug/1491350
Closes-bug: #1489381
Change-Id: Ibeba6bc86db2a97b8a2b4bd042248464cd792e5e
Custom SSL handling was introduced because disabling SSL layer compression
provided an approximately five fold performance increase in some
cases. Without SSL layer compression disabled the image transfer would be
CPU bound -- with the CPU performing the DEFLATE algorithm. This would
typically limit image transfers to < 20 MB/s. When --no-ssl-compression
was specified the client would not negotiate any compression algorithm
during the SSL handshake with the server which would remove the CPU
bottleneck and transfers could approach wire speed.
In order to support '--no-ssl-compression' two totally separate code
paths exist depending on whether this is True or False. When SSL
compression is disabled, rather than using the standard 'requests'
library, we enter some custom code based on pyopenssl and httplib in
order to disable compression.
This patch/spec proposes removing the custom code because:
* It is a burden to maintain
Eg adding new code such as keystone session support is more complicated
* It can introduce additional failure modes
We have seen some bugs related to the 'custom' certificate checking
* Newer Operating Systems disable SSL for us.
Eg. While Debian 7 defaulted to compression 'on', Debian 8 has compression
'off'. This makes both servers and client less likely to have compression
enabled.
* Newer combinations of 'requests' and 'python' do this for us
Requests disables compression when backed by a version of python which
supports it (>= 2.7.9). This makes clients more likely to disable
compression out-of-the-box.
* It is (in principle) possible to do this on older versions too
If pyopenssl, ndg-httpsclient and pyasn1 are installed on older
operating system/python combinations, the requests library should
disable SSL compression on the client side.
* Systems that have SSL compression enabled may be vulnerable to the CRIME
(https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2012-4929) attack.
Installations which are security conscious should be running the Glance
server with SSL disabled.
Full Spec: https://review.openstack.org/#/c/187674
Blueprint: remove-custom-client-ssl-handling
Change-Id: I7e7761fc91b0d6da03939374eeedd809534f6edf
This commit enables new flake8 checks:
* E265 block comment should start with '# '
* H405 multi line docstring summary not separated with an empty line
* E123 closing bracket does not match indentation of opening bracket's line
* H238 old style class declaration, use new style (inherit from `object`)
* E128 continuation line under-indented for visual indent
and makes related changes in the code.
Change-Id: Ie993afc930f6b74d7a990bcaa9fc0e9f5ba1585c
As stated in the OpenStack Hacking Guidelines, it is prefered
that only modules should be imported.
Also updated tox.ini to ignore opestack/common among others.
Change-Id: I2f0a603c31052eadee581c11880c0ec6bd392829
To make this work we create a different HTTPClient that extends the
basic keystoneclient Adapter. The Adapter is a standard set of
parameters that all clients should know how to use like region_name and
user_agent. We extend this with the glance specific response
manipulation like loading and sending iterables.
Implements: bp session-objects
Change-Id: Ie8eb4bbf7d1a037099a6d4b272cab70525fbfc85
The oslo.utils libraries are moving away from namespace packages.
This requires oslo.utils>=1.2.0
bp drop-namespace-packages
Change-Id: I803df61e91eabb96329d859aef6bea03530fb84f
Some generated header values may in fact be None. Trying to encode None
causes the client to fail with an exception and cannot be worked around
by the user.
Change-Id: I638b1fba0ef9a07d726445d8c2cdd774140f5b83
Closes-bug: 1415935
This module now lives in oslo.utils, so import it from there instead.
Co-Authored-By: Ian Cordasco <ian.cordasco@rackspace.com>
Change-Id: Ib35dc840992433542490670781badd9529ec8947
If we don't explicitly close a response after streaming its download,
then we can run into HTTPConnectionPool full warnings. It also will hurt
performance if we have to continuously create new sockets for new
responses. Calling close will return the connection to the pool so it
can be reused. Note this is only necessary when streaming a response. If
we don't stream it, then requests will return the connection to the pool
for us.
Change-Id: I803bd4dd0e769c233501d5e5ff07a19705fbe233
Closes-bug: 1341777
The returns from requests' response.content is a bytes type. Under
python 3 this fails in error handling and string conversion. The
response.text variable should be used to treat a response body as a
string.
Closes-Bug: #1407531
Change-Id: Ifd588b5f6820ef21beb186d88d0b3f1a267695aa
When file size is an exact multiple of chunk_size, glance client
is processing EOF in image-data as a chunk and sends to
glance-api. The server treats this as the end of chunked
transmission and sends a http response.
When the actual last chunk is sent by the 'requests' library, the
server sends a 400 response and tracebacks with broken pipe
as the client has already closed the socket.
Closes-Bug: #1342080
Change-Id: Icdbff838450db1c252ddc919a230a7d3ca16765f
Change I09f70eee3e2777f52ce040296015d41649c2586a, introduced a bug where
the identity_headers are not added to the request headers anymore
causing the former to be completely ignored and useless.
This patch fixes that issue by restoring the previous code. A new test
has been added to avoid regressions.
Closes-bug: #1394965
Change-Id: I1b1633636448398cf3f41217f1d671b43ebd9946
python-glanceclient displays curl statements for debugging/troubleshooting
purposes. For IPv6 URLs, curl requires --globoff to be passed in the
arguments. Since glanceclient does not use curl directly, this patch
displays the curl commands with globoff option which works for both
IPv4 and IPv6 URLs.
Fix adapted from python-novaclient Ib7099e8e3bbc15f29bbaa1db37ef21e78a74e7bc
Closes-Bug: #1228744
Change-Id: Ie02c4e75ca1ab995102aa55bbff39b2161218b2d
Commit f980fc549247fa2deb87dfacebc6d8d13ccd45d1 changed how the
X-Auth-Token header was scrubbed when logging the request, but
effectively made the value required which can lead to an AttributeError
if the value for the header is None.
The value can be None if you're using Nova but don't have Nova
configured with auth_strategy='keystone' (see
nova.image.glance._create_glance_client for details).
This patch simply checks if the auth_token is set in the http client
object and if not, it doesn't set the X-Auth-Token key in the session
header.
Closes-Bug: #1381295
Change-Id: Ie285d5253df28a9f0f964147a53c99ceaa919c5c
In order to keep the support for `--ssl-nocompression` it was decided to
overwrite the https HTTPAdapter in `requests` poolmanager. Although this
seemed to work correctly, it was causing some issues when using
glanceclient from other services that rely on requests and that were
also configured to use TLS.
THis patch changes implements a different strategy by using
`glance+https` as the scheme to use when `no-compression` is requested.
Closes-bug: #1350251
Closes-bug: #1347150
Closes-bug: #1362766
Change-Id: Ib25237ba821ee20a561a163b79402d1375ebed0b
Similar to commit dec9c9f35 and log_curl_request,
this ignores decoding errors when logging response
headers by passing errors='ignore' to safe_encode.
Change-Id: Ic915a7d8334e9473f300c9db670a3a8f5cda8976
Closes-Bug: #1369756
The new version of requests (2.4.0) has updated underlying urllib3
to version 1.9. Unfortunately urllib3 introduced new exception
ProtocolError. Because of that unit tests in glance are failing:
ProtocolError: ('Connection aborted.', gaierror(-2, 'Name or service not known'))
To solve this problem new urllib3 exception is caught in the same place
that the old one was. Unfortunately both exception are still in use so
I couldn't remove the old one.
Change-Id: I55eef98e734c59b9b627f182768a633b2b701e43
Closes-Bug: #1364893
If no CA certificate is provided to be used for validation then
fall back to the system-wide CA bundle as presented by
python-requests.
Change-Id: I05206a868150d4b62b6f1b833310eb9b86b7c4f8
Closes-bug: #1362179
