Stop decoding object content

Previously, we had urllib3 (via requests) automatically decode all
responses with a Content-Encoding of deflate or gzip. This included
object downloads, which would in turn cause etag or content-length
mismatch errors. (See bug 1338464)

This was apparently added in response to a third-party proxy sitting
between the client and server which, having observed that the client
would accept gzip-encoded content while the server sent an unencoded
response, would perform the compression. (See bug 1282861)

Now, we'll no longer let requests send any default headers, nor do any
decoding.

Change-Id: I6cc30a5c12e37de06d7322533a3c36ad15397cc8
Closes-Bug: 1338464
Related-Bug: 1282861
This commit is contained in:
Tim Burke 2015-05-20 16:07:04 -07:00
parent 52b849217f
commit 7d5c85ad10

@ -170,6 +170,8 @@ class HTTPConnection(object):
self.port = self.parsed_url.port
self.requests_args = {}
self.request_session = requests.Session()
# Don't use requests's default headers
self.request_session.headers = None
if self.parsed_url.scheme not in ('http', 'https'):
raise ClientException('Unsupported scheme "%s" in url "%s"'
% (self.parsed_url.scheme, url))
@ -238,7 +240,6 @@ class HTTPConnection(object):
return old_getheader(k.lower(), v)
def releasing_read(*args, **kwargs):
kwargs['decode_content'] = True
chunk = self.resp.raw.read(*args, **kwargs)
if not chunk:
# NOTE(sigmavirus24): Release the connection back to the