Don't stream non-binary requests

Setting stream=True with requests can lead to issues
with not closing the connection so the urllib3 connection
pool is not freed up, so only set stream=True if making
a request with application/octet-stream content-type.

See the body-content-workflow and keep-alive sections
in the requests docs here for more information:

http://docs.python-requests.org/en/latest/user/advanced/

Note that commit dbb242b changed the response body_iter
code to potentially return a six.StringIO object rather
than the old ResponseBodyIterator class and since the
images client code is not converting the body_iter into
a dict using json.loads, we have to do that directly
in the _request method where the body_iter is
constructed.

Co-authored-by: Flavio Percoco <fpercoco@redhat.com>

Change-Id: I86572b69c4511f933c9676108190271874346302
Partial-Bug: #1341777
This commit is contained in:
Matt Riedemann 2014-07-15 07:58:34 -07:00
parent f75a81000e
commit 68c1d1fbc6

@ -160,6 +160,7 @@ class HTTPClient(object):
data = chunk_body(data)
headers['Content-Type'] = content_type
stream = True if content_type == 'application/octet-stream' else False
# Note(flaper87): Before letting headers / url fly,
# they should be encoded otherwise httplib will
@ -172,7 +173,7 @@ class HTTPClient(object):
resp = self.session.request(method,
conn_url,
data=data,
stream=True,
stream=stream,
headers=headers,
**kwargs)
except requests.exceptions.Timeout as e:
@ -217,6 +218,10 @@ class HTTPClient(object):
body_iter = resp.json()
else:
body_iter = six.StringIO(content)
try:
body_iter = json.loads(''.join([c for c in body_iter]))
except ValueError:
body_iter = None
return resp, body_iter
def head(self, url, **kwargs):