Force Swift storage driver to always get a new client

At the moment, the Swift storage codebase does not have a consistent
way of using the Swift client, as it switches from grabbing an
existing client and creating new ones.

Due to the architecture of how restores happen, there are several
times where the same storage instance is held by the main process,
the process which reads from the storage and the engine (to retrieve
metadata).  All of these accesses result in unexpected behaviour which
can raise SSL errors such as: DECRYPTION_FAILED_OR_BAD_RECORD_MAC

The overall cost of creating a new client for every request is very
minimal and it eliminates all possibility of multithreading issues
that occur as per suggested by the author of the requests Python
library[0], which is used by the Swift client.

This patch eliminates all calls to create_swift() indirectly and makes
sure that all accesses to the Swift client create a new instance. In
my local testing, there was no noticable performance change but
no more SSL failures.

Closes-Bug: #1682606

[0]: https://github.com/kennethreitz/requests/issues/1906#issuecomment-201238170

Change-Id: I2d43f14943fc52fd6b6cb994251c97e00d3eb863
This commit is contained in:
Mohammed Naser 2017-04-13 16:55:04 -04:00
parent bcfe5e2f55
commit 4c83a3f7c3
No known key found for this signature in database
GPG Key ID: 481CBC90384AEC42
1 changed files with 3 additions and 6 deletions

View File

@ -46,7 +46,6 @@ class SwiftStorage(physical.PhysicalStorage):
raise
def put_file(self, from_path, to_path):
self.client_manager.create_swift()
split = to_path.rsplit('/', 1)
file_size = os.path.getsize(from_path)
with open(from_path, 'r') as meta_fd:
@ -72,7 +71,7 @@ class SwiftStorage(physical.PhysicalStorage):
:rtype: swiftclient.Connection
:return:
"""
return self.client_manager.get_swift()
return self.client_manager.create_swift()
def upload_chunk(self, content, path):
"""
@ -100,7 +99,6 @@ class SwiftStorage(physical.PhysicalStorage):
'Retrying to upload file chunk index: {0}'.format(
path))
time.sleep(60)
self.client_manager.create_swift()
count += 1
if count == 10:
LOG.critical('Error: add_object: {0}'
@ -115,7 +113,6 @@ class SwiftStorage(physical.PhysicalStorage):
:type backup: freezer.storage.base.Backup
"""
backup = backup.copy(storage=self)
self.client_manager.create_swift()
headers = {'x-object-manifest': backup.segments_path}
LOG.info('[*] Uploading Swift Manifest: {0}'.format(backup))
split = backup.data_path.rsplit('/', 1)
@ -191,12 +188,12 @@ class SwiftStorage(physical.PhysicalStorage):
"""
split = backup.data_path.split('/', 1)
try:
chunks = self.client_manager.create_swift().get_object(
chunks = self.swift().get_object(
split[0], split[1],
resp_chunk_size=self.max_segment_size)[1]
except requests.exceptions.SSLError as e:
LOG.warning(e)
chunks = self.client_manager.create_swift().get_object(
chunks = self.swift().get_object(
split[0], split[1],
resp_chunk_size=self.max_segment_size)[1]