openstack/tripleo-common
Code Issues Proposed changes

Increase upload concurrency

Browse Source 
Given that image uploading is likely not CPU bound, lets increase our
default number of works from max(2, CPU/2) to a minimum of 4 to a
maximum of 12 based on available CPUs.

Change-Id: I00bd1085e595e3fef337feffb44d11f550b43e39
This commit is contained in:
Alex Schultz
2018-09-26 13:51:14 -06:00
parent a28bdf912c
commit 1706489a14
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
tripleo_common/image/image_uploader.py
View File

@@ -569,8 +569,8 @@ class DockerImageUploader(BaseImageUploader):
result = self.upload_image(*first)
local_images.extend(result)
# workers will be half the CPU count, to a minimum of 2
workers = max(2, processutils.get_worker_count() // 2)
# workers will be based on CPU count with a min 4, max 12
workers = min(12, max(4, processutils.get_worker_count()))
p = futures.ThreadPoolExecutor(max_workers=workers)
for result in p.map(docker_upload, self.upload_tasks):