openstack/python-swiftclient
Code Issues Proposed changes

Fix container delete throughput and 409 retries.

Browse Source 
Fix race condition in _delete_container() where all elements of
object_queue have been removed, but the last one (per thread) may not
have actually been deleted yet when the container deletion thread calls
conn.delete_container(container).  Fixes bug 1032879.

Improves container deletion throughput by immediately deleting
containers with no objects instead of waiting for all pending object
deletes to complete.  Fixes bug 1032878.

Change-Id: I404229a4c608995294e0ada77724ac8afe8d6f3c
This commit is contained in:
Darrell Bishop 2012-08-03 19:27:07 -07:00
parent 1f6096e063
commit c87458e229
1 changed files with 6 additions and 2 deletions

8
bin/swift

@ -215,16 +215,20 @@ def st_delete(parser, args, print_queue, error_queue):
def _delete_container(container, conn): def _delete_container(container, conn):
try: try:
marker = '' marker = ''
had_objects = False
while True: while True:
objects = [o['name'] for o in objects = [o['name'] for o in
conn.get_container(container, marker=marker)[1]] conn.get_container(container, marker=marker)[1]]
if not objects: if not objects:
break break
had_objects = True
for obj in objects: for obj in objects:
object_queue.put((container, obj)) object_queue.put((container, obj))
marker = objects[-1] marker = objects[-1]
while not object_queue.empty(): if had_objects:
sleep(0.01) # By using join() instead of empty() we should avoid most
# occurrences of 409 below.
object_queue.join()
attempts = 1 attempts = 1
while True: while True:
try: try: