Do not include ETag when puting manifest in chunked uploads

While testing glance with Ceph Rados Gateway using latest Ceph release
(Nautilus), i've found that glance fails to upload the manifest using
dynamic large objects mode because of the value used in ETag request.

This issue has been reported to Ceph as it seems related to some recent
change in radosgw code [1].

However, checking at the upload workflow used by glance and comparing
to Swift documentation [2], I wonder if adding the etag is actually
providing any value. In the Swift the ETag header is used to validate
integrity when uploading chunks, not the manifest while glance is doing
exactly the oposite, not sending the etag in the chunks (I guess to
avoid checksuming big images, which makes sense to me) and sending it
when puting the manifest.

This patch is removing the etag header when sending the PUT request for
the manifest in chunked uploads.

[1] https://tracker.ceph.com/issues/39160
[2] https://docs.openstack.org/swift/latest/api/large_objects.html#dynamic-large-objects

Closes-bug: #1824533
Change-Id: I0b563dfcdc30026669fb089c82db8c3df7edc808
(cherry picked from commit ed356fc6b4)
This commit is contained in:
Alfredo Moralejo 2019-04-09 19:27:29 +02:00
parent 9c8364bacf
commit 053614be14
3 changed files with 9 additions and 6 deletions

View File

@ -1030,11 +1030,12 @@ class BaseStore(driver.Store):
if image_size == 0:
image_size = combined_chunks_size
# Now we write the object manifest and return the
# manifest's etag...
# Now we write the object manifest in X-Object-Manifest
# header as defined for Dynamic Large Objects (DLO) Mode.
# This request does not include ETag as PUT request has not
# actual content that we need to verify.
manifest = "%s/%s-" % (location.container, location.obj)
headers = {'ETag': hashlib.md5(b"").hexdigest(),
'X-Object-Manifest': manifest}
headers = {'X-Object-Manifest': manifest}
# The ETag returned for the manifest is actually the
# MD5 hash of the concatenated checksums of the strings

View File

@ -112,8 +112,9 @@ class SwiftTests(object):
fixture_key = "%s/%s" % (container, name)
if fixture_key not in fixture_headers:
if kwargs.get('headers'):
etag = kwargs['headers']['ETag']
manifest = kwargs.get('headers').get('X-Object-Manifest')
etag = kwargs.get('headers') \
.get('ETag', hashlib.md5(b'').hexdigest())
fixture_headers[fixture_key] = {
'manifest': True,
'etag': etag,

View File

@ -109,8 +109,9 @@ class SwiftTests(object):
fixture_key = "%s/%s" % (container, name)
if fixture_key not in fixture_headers:
if kwargs.get('headers'):
etag = kwargs['headers']['ETag']
manifest = kwargs.get('headers').get('X-Object-Manifest')
etag = kwargs.get('headers') \
.get('ETag', hashlib.md5(b'').hexdigest())
fixture_headers[fixture_key] = {
'manifest': True,
'etag': etag,