Fix CORS and endpoint in AWS log upload

The AWS log upload module does not behave like the rest of the
modules: it does not set CORS headers, which means that logs will
not be usable in the Zuul web UI without additional manual
configuration.  The other uploaders all set CORS settings
automatically.

Correct this in the same way that we do for other clouds.

Additionally, if an S3 bucket is created outside of us-east-1, then
objects in it must be addressed using the bucket name as hostname,
rather than the bare host s3.amazonaws.com.  Update the returned
URL to use the bucket hostname.  This works as well for buckets in
us-east-1.

Change-Id: Id92f3cf506b442d0ee5932c6e9d931df19c2cc71
This commit is contained in:
James E. Blair 2022-09-21 07:12:31 -07:00
parent 9e22cfdb0f
commit 3e7f0f5bea
1 changed files with 23 additions and 3 deletions

View File

@ -73,11 +73,13 @@ class Uploader():
if endpoint:
self.endpoint = endpoint
self.url = os.path.join(endpoint,
bucket, self.prefix)
else:
self.endpoint = 'https://s3.amazonaws.com/'
self.url = os.path.join(self.endpoint,
bucket, self.prefix)
return_endpoint = 'https://' + bucket + '.s3.amazonaws.com/'
self.url = os.path.join(return_endpoint,
self.prefix)
self.s3 = boto3.resource('s3',
endpoint_url=self.endpoint,
@ -85,6 +87,24 @@ class Uploader():
aws_secret_access_key=aws_secret_key)
self.bucket = self.s3.Bucket(bucket)
cors = {
'CORSRules': [{
'AllowedMethods': ['GET', 'HEAD'],
'AllowedOrigins': ['*'],
}]
}
client = boto3.client('s3',
endpoint_url=self.endpoint,
aws_access_key_id=aws_access_key,
aws_secret_access_key=aws_secret_key)
try:
client.put_bucket_cors(Bucket=bucket,
CORSConfiguration=cors)
except Exception as e:
# MinIO (which we use in testing) doesn't implement this method
if 'MalformedXML' not in str(e):
raise
def upload(self, file_list):
"""Spin up thread pool to upload to storage"""