This patch removed the separate s3api, staticweb functional tests
gate jobs and added them across all other functional test jobs.
Change-Id: Ie1c606132a054defc2b3cc14a66031090e7b8449
Previously, the lack of container ACLs on the reserved container would
mean that attempting to grant access to the user-visible container would
not work; the user could not access the backing object.
Now, have symlinks with the allow-reserved-names sysmeta set be
pre-authed. Note that the user still has to be authorized to read the
symlink, and if the backing object was *itself* a symlink, that will be
authed separately.
Change-Id: Ifd744044421ef2ca917ce9502b155a6514ce8ecf
Closes-Bug: #1880013
The repo is Python using both Python 2 and 3 now, so update hacking to
version 2.0 which supports Python 2 and 3. Note that latest hacking
release 3.0 only supports version 3.
Fix problems found.
Remove hacking and friends from lower-constraints, they are not needed
for installation.
Change-Id: I9bd913ee1b32ba1566c420973723296766d1812f
When client sends a '?multipart-manifest=get&format=raw' request
middleware will change the manifest returned from object server.
This patch makes sure the response etag is updated to reflect
changes to manifest content
Change-Id: I0ac6dd0808fb041ba7663f4a472a06ee3f1d9a71
Users have complained for a while that Swift's ETags don't match the
expected RFC formats. We've resisted fixing this for just as long,
worrying that the fix would break innumerable clients that expect the
value to be a hex-encoded MD5 digest and *nothing else*.
But, users keep asking for it, and some consumers (including some CDNs)
break if we *don't* have quoted etags -- so, let's make it an option.
With this middleware, Swift users can set metadata per-account or even
per-container to explicitly request RFC compliant etags or not. Swift
operators also get an option to change the default behavior
cluster-wide; it defaults to the old, non-compliant format.
See also:
- https://tools.ietf.org/html/rfc2616#section-3.11
- https://tools.ietf.org/html/rfc7232#section-2.3
Closes-Bug: 1099087
Closes-Bug: 1424614
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Change-Id: I380c6e34949d857158e11eb428b3eda9975d855d
This patch adds a new object versioning mode. This new mode provides
a new set of APIs for users to interact with older versions of an
object. It also changes the naming scheme of older versions and adds
a version-id to each object.
This new mode is not backwards compatible or interchangeable with the
other two modes (i.e., stack and history), especially due to the changes
in the namimg scheme of older versions. This new mode will also serve
as a foundation for adding S3 versioning compatibility in the s3api
middleware.
Note that this does not (yet) support using a versioned container as
a source in container-sync. Container sync should be enhanced to sync
previous versions of objects.
Change-Id: Ic7d39ba425ca324eeb4543a2ce8d03428e2225a1
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Co-Authored-By: Thiago da Silva <thiagodasilva@gmail.com>
unittest2 was needed for Python version <= 2.6, so it hasn't been needed
for quite some time. See unittest2 note one:
https://docs.python.org/2.7/library/unittest.html
This drops unittest2 in favor of the standard unittest module.
Change-Id: I2e787cfbf1709b7f9c889230a10c03689e032957
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Users are able to change versioning in a container
from X-Versions-Location to X-History-Location, which affects
how DELETEs are handled. We have some unit tests that check this
behavior, but no functional tests.
This patch adds a functional test that helps us understand and
document how changing modes affects the handling of DELETE
requests.
Change-Id: I5dbe5bdca17e624963cb3a3daba3b240cbb4bec4
Previously, versioned_writes middleware copy an already existing
object using PUT. However, SLO requires the additional query
to properly update the object size when listing.
Propose fix: In _put_versioned_obj - which is called when on
creating version obj and also on restoring obj,
if 'X-Object-Sysmeta-Slo-Size' header is present it will
add needed headers for container to update obj size
Added a new functional test case with size assertion for slo
Change-Id: I47e0663e67daea8f1cf4eaf3c47e7c8429fd81bc
Closes-Bug: #1840322
AWS seems to support this, so let's allow s3api to do it, too.
Previously, S3 clients trying to use multi-character delimiters would
get 500s back, because s3api didn't know how to handle the 412s that the
container server would send.
As long as we're adding support for container listings, may as well do
it for accounts, too.
Change-Id: I62032ddd50a3493b8b99a40fb48d840ac763d0e7
Co-Authored-By: Thiago da Silva <thiagodasilva@gmail.com>
Closes-Bug: #1797305
We were (indirectly) importing swiftclient (and therefore requests and
urllib3) before doing our eventlet monkey-patching. This would lead
boto3 (which digs an SSLContext out of urllib3) to trip RecursionErrors
on py3 similar to
>>> from ssl import SSLContext, PROTOCOL_SSLv23
>>> import eventlet
>>> eventlet.monkey_patch(socket=True)
>>> SSLContext(PROTOCOL_SSLv23).options |= 0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.6/ssl.py", line 465, in options
super(SSLContext, SSLContext).options.__set__(self, value)
File "/usr/lib/python3.6/ssl.py", line 465, in options
super(SSLContext, SSLContext).options.__set__(self, value)
File "/usr/lib/python3.6/ssl.py", line 465, in options
super(SSLContext, SSLContext).options.__set__(self, value)
[Previous line repeated 330 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object
Change-Id: I4bb59edd87336597791416c4f2a096efe0e72fe3
We already had a unit test for this, but it was inadvertently retesting
what test_get_suffix_range_many_segments tested.
Change-Id: I43eee7029ca884268c3f2ad74300b94b299fd8d2
Closes-Bug: #1831790
... by embedding something like `If-Match: <etag>` semantics in the
symlink.
When creating a symlink, users may now specify an optional
X-Symlink-Target-Etag header. If present, the etag of the final object
returned to the client will be checked; if it does not match the
X-Symlink-Target-Etag header, a 409 Conflict error will be returned to
the client.
Note that, unlike "dynamic symlink" behavior, the target object must
exist with the matching Etag for the "static symlink" to be created.
Since we're validating the Etag anyway, we also set the content-type of
the symlink to match if the client didn't otherwise specifiy and send
the etag & content-length along to the container listing as well.
Bonus goodness:
- Tighten assertions on Content-Location
- Get rid of swift.source-sniffing by making versioned_writes
symlink-aware ('cause I'm going to want to make it symlink-aware
later anyway)
- Allow middlewares left of symlink to set their own
Container-Update-Override-Etag when creating a symlink
- Set dynamic symlink content type if client doesn't supply something
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: I179ea6180d31146bb947061c69b1807c59529ac8
