Users are able to change versioning in a container
from X-Versions-Location to X-History-Location, which affects
how DELETEs are handled. We have some unit tests that check this
behavior, but no functional tests.
This patch adds a functional test that helps us understand and
document how changing modes affects the handling of DELETE
requests.
Change-Id: I5dbe5bdca17e624963cb3a3daba3b240cbb4bec4
There is a sharding edge case where more CleaveContext are generated and
stored in the sharding container DB. If this number get's high enough,
like in the linked bug. If enough CleaveContects build up in the DB then
this can lead to the 503's when attempting to list the container due to
all the `X-Container-Sysmeta-Shard-Context-*` headers.
This patch resolves this by tracking the a CleaveContext's last
modified. And during the sharding audit, any context's that hasn't been
touched after reclaim_age are deleted.
This plus the skip empty ranges patches should improve these handoff
shards.
Change-Id: I1e502c328be16fca5f1cca2186b27a0545fecc16
Closes-Bug: #1843313
As part of Train community goal 'Support IPv6-Only Deployments and Testing'[1],
Tempest has defined the new job 'tempest-ipv6-only'(adding
in Depends-On patch) which will deploy services on IPv6 and run smoke
tests and IPv6 related tests present in Tempest.
This job will be part of Nova, Neutron, Cinder, Keystone, Glance, Swift
gate.
Verification structure will be:
- 'devstack-IPv6' deploy the service on IPv6
- 'devstack-tempest-ipv6' run will verify the IPv6-only setting and listen address
- 'tempest-ipv6-only' will run the smoke + IPv6 related test case.
This commit adds the new job 'tempest-ipv6-only' run on gate.
Story: #2005477
Task: #35932
[1] https://governance.openstack.org/tc/goals/train/ipv6-support-and-testing.html
Change-Id: I78be2ee5a7f1e5d3188ece98d7d8324f1c9bd0e3
Previously, versioned_writes middleware copy an already existing
object using PUT. However, SLO requires the additional query
to properly update the object size when listing.
Propose fix: In _put_versioned_obj - which is called when on
creating version obj and also on restoring obj,
if 'X-Object-Sysmeta-Slo-Size' header is present it will
add needed headers for container to update obj size
Added a new functional test case with size assertion for slo
Change-Id: I47e0663e67daea8f1cf4eaf3c47e7c8429fd81bc
Closes-Bug: #1840322
We have implicit branch matchers, so there's no need to add a check
for not-ocata etc, a job is only run for the branch it's on - like
master now.
Remove it to not confuse Zuul when multiple branches matches and the job
definition is different.
Change-Id: I6a346c9141aad1aa8a7393c899d5571057073e7a
Recent versions of py27 [1] have begun raising InvalidURL if you try to
include non-ASCII characters in the request path. This was observed
recently in the periodic checks of stable/ocata and stable/pike. In
particular, we would spin up some in-process servers in
test.unit.proxy.test_server.TestSocketObjectVersions and do a container
listing with a prefix param that included raw (unquoted) UTF-8. This
query string would pass unmolested through the proxy, tripping the
InvalidURL error when bufferedhttp called putrequest.
More recent versions of Swift would not exhibit this particular failure,
as the listing_formats middleware would force a decoding/re-encoding of
the query string for account and container requests. However, object
requests with errant query strings would likely be able to trip the same
error.
Swift on py3 should not exhibit this behavior, as we so
thoroughly re-write the request line to avoid hitting
https://bugs.python.org/issue33973.
Now, always parse and re-encode the query string in bufferedhttp. This
prevents any errors on object requests and cleans up any callers that
might use bufferedhttp directly.
[1] Anything after https://github.com/python/cpython/commit/bb8071a;
see https://bugs.python.org/issue30458
Closes-Bug: 1843816
Change-Id: I73f84b96f164e6fc5d3cb890355871c26ed271a6
Related-Change: Id3ce37aa0402e2d8dd5784ce329d7cb4fbaf700d
Related-Change: Ie648f5c04d4415f3b620fb196fa567ce7575d522
AWS seems to support this, so let's allow s3api to do it, too.
Previously, S3 clients trying to use multi-character delimiters would
get 500s back, because s3api didn't know how to handle the 412s that the
container server would send.
As long as we're adding support for container listings, may as well do
it for accounts, too.
Change-Id: I62032ddd50a3493b8b99a40fb48d840ac763d0e7
Co-Authored-By: Thiago da Silva <thiagodasilva@gmail.com>
Closes-Bug: #1797305
Modify the AWS S3 Api section in middleware document.
Add how to create ec2 credential and minimun configuration to use
s3 api.
Change-Id: Id4d614d8297662f16403fdfe526e14714a21249d
Closes-Bug: #1842884
There's still one problem, though: since swiftclient on py3 doesn't
support non-ASCII characters in metadata names, none of the tests in
TestReconstructorRebuildUTF8 will pass.
Change-Id: I4ec879ade534e09c3a625414d8aa1f16fd600fa4
Either all or none of account, container, and object should be provided.
If we get some but not all, that's indicating some kind of a coding bug;
there's a chance it may be benign, but it seems safer to fail early and
loudly.
Change-Id: Ia9a0ac28bde4b5dcbf6e979c131e61297577c120
Related-Change: Ic2e29474505426dea77e178bf94d891f150d851b
Previously, our unit tests with socket servers would let eventlet
capitalize headers on the way out, which
- isn't something we want to have eventlet do, because it
- breaks unicode-in-header-names on py3, so it
- is already disabled in swift.common.wsgi.run_server() for real servers.
Include a test to make sure we don't forget about it in the future.
Change-Id: I0156d0059092ed414b296c65fb70fc18533b074a
Whether the frag is durable or non-durable greatly affects how much I
care whether I can reconstruct it.
Change-Id: Ie6f46267d4bb567ecc0cc195d1fd7ce55c8cb325
