Commit cf48e75 changed the default account/container/object ports in a
lot of places, including the probetests. However, it didn't change
them in doc/saio/bin/remakerings, and since the probe tests must match
the rings, they started failing.
This commit just backs out the changes to the test/probe directory so
that remakerings and the probe tests match again.
Change-Id: I316a09e6ee1a911f37ce9df3d641644739f88eeb
Changing the recommended ports for Swift services
from ports 6000-6002 to unused ports 6200-6202;
so they do not conflict with X-Windows or other services.
Updated SAIO docs.
DocImpact
Closes-Bug: #1521339
Change-Id: Ie1c778b159792c8e259e2a54cb86051686ac9d18
Previously, if object versioning was enabled via the old-style
allow_versions container-server setting rather than the new-style
allow_versioned_writes proxy-server setting,
TestCrossPolicyObjectVersioning would skip tests while
TestObjectVersioning and TestObjectVersioningUTF8 would run them.
Additionally, if versioned_writes was explicitly included in the
proxy-server's pipeline and allow_versioned_writes was disabled,
the functional tests would fail with a 412.
Now, all three will use the same logic to check whether versioning is
enabled. Specifically, they will all try to set an X-Versions-Location
header and skip if it doesn't stick.
Additionally, the TestCrossPolicyObjectVersioningEnv will now properly
clean up after itself.
Change-Id: I4c788a0e18587ff17d3c6e346fd22b881495f06d
Previously, the pids type was list(), which raised a ValueError in case if
pid was not in pids during the pids.remove(pid) call. The fix changes
the type of pids to set() and substitutes remove() with discard(), which
doesn't raise an Exception if a value is not in the set.
Change-Id: I8d41cb2a8ec14d29e5c7411ddfe48ae7a41deb45
Closes-Bug: #1567638
If the client asked for "Range: bytes=--123", Swift would respond with
a 206 and a Content-Length of -123. Now that Range header is ignored
just like all kinds of other invalid Range headers.
Change-Id: I30d4522d223076ce342d20c52f57ff0eb2aea1f4
Closes-Bug: 1571106
Oslo.messaging pika driver requires patching of select module if thread
is patched.
Pika driver uses select call and if it is not patched onsuming messages
blocks whole eventlet loop
Closes-Bug: #1570242
Change-Id: I9756737309f401ebddb7475eb84725f65bca01bf
In addition to the container sync stat. report, keeping per container
statistics allows administrator with more control over bytes
transfered over a specific time per user account: The per container stats
are crucial for billing purposes and provides the operator a 'progress
bar' equivalent on the container's replication status.
Change-Id: Ia8abcdaf53e466e8d60a957c76e32c2b2c5dc3fa
Previously, attempting to PUT a SLO manifest with `If-None-Match: *`
would include the header when validating the segments, causing the
upload to fail.
Now when SLO validates segments, no conditional headers will be
included in the HEAD request.
Change-Id: I03ad454092d3caa73d29e6d30d8033b45bc96136
Closes-Bug: #1569253
Previously, fallocate_reserve could result in a traceback. The
OSError being raised didn't have the proper errno set. This patch
sets the errno to ENOSPC.
Change-Id: I017b0584972ca8832f3b160bbcdff335ae9a1aa6
This change removes the use of the COPY request in the versioned
writes middleware. It changes the COPY verb for GETs and PUTs
requests. The main reasoning for this change is to remove any
dependency that versioning had on copy, which will allow for the COPY
functionality to be moved to middleware and to be to the left of the
versioned writes middleware in the proxy pipeline. In this way,
no COPY request will ever arrive at the versioned writes middleware.
A side benefit of this change is that it removes a HEAD request from
the PUT path. Instead of checking if a current version exists, a
GET request is sent, in case of success, a PUT is sent to the
versions container.
A unit test was removed that tested non-default storage policies.
This test is no longer necessary, since it was used to test
specific policy handling code in the COPY method in the proxy
object controller.
Closes-Bug: #1365862
Change-Id: Idf34fa8d04ff292df7134b6d4aa94ff40887b3a4
Co-Authored-By: Alistair Coles <alistair.coles@hp.com>
Co-Authored-By: Janie Richling <jrichli@us.ibm.com>
Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Follow up for change Ibbc89449e7878fc4215e47e3f7dfe4ae58a2d638
to add a test assertion that the ChunkWriteTimeout contexts are
exited without raising the timeout exception in
iter_bytes_from_response_part().
Change-Id: I6d323cb26779e457fb5940093a81b349b333a0af
When direct_client.direct_get_suffix_hashes raises a
DirectClientException the exception message and variables
should report the replication_ip and replication_port, as
opposed to the ip and port values reported for all other
case when the exception is raised.
Add option to override ip and port reported in
DirectClientException.
Also adds unit tests to verify both cases.
Related-Bug: 1566395
Change-Id: If3d952847c7199f4e9f6164858085367266386d2
This is a follow-up patch for https://review.openstack.org/#/c/258392/
That one added good unit test cases for various kinds of
allowe_origin like '*' or ''(empty). However, the result of handling
in Swift proxy will depend on strict_cors_mode option configuration.
This patch refactors the unit tests to split out for
strict_cors_mode = on/off and add some missing unit tests for
each case.
Change-Id: I55f7cd279436b5c9f71d81fecf06021380e35579
direct_client.direct_get_suffix_hashes doesn't use replication ip and
port for REPLICATE request. Since we have an option of doing
replication in separate network, we can add replication_ip and port
while creating rings if not it will get filled in with the regular
node's ip.
Change-Id: I34067df27042fc3146b795191ab8043ee1aed3ce
Closes-Bug:1566395
Previously, versioned_writes assumed that all container servers would
always have the latest Swift code, allowing them to return reversed
listings. This could cause the wrong version of a file to be restored
during rolling upgrades.
Now, versioned_writes will check that the listing returned is actually
reversed. If it isn't, we will revert to getting the full (in-order)
listing of versions and reversing it on the proxy.
Change-Id: Ib53574ff71961592426cb386ef00a75eb5824def
Closes-Bug: 1562083
_validate_policy_name always either returns True or raises an exception.
Simplify it to just being a callable that may raise an exception.
Also, move the check for blank/None names into _validate_policy_name, so
it will be applied in more cases.
Change-Id: I7832a0c9c895cd75ba4c6d0e8b5568a3c8a0ea25
When eventlet.wsgi closes an ECAppIter on client disconnect we need to
make sure our sub-iterators are also closed. We already kill the
backend sockets, but the executing contexts are left running until
they timeout. A slow client can result in needlessly holding queued
backend fragments until the client_timeout (default 60s).
Update associated test that exposed the problem to demonstrate the
issue more quickly.
Change-Id: Ibbc89449e7878fc4215e47e3f7dfe4ae58a2d638
Extended the use of the DatabaseBroker "stale_reads_ok" flag to the
AccountBroker and ContainerBroker. Now checks for an sqlite3 error
from the _commit_puts call that processes the pending files.
If this error is raised, then the stale_reads_ok flag will be checked
to determine how to proceed as opposed to simply raising.
The first time that print_info is attempted, the flag will be
false, but swift-[account|container]-info will check for the
raised exception. If it was raised, then a warning is reported
that the data may be stale, and another attempt will be
made using the stale_reads_ok=True flag.
Change-Id: I761526eef62327888c865d87a9caafa3e7eabab6
Closes-Bug: 1531302
Content-Disposition headers should make no difference between
GET and HEAD according to HTTP rfc.
Closes-Bug: #1539805
Change-Id: Ifa41a7cda2f321eb8e36420ede7912ed0a549712
Follow up for change [1] to add some assertions to check that
marker param is included in sequential GET requests sent during
a full listing.
Extract multiple FakeConn class definitions to single class at
module level and share between all classes.
Also, explicitly unpack the return values from base request calls
made in the full listing section of base_request, and explicitly
return a list to make more consistent with rest of the method.
[1] Change-Id: I6892390d72f70f1bc519b482d4f72603e1570163
Change-Id: Iad038709f46364b8324d25ac79be4317add79df5
The internal_client is used in swift-dispersion-report, and in case one has more
than 10000 containers or objects these are not queried.
This patch adds support to the internal_client to iterate over all
containers/objects if the listing exceeds the default of 10000 entries and the
argument full_listing=True is used.
Closes-Bug: 1314817
Closes-Bug: 1525995
Change-Id: I6892390d72f70f1bc519b482d4f72603e1570163
The object auditor raises an exception if there are some files in
/srv/node (or any other defined "devices" directory). This change
simply skips any file in the devices directory when generating
locations for the object auditor.
Change-Id: I934594994adc577799723edb6c5648685682a9e7
