Proxy-server now requires Content-Length in the response header
when getting object and does not support chunked transferring with
"Transfer-Encoding: chunked"
This doesn't matter in normal swift, but prohibits us from putting
any middelwares to execute something like streaming processing of
objects, which can't calculate the length of their response body
before they start to send their response.
Change-Id: I60fc6c86338d734e39b7e5f1e48a2647995045ef
This patch makes test_ringbuilder create a temporal directory,
run ring builder commands under it, and delete it after testing
for each test cases, to fix temp file leaking.
Change-Id: I6f59fe095ea6485af0e60b5a8e8fc3892e0a0f90
This renames test_object_delete_at_aysnc to
test_object_delete_at_async to match with the function
async_update which it tests.
Change-Id: I726afcbf08a3449c7af2834e573b97be378a86cd
Closes-Bug: 1550067
The change in 684c4c04 to improve py3 support breaks older middleware
that consumes the SwiftLogger.
Good for Swift code to not to use warn because it's deprecated.
Bad for LogAdapter to not faithfully provide the current python logging
module API (deprecated or otherwise).
Change-Id: Ib0baf66b7f33f91562867076ff0526a287d8e5a6
In common/test_utils.py, TestStatsdLogging had the majority of its
test cases calling the real socket.getaddrinfo(), which uses real
DNS. This is very slightly slower than using a mock getaddrinfo() when
the machine running the tests has functioning DNS, but on a machine
with no network connection at all, the tests are excruciatingly slow
due to timeouts.
This commit mocks things out as appropriate. There's still one user of
the real getaddrinfo(), but it's for ::1, so that's just local
resolution based on /etc/hosts.
Timing numbers for "./.unittests test.unit.common.test_utils:TestStatsdLogging":
* network, without this patch: 1.8s
* no network, without this patch: 221.2s (ouch)
* network, with this patch: 1.1s
* no network, with this patch: 1.1s
Change-Id: I1a2d6f24fc9bb928894fb1fd8383516250e29e0c
Change Ifcc4392ffa4df485ce3c62a35d0fcc8c93de74aa introduced
TestContainerController4Replicas as a subclass of
TestContainerController. And there is duplicated code
block to override setUp method.
This patch removed the duplicated code block.
Change-Id: Iad0b64cba52554b89dbd959475a2ce7c0d9a20eb
There are some unit test cases which assumes 200 as the response
of PUT request, but 200 is never returned in PUT.
This patch replaces 200 in PUT by 201, and adapt unit test cases
to real case.
Change-Id: I35fc7a4fe7fcc5558a9e6c6371105793c2710f3e
This patch provides unit tests to check responses for various situation
in three/four replicas env.
Change-Id: Ifcc4392ffa4df485ce3c62a35d0fcc8c93de74aa
Previously, if a user could write to (but not read from) a container,
the behavior for object POST would vary depending on whether
object_post_as_copy was enabled (403 response) or disabled (202
response).
Now, POSTs will consistently be allowed, regardless of whether fast-POST
is enabled.
Change-Id: I1d6dcbc4f5034a322a1073850fc3b059ebb1c0fa
Consider two replicas of the same object whose ondisk files
have diverged due to failures:
A has t2.ts
B has t1.data, t4.meta
(The DELETE at t2 did not make it to B. The POST at t4 was
rejected by A.)
After ssync replication the two ondisk file sets will not be
consistent:
A has t2.ts (ssync cannot POST t4.meta to this node)
B has t2.ts, t4.meta (ssync should not delete t4.meta,
there may be a t3.data somewhere)
Consequenty the two nodes will report different hashes for the
object's suffix, and replication will repeat, always with the
inconsistent outcome. This scenario is reproduced by the probe
test added in this patch.
(Note that rsync replication does result in (t2.ts, t4.meta)
on both nodes.)
The solution is to change the way that suffix hashes are
calculated. Currently the names of *all* files found in each
object dir are added to the hash. With this patch the
timestamps of only those files that could be used to
construct a valid diskfile are added to the hash. File
extensions are appended to the timestamp so that in most
'normal' situations the result of the hashing is the same
as before this patch. That avoids a storm of hash mismatches
when this patch is deployed in an existing cluster.
In the problem case described above, t4.meta is no longer
added to the hash, since it is not useful for constructing
a diskfile. (Note that t4.meta is not deleted because it
may become useful should a t3.data be replicated in future).
Closes-Bug: 1534276
Change-Id: I99e88b8d5f5d9bc22b42112a99634ba942415e05
Swift object-server will return 409 Conflict when the incomming
request X-Timestamp is older than the local disk file to prevent
wasted data transfer and disk space consumption. However, IFF,
the local disk file is a tombstone, current object-server will
write the data in the device and cleanup it according to timestamp
comparison result in hash_cleanup_list_dir, and then return 201
Created. That's wasted and far from the semantics in the normal case.
This patch fixes object-server to catch the DiskFileDeleted exception
at PUT request, and then, set the timestamp from tombstone as
original_timestamp to be able to compare with request timestamp.
Co-Authored-By: Kazuhiro Miyahara <miyahara.kazuhiro@lab.ntt.co.jp>
Change-Id: I078c9cb90707a3c320708e76ea42cbfa73e1ea4b
Closes-Bug: #1546865
Some doc files and swift.conf-sample were added execute permissions in past changes.
This patch remove execute permissions from them.
Change-Id: Id8844989a8321578e9207566ebd6660f5b9523f0
Using unprintable characters for swift_hash_path_prefix/suffix might lead to
hard problems when parsing of these values changes, for example due to newer
Python versions or changes in the parsers itself. Let's avoid this and add a
note that deployers should use printable strings for these values.
Change-Id: I976982b753b6af831ab91d7190f50f8f15bf73bf
Added a link to the API reference (where headers and query
strings are documented; click the "detail" button to see).
Also added a reference to Swift section of the OpenStack end
user guide. This contains some additional details about the API.
No attempt was made to reconcile duplicate information. Instead
this patch links documents that might otherwise be overlooked.
However, I fixed text, originally in a table, that had become
garbled in a prior patch.
Change-Id: I0910cbeb0c8bffc00e510f35585603e7b7a67790
