swift-form-signature would give you the required expiration-time and
HMAC signature, but it wouldn't help you actually construct the HTML
form. To do that, you had to go look at the formpost middleware's doc
string and make up a form yourself.
For convenience, this commit makes swift-form-signature output a
sample form with the computed values filled in already; the user only
needs to fill in the Swift cluster's hostname.
Change-Id: I70d70a648b78b382dbfbe8ff918e6158a7f6a0ab
Moved the body of bin/swift-form-signature into
swift/cli/form_signature.py, like was done with swift-ring-builder and
others. Added a couple basic tests; there's not 100% coverage, but
it's better than the 0% coverage we had before.
It's almost a straight forklift, but I changed exit() calls to return
statements.
Change-Id: Ie2f702c070da24d9cdface83b9e838e9e2965085
auth_token middleware in python-keystoneclient is deprecated and has
been moved to the keystonemiddleware repo.
Change-Id: Ia04aa83348e0776cb3239cb5420ee1450a990d5b
Closes-Bug: #1342274
I attempted to use this function and found a few problems.
We shouldn’t unlink the file after closing it, because someone else could lock
it in between. Switch to unlink before close.
If someone else locked the file between our open and flock, they are likely to
unlink it out from underneath us. Then we have a lock on a file that no longer
exists. So stat the filename after locking to make sure the inode hasn't
changed or gone away.
We probably shouldn’t unlink the file if we time out waiting for a lock. So
move that to before the finally block.
Change-Id: Id1858c97805d3ab81c584eaee8ce0d43d34a8089
This change adds the ability to specify a X-Delete-At or
X-Delete-After attribute when using the FormPost middleware.
Here is an example of what you need to add to the form:
<input type="hidden" name="x_delete_at" value="<unix-timestamp>"/>
<input type="hidden" name="x_delete_after" value="<seconds>"/>
To be inline with the other form imput names, x-delete-at/after has
changed to x_delete_at/after.
DocImpact
Change-Id: Ib1cc0bcf1dd7d2b689f2f26d100f9bab36880c81
Closes-Bug: #1065522
The new API adds better support for storage policies and changes the
response from a list of backend urls to a dictionary with a key
"endpoints" that's a list of of the backend urls and a new key headers
that's a dictionary of headers to send along with the backend request.
In the v2 response format for object requests, there the headers key
includes "X-Backend-Storage-Policy-Index" which indicates the storage
policy index for the endpoints returned in the response.
Change-Id: I706a5b5be8002c633fe97b2429256735800a902e
Added call to delete container in teardown_package().
Tests are leaving objects and containers in saio environment
after finishing running the tests.
Currently, delete_containers() is called in the setUp of each test
class, which cleans up the containers of the previous test,
but leaves the containers of the last test in the saio environment.
Change-Id: I643d7083a2a310fc3d24eab48c565f3798cff25f
Signed-off-by: Thiago da Silva <thiago@redhat.com>
If a swift cluster configures the proxy server to not accept
/info requests, just do the best we can, skipping tests as
necessary.
To that end, if cors mode is not known we also skip the test.
Change-Id: I34f296c8717e0baf24ae552a7e38f9354ee91974
If container counter per account is equal or greater than
max_container_per_account, then all PUT requests are failed and
403 is returned.
This is correct behaviour if the request is to create a new
container, however if container already exists PUT should be
allowed, even the max_container_per_account condition has met.
This patch allows to process PUT requests for existing containers,
even if max_container_per_account > = container count.
It indirectly resolve the bug 1306711, since swift-client
uses internally PUT requests for container, prior it upload an
object there.
Change-Id: I2dcf20b6feb27e346111466a565695eba4b4b1da
May not be obvious, but existing code will let you change the
disk_chunk_size just for the auditor so this just points that
out in the docs. In one short test I ran with a 4 node cluster
with 18GB of 4MB objects on it, changint he auditor chunk size
from the default of 64K to 1MB creased the auditor CPU time from
10% to 4%.
Also added test code to make sure this overridden value is
actually used and checked other auditWorker conf values as
well.
Change-Id: Ia12e1c6127877dc2124b60cd963cd0b6d5f3d6ef
...with commit 9eb797b09916574c2a4e0f5f9ea61fe120879075
With this setting to True in the code, any tools anyone writes will
automatically start preallocating, which is almost always bad these days
with SSDs. So, if you write a tool that just scan dbs for stats, it'll
commit any pending info triggering preallocation and bam, your SSDs
start to fill up. :/
Anyway, back when I wrote 9eb797b09916574c2a4e0f5f9ea61fe120879075 it
was my intention to have this off by default, everywhere; I changed all
the settings points but I failed to change the dang constant itself.
This corrects that.
Change-Id: Iada0a3f1ecfabda4a96f574903d8e2eed1f69064
Add comment to explain why we invoke the get_part method even if the
_part_shift is equal to 32.
Closes-Bug: #1335581
Change-Id: I160e9383b5e65f75ed5e89511cc7e63c51958a25
According to https://docs.python.org/3/howto/pyporting.html the
syntax changed in Python 3.x. The new syntax is usable with
Python >= 2.6 and should be preferred to be compatible with Python3.
Enabled hacking check H231.
Change-Id: I2c41dc3ec83e79181e8fd50e76771a74c393269c
Per information filed in bug (and related bugs referenced in the
report) it appears that in TestObjectController.test_client_timeout
having a matching timeout value for both self.app.client_timeout
and SlowBody() appears to work however, as expected, a smaller
value in self.app.client_timeout also works where a larger
value fails. With that short test combined with speculation and
related fixes, seems reasonable to merge what is suggested in
the bug report, drop the self.app.client_timeout to 0.05
Fixes Bug #1316716
Change-Id: Ib4c6d3bb275f6c50c62505c90656efa7ee566bc0
If upgrading from a non-storage policy enabled version of
swift to a storage policy enabled version its possible that
memcached will have an info structure that does not contain
the 'storage_policy" key resulting in an unhandled exception
during the lookup. The fix is to simply make sure we never
return the dict without a storage_policy key defined; if it
doesn't exist its safe to make it '0' as this means you're
in the update scenario and there's xno other possibility.
Change-Id: If8c88f67ba7a3180ad06b586372fe35c65807aac
With the two vector timestamp change some resolution was lost in the queue
entries that could lead to the reconciler being unable to successfully remove
a processed item from the queue in pop_queue. To ensure the queue entries
with a significant offset can be successfully removed while still handling
the re-enqueued object case issue the DELETE with the timestamp slightly later
than the maximum of the queue entries last modified time (q_record) and
misplaced objects timestamp (q_ts).
Change-Id: I87b1e4b4256af3f71defb58c8137c303d1ae63be
In the proxy, container_info can return a 'storage_policy' of None. When
you set a header value on a swob.Request to None that effectively just
delete's the key. One path through the proxy during container sync was
counting on the the 'X-Backend-Storage-Policy-Index' being set which isn't
the case if the cached container_info if for a pre-policies container.
Also clean up some test cruft, tighten up the interface on FakeConn, and add
some object controller tests to exercise more interesting failure and handoff
code paths.
Change-Id: I643e08c208efdc5a39e840d9e96710ddad54236b
It does not appear that, aside from the user-agent string, the strings
"obj-server", "obj-updater", or "obj-replicator" (or "obj-<anything>"*)
appear in the swift code base, aside from the directory containing the
object services code being named "obj".
Furthermore, the container, account, and proxy services construct their
user-agent string, as reported in the logs, using their full name. In
addition, this full name also shows up as the name of the process via
"ps" or "top", etc., which can make it easier for admins to match log
entries with other tools.
For consistency, we update the object services to use an "object-"
prefix rather than "obj-" in its user agent string.
* obj-etag does appear in a unit test, but not part of the regular
code.
Change-Id: I914fc189514207df2535731eda10cb4b3d30cc6c
If upgrading from a non-storage policy enabled version of
swift to a storage policy enabled version its possible that
memcached will have an info structure that does not contain
the 'storage_policy" key resulting in an unhandled exception
during the lookup. The fix is to simply make sure we never
return the dict without a storage_policy key defined; if it
doesn't exist its safe to make it '0' as this means you're
in the update scenario and there's xno other possibility.
Change-Id: If8e8f66d32819c5bfb2d1308e14643f3600ea6e9
