This is just tests to demonstrate the current behavior - and provide
context of behavioral differences/consistency in the related change.
See the related change for context.
The related change slightly modifies the outcome of the
reclaim/REPLICATE test.
The related change has no effect on the config handling.
The related change greatly simplfies the reclaim_age option
cleanup_ondisk_files plumbing tests.
Related-Change-Id: I2b9189941ac29f6e3be69f76ff1c416315270916
Co-Author: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: I5b5f90bb898a335e6336f043710a05a44e3b810f
This is a kind of the follow-up patch
https://review.openstack.org/#/c/419579
And I'd address one more my mistake with this patch, that is
inconsistent timestamp setting between
test_GET_limit_marker_json and test_GET_limit_marker_xml
Change-Id: Ia2e0c62c6146c0d79c7de2de82fe13ec4dd98165
SLOs created after commit 2d25fe6ad3573b2a06b6b3e5e66493d7b0c55693
have an explicit X-Object-Sysmeta-SLO-Size attribute. Fetching one of
those with a range like "bytes=100-" would crash and cause a 500. This
commit fixes the crash.
Closes-Bug: #1656147
Change-Id: I174ed98b4f01a5943af12177d6f8c3586f049778
Trivial fixes in follow up to [1]
- give container unique timestamps in listing test
- remove accidental line duplication
- add missing last modified time assertion
[1] Related-Change: Iba0503916f1481a20c59ae9136436f40183e4c5b
Change-Id: Ia1e9f292c62197f1f19e86936c8d05e6cff131be
For now, last modified timestamp is supported only on
object listing. (i.e. GET container)
For example:
GET container with json format results in like as:
[{"hash": "d41d8cd98f00b204e9800998ecf8427e", "last_modified":
"2015-06-10T04:58:23.460230", "bytes": 0, "name": "object",
"content_type": "application/octet-stream"}]
However, container listing (i.e. GET account) shows just a dict
consists of ("name", "bytes", "name") for each container.
For example:
GET accounts with json format result in like as:
[{"count": 0, "bytes": 0, "name": "container"}]
This patch is for supporting last_modified key in the container
listing results as well as object listing like as:
[{"count": 0, "bytes": 0, "name": "container", "last_modified":
"2015-06-10T04:58:23.460230"}]
This patch is changing just output for listing. The original
timestamp to show the last modified is already in container table
of account.db as a "put_timestamp" column.
Note that this patch *DOESN'T* change the put_timestamp semantics.
i.e. the last_modified timestamp will be changed only at both PUT
container and POST container.
(PUT object doesn't affect the timestamp)
Note that the tuple format of returning value from
swift.account.backend.AccountBroker.list_containers is now
(name, object_count, bytes_used, put_timestamp, 0)
* put_timestamp is added *
Original discussion was in working session at Vancouver Summit.
Etherpads are around here:
https://etherpad.openstack.org/p/liberty-swift-contributors-meetuphttps://etherpad.openstack.org/p/liberty-container-listing-update
DocImpact
Change-Id: Iba0503916f1481a20c59ae9136436f40183e4c5b
This does not address the underlying lp bug #1652323 with ssync that
makes ssync inconsistent with expired objects.
Change-Id: Ia12dc27a12418637587c57ff4f5744d00c661691
Related-Bug: #1652323
Closes-Bug: #1653836
Currently swift ignores a lot of the Hacking style guide. This patch
enables the H401 and H403 checks and fixes any violations. With this
we can get a little closer to following the OpenStack style guidelines.
Change-Id: I5109a052f2ceb2e6a9a174cded62f4231156d39b
The following file(s) added utf-8 encoding but never used. So we can
remove them at all.
test/functional/test_access_control.py
test/unit/common/middleware/crypto/test_keymaster.py
test/unit/obj/test_diskfile.py
Change-Id: I00adc968872ebe9f9c0619a4e393e048c7c1a91e
Addresses a TODO in test/functional/test_account.py where
an account metadata test was having to clean up tempurl keys
in the account metadata that were left by another test in
a different module. This cleanup is necessary because tests
in test_account.py fail if there is any pre-existing
account metadata.
This patch:
* makes the tempurl tests clean up their keys from account
metadata.
* makes the test_account.py:TestAccount class remove any
pre-existing metadata before attempting any tests and
replacing that metadata when all the tests in that class
have completed. This is more robust than the existing code
which only removes any tempurl keys that might be in the
account - now you could have x-account-meta-foo = bar in
the test account and test_account.py will still pass.
* consolidates some common setup code currently repeated for
many of the functional test classes into into a BaseEnv class.
Change-Id: I874a9e23dfcdd1caa934945b46089f11b9f6de65
The bandit gate is passing, let's make it voting.
Since bandit is a security linter, add it to the linter target - called
pep8 - and run it there. This will save the CI systems spawning an extra
test machine for this.
Needed-By: I90e5401ee8e46a2a153adb62d0f99a8a88a686b8
Change-Id: I75eeefa2441466da60b9b206769a2e64e9c1e7a9
Instead of printing the error message and
calling sys.exit() when a section not exists
or reading the file failed rais an Exception
from readconfig. Depending on the Value or IO-Error,
the caller can decide if he wants to exit or continue.
If an Exception reaches the wsgi utilities
it bubbles all the way up.
Change-Id: Ieb444f8c34e37f49bea21c3caf1c6c2d7bee5fb4
Closes-Bug: 1578321
TestTempURLPrefix.setUp() almost duplicates the superclass
setUp method, except it set "expires" as an instance variable.
The duplicated setUp code can be removed by simply having the
super class setUp set the "expires" instance variable.
Change-Id: Icf7604460e475523e156daa8c578765336820c8e
Change default py3x to py35 in tox.ini to support python3.5,
and fix the error when run tox test for Swfit.
Closes-Bug: #1649773
Change-Id: I8e26183ffe270518503da8b5aa394e99ae4cd8c6
This patch fixes the object-reconstructor to calculate device_count
as the total number of local devices in all policies. Previously
Swift counts it for each policy but reconstruction_device_count
which means the number of devices actually swift needs to reconstruct
is counted as sum of ones for all polices.
With this patch, Swift will gather all local devices for all policies
at first, and then, collect parts for each devices as well as current.
To do so, we can see the statuses for remaining job/disks percentage via
stats_line output.
To enable this change, this patch also touchs the object replicator
to get a DiskFileManager via the DiskFileRouter class so that
DiskFileManager instances are policy specific. Currently the same
replication policy DiskFileManager class is always used, but this
change future proofs the replicator for possible other DiskFileManager
implementations.
The change also gives the ObjectReplicator a _df_router variable,
making it consistent with the ObjectReconstructor, and allowing a
common way for ssync.Sender to access DiskFileManager instances via
it's daemon's _df_router instance.
Also, remove the use of FakeReplicator from the ssync test suite. It
was not necessary and risked masking divergence between ssync and the
replicator and reconstructor daemon implementations.
Co-Author: Alistair Coles <alistair.coles@hpe.com>
Closes-Bug: #1488608
Change-Id: Ic7a4c932b59158d21a5fb4de9ed3ed57f249d068
Previously, we still required that clients send "etag" and "size_bytes"
keys in their segment definitions. This was done as a way to guard
against typos leading to an accidental lack of verification.
However, typos should already be caught when we check for extra keys. As
a result, the only truly required key is "path".
Change-Id: Ie1d8691115f8c68b5a3f3b59317cdab59f9a3fca
Fixes: NameError: name 'file' is not defined
file() is removed in python3.x using open()
will solve it.
Fixes: AttributeError: 'generator' object has no
attribute 'next'
Python3 it is __next__()
Change-Id: Ib2cbdfbbedb86f6e01239d2ee991677727b0548b
Installed my workstation with Ubuntu Xenial and then hit a few
snags running tox.
python3-dev is needed to build the python3 envs. Did not actually
test python3-devel (rpm) as I don't have such a system.
While python2.7 is the default, tox (on Xenial) itself uses
python3.5, hence 3.5 becomes the basepython and a number
of envs don't work.
Driveby, removed warning about /bin/echo in py3pep8
Change-Id: Ie880246dd757570b8e444da0e643f3c50093bdc5
The middleware now allows the usage of signatures with a prefix-based
scope. A prefix-based signature grants access to all objects which share
the same prefix. This avoids the creation of a large amount of signatures,
when a whole container or pseudofolder is shared.
Please see spec: https://review.openstack.org/#/c/199607/
Change-Id: I03b68eb74dae6196b5e63e711ef642ff7d2cfdc9
Currently, conditional headers, e.g. If-Match, If-Modified-Since, etc.,
are listed for the GET objects API call, but not in the HEAD objects
call in the api-ref documentation. This patch adds the missing headers
to the HEAD documentation.
Change-Id: Idd1c248cc27415d148a1b9a5eb2e95f25b8c4da0
Closes-Bug: #1630212
