Even if stdlib socket is probably patched by the time StatsdClient
creates a socket, we want to import the green socket module explicitly
for better testing.
Move test_statsd.py to test_statsd_client.py so it matches the naming
convention of the rest of our test files.
Fix some patching of utils in test_statsd_client to patch
statsd_client.
Rename some vars in test_statsd_client that shadowed the statsd_client
module name.
Move some utils tests out of test_statsd_client and back into
test_utils.
Related-Change: I4b5b12a3b0288b696a39903264741bc862a94ad7
Change-Id: I3de22b7f15dd386fa9c873587782f0dfc4c42a27
Adds ClosingMapper class which is like map() but closes the
iterable.
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Change-Id: Idd0ac21b365a138b065f01d05a257af62ea88177
This shouldn't impact real servers, as those processes were about to
wrap up anyway. It *can* cause some confusing behaviors in tests,
though.
Change-Id: Ifd8a64efcd3fc983596ba7cd9fe28eb9663c93d6
- Move statsd client into it's own module
- Move all logging functions into their own module
- Move all config functions into their own module
- Move all helper functions into their own module
Partial-Bug: #2015274
Change-Id: Ic4b5005e3efffa8dba17d91a41e46d5c68533f9a
If the global configuration option 'enable_open_expired' is set
to true in the config, then the client will be able to make a
request with the header 'x-open-expired' set to true in order
to access an object that has expired, provided it is in its
grace period. If this config flag is set to false, the client
will not be able to access any expired objects, even with the
header, which is the default behavior unless the flag is set.
When a client sets a 'x-open-expired' header to a true value for a
GET/HEAD/POST request the proxy will forward x-backend-open-expired to
storage server. The storage server will allow clients that set
x-backend-open-expired to open and read an object that has not yet
been reaped by the object-expirer, even after the x-delete-at time
has passed.
The header is always ignored when used with temporary URLs.
Co-Authored-By: Anish Kachinthaya <akachinthaya@nvidia.com>
Related-Change: I106103438c4162a561486ac73a09436e998ae1f0
Change-Id: Ibe7dde0e3bf587d77e14808b169c02f8fb3dddb3
Add an example of a delay_reaping config option with quoted key.
Change-Id: I0c7ead6795822ea0fb0e81abc1e4685d7946942c
Related-Change: I106103438c4162a561486ac73a09436e998ae1f0
The object expirer can be configured to delay the reaping of
objects from disk after their expiration time using account
and container level delay_reaping values. The delay_reaping
value of accounts and containers in seconds is configured in
the object server config. The object expirer references these
configured values to only reap objects from specified accounts
and containers after their corresponding delays.
The goal of the delay_reaping feature is to prevent accidental or
premature data loss if an object marked for deletion with the
'x-delete-at' feature should not be reaped immediately, for
whatever reason.
Configuring the delay_reaping value at a granular account and
container level is beneficial for being able to keep storage
capacity consumption in control while maintaining a desired
data recovery window.
This patch also adds a sample configuration, documentation, and
tests for bad configurations and grace period functionality.
Co-Authored-By: Anish Kachinthaya <akachinthaya@nvidia.com>
Change-Id: I106103438c4162a561486ac73a09436e998ae1f0
Add some test assertions to cover the first-byte timing metrics
introduced in the related change.
Add ttfb param to log_request docstring.
Change-Id: I530652dd672d7d4e5eac351ccbad318773414f7d
Related-Change: I1611e34846e586703e9d3709fa64e8df41f2d685
The main motivation here is that mock.call becomes a namedtuple and you
can say `m.call_args_list[0].args` instead of `m.call_args_list[0][0]`
Change-Id: Ibb1a64ef0bfdebf06d26636cdb6ea191c10705f7
Previously, we relied on raising a swob.HTTPException from within a
wrapped wsgi.input and just hoping that it either
- comes uncaught all the way back through the pipeline, or at least
- gets translated back to a 422 response.
This was brittle, however; if any middleware between s3api and the
proxy took s3api's approach of turning unexpected responses into 500s,
for example, it all breaks down.
Take a cue from eventlet's Timeout and create a new s3api-specific
BaseException (which does *not* inherit from Exception) to improve
our ability to cut through all the layers.
Change-Id: I9924ff3b8d7d246631fe61b916823e028e2c01f2
We stuff the access key into the request path until we get back a
more-authoritative account name from auth. But it needs to be a WSGI
string when we do!
Closes-Bug: #2058748
Change-Id: I34adb8141cc9e62d17a27f01c63f40d1dd25991c
Any of these directories may get unlinked between when we saw them in
their parent's directory listing and when we go to descend.
Change-Id: I1dfc0ee1d9e70cb0600557cde980bd5880bd40b3
