Added logging of the x-delete-at value for object PUTs and POSTs and
made the additional log field generic for other similar use cases.
Logging the x-delete-at is useful because otherwise you might never
know why an object "disappeared" just by looking at saved proxy logs.
You'd see a successful PUT of the object followed some time by a GET
that 404s. If you still have the other logs, you could track down
what happened; but, at least with our clusters, we keep proxy logs
forever but only keep the other logs for a short while.
Change-Id: Ida559abcf8dc5666e2a7eea250a67d1c4b28b1be
Previous logging made a mess of ipv6 addresses. This just makes the
colon quote-safe so it passes through unscathed. The logs will still
be backward compatible because unquote don't care:
$ python
>>> from urllib import quote, unquote
>>> quote('2001:db8:85a3:8d3:1319:8a2e:370:7348')
'2001%3Adb8%3A85a3%3A8d3%3A1319%3A8a2e%3A370%3A7348'
>>> unquote(quote('2001:db8:85a3:8d3:1319:8a2e:370:7348'))
'2001:db8:85a3:8d3:1319:8a2e:370:7348'
>>> quote('2001:db8:85a3:8d3:1319:8a2e:370:7348', '/:')
'2001:db8:85a3:8d3:1319:8a2e:370:7348'
>>> unquote(quote('2001:db8:85a3:8d3:1319:8a2e:370:7348', '/:'))
'2001:db8:85a3:8d3:1319:8a2e:370:7348'
Change-Id: Ia13a9bc8a320cde5c56034a7f0b645913428bf21
This was an oustanding TODO for StatsD Swift metrics. A new timing
metric is tracked for (only) GET requests for accounts, containers,
and objects:
proxy-server.<req_type>.GET.<status_int>.first-byte.timing
Also updated StatsD documentation in the Admin Guide to clarify that
timing metrics are sent in units of milliseconds.
Change-Id: I5bb781c06cefcb5280f4fb1112a526c029fe0c20
The proxy_logging middleware was asserting that the response contained
either a Content-Length header or a Transfer-Encoding header. If not,
it would either add one (if app_iter was a list) or blow up
(otherwise). This blowing up is observable on a GET request to a
manifest object that references more than
swift.common.constraints.CONTAINER_LISTING_LIMIT segments.
If a response makes it up to eventlet.wsgi without a Content-Length
header, then a "Transfer-Encoding: chunked" header is automatically
stuffed into the response by eventlet. Therefore, it's not an error
for a response to not have a Content-Length header, and proxy_logging
should just let it happen.
Fixes bug 1078113.
Change-Id: I3751a8ae14dc68bab546f2746b61267a5115e252
This change replaces WebOb with a mostly compatible local library,
swift.common.swob. Subtle changes to WebOb's API over the years have been a
huge headache. Swift doesn't even run on the current version.
There are a few incompatibilities to simplify the implementation/interface:
* It only implements the header properties we use. More can be easily added.
* Casts header values to str on assignment.
* Response classes ("HTTPNotFound") are no longer subclasses, but partials
on Response, so things like isinstance no longer work on them.
* Unlike newer webob versions, will never return unicode objects.
Change-Id: I76617a0903ee2286b25a821b3c935c86ff95233f
Removed many StatsD logging calls in proxy-server and added
swift-informant-style catch-all logging in the proxy-logger middleware.
Many errors previously rolled into the "proxy-server.<type>.errors"
counter will now appear broken down by response code and with timing
data at: "proxy-server.<type>.<verb>.<status>.timing". Also, bytes
transferred (sum of in + out) will be at:
"proxy-server.<type>.<verb>.<status>.xfer". The proxy-logging
middleware can get its StatsD config from standard vars in [DEFAULT] or
from access_log_statsd_* config vars in its config section.
Similarly to Swift Informant, request methods ("verbs") are filtered
using the new proxy-logging config var, "log_statsd_valid_http_methods"
which defaults to GET, HEAD, POST, PUT, DELETE, and COPY. Requests with
methods not in this list use "BAD_METHOD" for <verb> in the metric name.
To avoid user error, access_log_statsd_valid_http_methods is also
accepted.
Previously, proxy-server metrics used "Account", "Container", and
"Object" for the <type>, but these are now all lowercase.
Updated the admin guide's StatsD docs to reflect the above changes and
also include the "proxy-server.<type>.handoff_count" and
"proxy-server.<type>.handoff_all_count" metrics.
The proxy server now saves off the original req.method and proxy_logging
will use this if it can (both for request logging and as the "<verb>" in
the statsd timing metric). This fixes bug 1025433.
Removed some stale access_log_* related code in proxy/server.py. Also
removed the BaseApplication/Application distinction as it's no longer
necessary.
Fixed up the sample config files a bit (logging lines, mostly).
Fixed typo in SAIO development guide.
Got proxy_logging.py test coverage to 100%.
Fixed proxy_logging.py for PEP8 v1.3.2.
Enhanced test.unit.FakeLogger to track more calls to enable testing
StatsD metric calls.
Change-Id: I45d94cb76450be96d66fcfab56359bdfdc3a2576
The old use of Eventlet's posthook process meant that responses that
forgot to include content-length or transfer-encoding headers would
get one tacked on, if Eventlet could guess what was probably meant. I
added a bit of that logic into proxy-logging now as we saw some
errors resulting from this.
Fixes Bug #1012714
Change-Id: I671453eaf3704eab814ff12c4625ba7d749cc7ed
