Merge "Documentation of the logs formats used in Swift"

doc/source/index.rst

@@ -80,6 +80,7 @@ Administrator Documentation
     apache_deployment_guide
     admin_guide
     replication_network
+    logs
 
 Source Documentation
 ====================

doc/source/logs.rst

@@ -0,0 +1,94 @@
+====
+Logs
+====
+
+Swift has quite verbose logging, and the generated logs can be used for
+cluster monitoring, utilization calculations, audit records, and more. As an
+overview, Swift's logs are sent to syslog and organized by log level and
+syslog facility. All log lines related to the same request have the same
+transaction id. This page documents the log formats used in the system.
+
+----------
+Proxy Logs
+----------
+
+The proxy logs contain the record of all external API requests made to the
+proxy server. Swift's proxy servers log requests using a custom format
+designed to provide robust information and simple processing. The log format
+is::
+
+    client_ip remote_addr datetime request_method request_path protocol
+        status_int referer user_agent auth_token bytes_recvd bytes_sent
+        client_etag transaction_id headers request_time source log_info
+        request_start_time request_end_time
+
+=================== ==========================================================
+**Log Field**       **Value**
+------------------- ----------------------------------------------------------
+client_ip           Swift's guess at the end-client IP, taken from various
+                    headers in the request.
+remote_addr         The IP address of the other end of the TCP connection.
+datetime            Timestamp of the request, in
+                    day/month/year/hour/minute/second format.
+request_method      The HTTP verb in the request.
+request_path        The path portion of the request.
+protocol            The transport protocol used (currently one of http or
+                    https).
+status_int          The response code for the request.
+referer             The value of the HTTP Referer header.
+user_agent          The value of the HTTP User-Agent header.
+auth_token          The value of the auth token. This may be truncated or
+                    otherwise obscured.
+bytes_recvd         The number of bytes read from the client for this request.
+bytes_sent          The number of bytes sent to the client in the body of the
+                    response. This is how many bytes were yielded to the WSGI
+                    server.
+client_etag         The etag header value given by the client.
+transaction_id      The transaction id of the request.
+headers             The headers given in the request.
+request_time        The duration of the request.
+source              The "source" of the reuqest. This may be set for requests
+                    that are generated in order to fulfill client requests,
+                    e.g. bulk uploads.
+log_info            Various info that may be useful for diagnostics, e.g. the
+                    value of any x-delete-at header.
+request_start_time  High-resolution timestamp from the start of the request.
+request_end_time    High-resolution timestamp from the end of the request.
+=================== ==========================================================
+
+In one log line, all of the above fields are space-separated and url-encoded.
+If any value is empty, it will be logged as a "-". This allows for simple
+parsing by splitting each line on whitespace. New values may be placed at the
+end of the log line from time to time, but the order of the existing values
+will not change. Swift log processing utilities should look for the first N
+fields they require (e.g. in Python using something like
+``log_line.split()[:14]`` to get up through the transaction id).
+
+
+-----------------
+Storage Node Logs
+-----------------
+
+Swift's account, container, and object server processes each log requests
+that they receive. The format for these log lines is::
+
+    remote_addr - - [datetime] "request_method request_path" status_int
+        content_length "referer" "transaction_id" "user_agent" request_time
+
+=================== ==========================================================
+**Log Field**       **Value**
+------------------- ----------------------------------------------------------
+remote_addr         The IP address of the other end of the TCP connection.
+datetime            Timestamp of the request, in
+                    "day/month/year:hour:minute:second +0000" format.
+request_method      The HTTP verb in the request.
+request_path        The path portion of the request.
+status_int          The response code for the request.
+content_length      The value of the Content-Length header in the response.
+referer             The value of the HTTP Referer header.
+transaction_id      The transaction id of the request.
+user_agent          The value of the HTTP User-Agent header. Swift's proxy
+                    server sets its user-agent to
+                    ``"proxy-server <pid of the proxy>".``
+request_time        The duration of the request.
+=================== ==========================================================

swift/common/middleware/proxy_logging.py

@@ -24,6 +24,7 @@ The logging format implemented below is as follows:
 client_ip remote_addr datetime request_method request_path protocol
     status_int referer user_agent auth_token bytes_recvd bytes_sent
     client_etag transaction_id headers request_time source log_info
+    request_start_time request_end_time
 
 These values are space-separated, and each is url-encoded, so that they can
 be separated with a simple .split()