swift/test/unit
Tim Burke 423f96293b Verify client input for v4 signatures
This is a combination of 2 commits.

==========

Previously, we would use the X-Amz-Content-SHA256 value when calculating
signatures, but wouldn't actually check the content that was sent. This
would allow a malicious third party that managed to capture the headers
for an object upload to overwrite that with arbitrary content provided
they could do so within the 5-minute clock-skew window.

Now, we wrap the wsgi.input that's sent on to the proxy-server app to
hash content as it's read and raise an error if there's a mismatch. Note
that clients using presigned-urls to upload have no defense against a
similar replay attack.

Notwithstanding the above security consideration, this *also* provides
better assurances that the client's payload was received correctly. Note
that this *does not* attempt to send an etag in footers, however, so the
proxy-to-object-server connection is not guarded against bit-flips.

In the future, Swift will hopefully grow a way to perform SHA256
verification on the object-server. This would offer two main benefits:

  - End-to-end message integrity checking.
  - Move CPU load of calculating the hash from the proxy (which is
    somewhat CPU-bound) to the object-server (which tends to have CPU to
    spare).

Closes-Bug: 1765834
(cherry picked from commit 3a8f5dbf9c)

----------

s3api: Allow clients to upload with UNSIGNED-PAYLOAD

(Some versions of?) awscli/boto3 will do v4 signatures but send a
Content-MD5 for end-to-end validation. Since a X-Amz-Content-SHA256
is still required to calculate signatures, it uses UNSIGNED-PAYLOAD
similar to how signatures work for pre-signed URLs.

Look for UNSIGNED-PAYLOAD and skip SHA256 validation if set.

(cherry picked from commit 82e446a8a0)
(cherry picked from commit 6ed165cf3f)

==========

Change-Id: I61eb12455c37376be4d739eee55a5f439216f0e9
2019-10-22 14:54:31 +00:00
..
account Add fallocate_reserve to account and container servers. 2018-07-18 17:27:11 +10:00
cli swift-ring-builder shows hint about composite builder file 2018-07-05 15:57:05 +01:00
common Verify client input for v4 signatures 2019-10-22 14:54:31 +00:00
container fixed _check_node() in the container sharder 2018-12-11 13:44:40 -08:00
obj Address some review comments 2018-06-27 17:17:19 -07:00
proxy PUT+POST: break out putter specific test classes 2018-07-02 23:02:13 -05:00
test_locale monkeypatch thread for keystoneclient 2015-11-03 16:36:19 +01:00
__init__.py fixed _check_node() in the container sharder 2018-12-11 13:44:40 -08:00
helpers.py py3: port common/ring/ and common/utils.py 2018-02-12 06:42:24 +00:00