435 Commits

Author SHA1 Message Date
Samuel Merritt
4f2ed8bcd0 EC: support multiple ranges for GET requests
This commit lets clients receive multipart/byteranges responses (see
RFC 7233, Appendix A) for erasure-coded objects. Clients can already
do this for replicated objects, so this brings EC closer to feature
parity (ha!).

GetOrHeadHandler got a base class extracted from it that treats an
HTTP response as a sequence of byte-range responses. This way, it can
continue to yield whole fragments, not just N-byte pieces of the raw
HTTP response, since an N-byte piece of a multipart/byteranges
response is pretty much useless.

There are a couple of bonus fixes in here, too. For starters, download
resuming now works on multipart/byteranges responses. Before, it only
worked on 200 responses or 206 responses for a single byte
range. Also, BufferedHTTPResponse grew a readline() method.

Also, the MIME response for replicated objects got tightened up a
little. Before, it had some leading and trailing CRLFs which, while
allowed by RFC 7233, provide no benefit. Now, both replicated and EC
multipart/byteranges avoid extraneous bytes. This let me re-use the
Content-Length calculation in swob instead of having to either hack
around it or add extraneous whitespace to match.

Change-Id: I16fc65e0ec4e356706d327bdb02a3741e36330a0
2015-06-03 11:42:00 -07:00
Samuel Merritt
666bf06c26 EC: don't 503 on marginally-successful PUT
On EC PUT in an M+K scheme, we require M+1 fragment archives to
durably land on disk. If we get that, then we go ahead and ask the
object servers to "commit" the object by writing out .durable
files. We only require 2 of those.

When we got exactly M+1 fragment archives on disk, and then one
connection timed out while writing .durable files, we should still be
okay (provided M is at least 3). However, we'd take our M > 2
remaining successful responses and pass that off to best_response()
with a quorum size of M+1, thus getting a 503 even though everything
worked well enough.

Now we pass 2 to best_response() to avoid that false negative.

There was also a spot where we were getting the quorum size wrong. If
we wrote out 3 fragment archives for a 2+1 policy, we were only
requiring 2 successful backend PUTs. That's wrong; the right number is
3, which is what the policy's .quorum() method says. There was a spot
where the right number wasn't getting plumbed through, but it is now.

Change-Id: Ic658a199e952558db329268f4d7b4009f47c6d03
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Closes-Bug: 1452468
2015-05-26 11:38:57 -07:00
Clay Gerrard
4aba2fbb25 Check if REST API version is valid
Swift doesn't check if the used API version is valid. Currently there
is only one valid REST API version, but that might change in the
future.

This patch enforces "v1" or "v1.0" as the version string when accessing
account, containers and objects.

The list of accepted version strings can be manually overridden using a
comma-separated list in swift.conf to make this backward-compatible.
The constraint loader has been modified slightly to accept strings as
well as integers.

Any request to an account, container, and object which does not
provide the correct version string will get a 400 BadRequest response.

The allowed api versions are by default excluded from /info.

Co-Authored-By: Christian Schwede <christian.schwede@enovance.com>
Co-Authored-By: John Dickinson <me@not.mn>

Closes Bug #1437442

Change-Id: I5ab6e236544378abf2eab562ab759513d09bc256
2015-04-14 16:00:37 -07:00
Mike Fedosin
b76d8c5156 Fix broken test_policy_IO_override test
Since the existing check depends on how quickly the test
passes, it causes to the unpredictable behavior and may break
the test.

This commit removes the unnecessary dependency on the
current time and leaves only the check that timestamp is 
greater than 0.

Change-Id: I3321e9aba0df52cb78523710b26f310f35801674
Closes-Bug: #1441599
2015-04-14 16:00:37 -07:00
John Dickinson
e910f7e07d Merge EC feature into master
Co-Authored-By: Alistair Coles <alistair.coles@hp.com>
Co-Authored-By: Thiago da Silva <thiago@redhat.com>
Co-Authored-By: John Dickinson <me@not.mn>
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Tushar Gohad <tushar.gohad@intel.com>
Co-Authored-By: Paul Luse <paul.e.luse@intel.com>
Co-Authored-By: Samuel Merritt <sam@swiftstack.com>
Co-Authored-By: Christian Schwede <christian.schwede@enovance.com>
Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com>
Change-Id: I002787f558781bd4d884129b127bc9f108ea9ec4
2015-04-14 08:57:15 -07:00
Samuel Merritt
decbcd24d4 Foundational support for PUT and GET of erasure-coded objects
This commit makes it possible to PUT an object into Swift and have it
stored using erasure coding instead of replication, and also to GET
the object back from Swift at a later time.

This works by splitting the incoming object into a number of segments,
erasure-coding each segment in turn to get fragments, then
concatenating the fragments into fragment archives. Segments are 1 MiB
in size, except the last, which is between 1 B and 1 MiB.

+====================================================================+
|                             object data                            |
+====================================================================+

                                   |
          +------------------------+----------------------+
          |                        |                      |
          v                        v                      v

+===================+    +===================+         +==============+
|     segment 1     |    |     segment 2     |   ...   |   segment N  |
+===================+    +===================+         +==============+

          |                       |
          |                       |
          v                       v

     /=========\             /=========\
     | pyeclib |             | pyeclib |         ...
     \=========/             \=========/

          |                       |
          |                       |
          +--> fragment A-1       +--> fragment A-2
          |                       |
          |                       |
          |                       |
          |                       |
          |                       |
          +--> fragment B-1       +--> fragment B-2
          |                       |
          |                       |
         ...                     ...

Then, object server A gets the concatenation of fragment A-1, A-2,
..., A-N, so its .data file looks like this (called a "fragment archive"):

+=====================================================================+
|     fragment A-1     |     fragment A-2     |  ...  |  fragment A-N |
+=====================================================================+

Since this means that the object server never sees the object data as
the client sent it, we have to do a few things to ensure data
integrity.

First, the proxy has to check the Etag if the client provided it; the
object server can't do it since the object server doesn't see the raw
data.

Second, if the client does not provide an Etag, the proxy computes it
and uses the MIME-PUT mechanism to provide it to the object servers
after the object body. Otherwise, the object would not have an Etag at
all.

Third, the proxy computes the MD5 of each fragment archive and sends
it to the object server using the MIME-PUT mechanism. With replicated
objects, the proxy checks that the Etags from all the object servers
match, and if they don't, returns a 500 to the client. This mitigates
the risk of data corruption in one of the proxy --> object connections,
and signals to the client when it happens. With EC objects, we can't
use that same mechanism, so we must send the checksum with each
fragment archive to get comparable protection.

On the GET path, the inverse happens: the proxy connects to a bunch of
object servers (M of them, for an M+K scheme), reads one fragment at a
time from each fragment archive, decodes those fragments into a
segment, and serves the segment to the client.

When an object server dies partway through a GET response, any
partially-fetched fragment is discarded, the resumption point is wound
back to the nearest fragment boundary, and the GET is retried with the
next object server.

GET requests for a single byterange work; GET requests for multiple
byteranges do not.

There are a number of things _not_ included in this commit. Some of
them are listed here:

 * multi-range GET

 * deferred cleanup of old .data files

 * durability (daemon to reconstruct missing archives)

Co-Authored-By: Alistair Coles <alistair.coles@hp.com>
Co-Authored-By: Thiago da Silva <thiago@redhat.com>
Co-Authored-By: John Dickinson <me@not.mn>
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Tushar Gohad <tushar.gohad@intel.com>
Co-Authored-By: Paul Luse <paul.e.luse@intel.com>
Co-Authored-By: Christian Schwede <christian.schwede@enovance.com>
Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com>
Change-Id: I9c13c03616489f8eab7dcd7c5f21237ed4cb6fd2
2015-04-14 00:52:17 -07:00
Alistair Coles
fa89064933 Per-policy DiskFile classes
Adds specific disk file classes for EC policy types.

The new ECDiskFile and ECDiskFileWriter classes are used by the
ECDiskFileManager.

ECDiskFileManager is registered with the DiskFileRouter for use with
EC_POLICY type policies.

Refactors diskfile tests into BaseDiskFileMixin and BaseDiskFileManagerMixin
classes which are then extended in subclasses for the legacy
replication-type DiskFile* and ECDiskFile* classes.

Refactor to prefer use of a policy instance reference over a policy_index
int to refer to a policy.

Add additional verification to DiskFileManager.get_dev_path to validate the
device root with common.constraints.check_dir, even when mount_check is
disabled for use in on a virtual swift-all-in-one.

Co-Authored-By: Thiago da Silva <thiago@redhat.com>
Co-Authored-By: John Dickinson <me@not.mn>
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Tushar Gohad <tushar.gohad@intel.com>
Co-Authored-By: Paul Luse <paul.e.luse@intel.com>
Co-Authored-By: Samuel Merritt <sam@swiftstack.com>
Co-Authored-By: Christian Schwede <christian.schwede@enovance.com>
Co-Authored-By: Yuan Zhou <yuan.zhou@intel.com>
Change-Id: I22f915160dc67a9e18f4738c1ddf068344e8ad5d
2015-04-14 00:52:16 -07:00
John Dickinson
dd9d97458e Prevent unauthorized delete in versioned container
An authenticated user can delete the most recent version of any
versioned object who's name is known if the user has listing access
to the x-versions-location container. Only Swift setups with
allow_version setting are affected.

This patch closes this bug, tracked as CVE-2015-1856

Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Christian Schwede <info@cschwede.de>
Co-Authored-By: Alistair Coles <alistair.coles@hp.com>

Closes-Bug: 1430645
Change-Id: Ibacc7413afe7cb6f77d92e5941dcfdf4768ffa18
2015-04-13 23:34:10 -07:00
Martin Kletzander
76b106fc01 Fix common misspellings
Wikipedia's list of common misspellings [1] has a machine-readable
version.  This patch fixes those misspellings mentioned in the list
which don't have multiple right variants (as e.g. "accension", which can
be both "accession" and "ascension"), such misspellings are left
untouched.  The list of changes was manually re-checked for false
positives.

[1] https://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings/For_machines

Change-Id: Ic9a5438629664f7cea216413a28acc0e8992da05
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2015-03-24 11:07:56 +01:00
Thiago da Silva
23d0842dec Refactoring the PUT method
Extracting large chunks of the PUT method into smaller
methods to improve maintainability and reuse of code.

Based on the work that Clay Gerrard started:
https://review.openstack.org/#/c/77812/

Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>

Change-Id: Id479fc5b159a2782361ac4a6e4a6d8bbaee4fe85
Signed-off-by: Thiago da Silva <thiago@redhat.com>
2015-03-21 12:30:58 -04:00
Arnaud JOST
7bc09dfdea Add support of x-remove- headers for container-sync
If the used tool to send header doesn't support empty headers (older versions
of curl), x-remove can be used to remove metadata.
sync-key and sync-to metadata, used by container-sync, can now be removed using
x-remove headers.

Change-Id: I0edb4d5425a99d20a973aa4fceaf9af6c2ddecc0
2015-02-19 15:43:31 +01:00
Alistair Coles
a82bfb25ba Make container GET call authorize when account not found
When an account was not found, ContainerController would
return 404 unconditionally for a container GET or HEAD request,
without checking that the request was authorized.

This patch modifies the GETorHEAD method to first call any
callback method registered under 'swift.authorize' in the
request environ and prefer any response from that over the 404.

Closes-Bug: 1415957

Change-Id: I4f41fd9e445238e14af74b6208885d83698cc08d
2015-02-09 18:08:15 +00:00
Jenkins
922a159e0b Merge "fix dlo manifest file getting versioned" 2014-12-18 22:03:30 +00:00
Thiago da Silva
24330771af fix dlo manifest file getting versioned
According to documentation dlo manifest files should not
be versioned. This patch fixes this issue and adds
some unit and functional for this scenario.

Change-Id: Ib5b29a19e1d577026deb50fc9d26064a8da81cd7
Signed-off-by: Thiago da Silva <thiago@redhat.com>
2014-12-18 15:14:06 -05:00
Jenkins
e8b1856de3 Merge "Adjust MAX_FILE_SIZE during test on 32 bit systems" 2014-12-16 14:37:37 +00:00
Jenkins
2cf24e914b Merge "Make error limits survive a ring reload" 2014-11-26 02:16:57 +00:00
Samuel Merritt
e429cd81be Make error limits survive a ring reload
The proxy was storing the error count and last-error time in the
ring's internal data, specifically in the device dictionaries. This
works okay, but it means that whenever a ring changes, all the error
stats reset.

Now the error stats live in the proxy server object, so they survive a
ring reload.

Better yet, the error stats are now keyed off of the node's
IP/port/device triple, so if you have the same device in two rings
(like with multiple storage policies), then the error stats are
combined. If the proxy server sees a 507 for an objec request in
policy X, then that will now result in that particular object disk
being error-limited for requests in policies Y and Z as well.

Change-Id: Icc72b68b99f37367bb16d43688e7e45327e3e022
2014-11-13 16:40:05 -08:00
Clay Gerrard
b98fe3b77b Prefer X-Backend-Timestamp for X-Newest
When a X-Backend-Timestamp is available it would generally preferred
over a less specific value and sorts correctly against any X-Timestamp
values anyway.

Change-Id: I08b7eb37ab8bd6eb3afbb7dee44ed07a8c69b57e
2014-11-12 12:18:45 -08:00
James Page
bd1ec7f89c Adjust MAX_FILE_SIZE during test on 32 bit systems
The default MAX_FILE_SIZE of (5 * 2 ** 30 + 2) exceeds the
capacity of an int on 32 bit systems; adjust this default
down to (2 ** 30 + 2) if the default exceeds the maxsize of
an int on the platform running the tests.

Closes-Bug: #1378810

Change-Id: Iafa2ce90ceb2de4e968ad48580270c8c572a9c9c
2014-10-09 16:45:20 +01:00
Jenkins
1a15de8e84 Merge "Fix delete versioning objects when previous is expired" 2014-09-16 14:35:08 +00:00
Jenkins
962209c001 Merge "Treat 404s as 204 on object delete in proxy" 2014-09-16 14:29:35 +00:00
Jenkins
034fae630c Merge "Restrict keystone cross-tenant ACLs to IDs" 2014-09-13 00:53:47 +00:00
Jenkins
8329095f11 Merge "account to account copy implementation" 2014-09-11 00:22:51 +00:00
Matthew Oliver
f4d3facdf4 Treat 404s as 204 on object delete in proxy
This change adds an optional overrides map to _make_request method
in the base Controller class.

  def make_requests(self, req, ring, part, method, path, headers,
                    query_string='', overrides=None)

Which will be passed on the the best_response method. If set and
no quorum it reached, the override map is used to attempt to find
quorum.

The overrides map is in the form:

    { <response>: <override response>, .. }

The ObjectController, in the DELETE method now passes an override map
to make_requests method in the base Controller class in the form of:

    { 404: 204 }

Statuses/responses that have been overridden are used in calculation
of the quorum but never returned to the user. They are replaced by:

    (STATUS, '', '', '')

And left out of the search for best response.

Change-Id: Ibf969eac3a09d67668d5275e808ed626152dd7eb
Closes-Bug: 1318375
2014-09-10 11:52:06 +10:00
Samuel Merritt
0221f1f847 Pay attention to all punctual nodes
The proxy sends requests to all storage nodes, but it only needs a
quorum of them to respond before the proxy can, in turn, respond to
the client. Thus, it gets quorum, and then briefly waits to see if the
remainder of the storage nodes respond before giving up on them.

However, the proxy was not paying any attention to the responses from
the non-quorum storage nodes. This would lead to some odd behavior,
like a container PUT where the backends returned (201, 201, 202) would
become a 201 to the client, but the permutation (201, 202, 201) would
become 202. Further, on object PUT, if the last node responded with an
error code, that error wouldn't count towards error-limiting.

The fix is quite simple: after getting quorum, the make_requests()
method was calling a method that returns responses from the remainder
of the nodes, but it was ignoring that return value and making up
responses with dummy values instead. Now, prior to making up dummy
responses, the proxy first uses the responses it already has, and only
fills in dummy responses for nodes that really didn't respond in time.

Change-Id: I0206b6b2272b0e7dcc80fb6c51840d8dae25cee2
2014-09-05 14:08:03 -07:00
Samuel Merritt
315af1737b Error limit the right node on object PUT
If any node had an error on object PUT, the proxy would count the
error against the last-connected-to node instead of the one with the
error. Now it counts the error against the right node.

Change-Id: I884eb73cebe0c723473a6d5e390a148fcad0d3ed
2014-09-05 11:39:36 -07:00
Jenkins
f9c149316d Merge "code shuffle post expired headers refactor" 2014-09-04 10:20:42 +00:00
Yuan Zhou
84a1e17f20 Fix delete versioning objects when previous is expired
When deleteing versioned objects proxy will try to restore the previous
copy. The COPY request will fail if the previous version is expired but
not handled by object-expirer.

This patch checks COPY respones on the previous copy, if it's
HTTP_NOT_FOUND(mostly because it's expired) proxy will try to restore
with the version before previous.

Closes-Bug #1308446
Change-Id: I17f049ea3ef62723effae8086ec427f6e151cd9c
2014-09-03 09:04:43 +08:00
Jenkins
75d9533466 Merge "Remove fake _get_part and use the real thing" 2014-08-22 01:50:33 +00:00
Clay Gerrard
21adf82cf1 code shuffle post expired headers refactor
Change-Id: I62248d7d3d7e0a3696a30e3d567ac6c2bea3c8eb
2014-08-21 10:45:22 -04:00
Jenkins
5f202b598a Merge "Shard expiring object container" 2014-08-20 15:33:29 +00:00
John Dickinson
75a329c7f5 add the account management config values to swift info
added account_autocreate and allow_account_management to
the /info endpoint

Change-Id: I4b239c9cefb728c3c93bf75cad065c72edf2fc0a
2014-08-18 10:38:43 -07:00
David Goetz
0abd2cba03 Shard expiring object container
All the expiring objects for a given X-Delete-At are funnelled into the
same expiring object container- this can act as a bottleneck.

Change-Id: I288a177a7ae3e213c727a2a81fa76d4ef9cf7eb3
2014-08-15 08:42:49 -07:00
Constantine Peresypkin
43ac76373a account to account copy implementation
Adds ability to copy objects between different accounts (on server side)

Adds new header to `PUT` request:
`X-Copy-From-Account: <account name>`
Account name corresponds to the last part of storage URL.

Adds new header to `COPY` request:
`Destination-Account: <account name>`
Account name corresponds to the last part of storage URL.

If your storage URL is: http://server:8080/v1/AUTH_test
Then the account name is `AUTH_test`

These headers should be used alongside `X-Copy-From` and `Destination` headers
The legacy headers should specify `<container name>/<object name>` path as usual.

DocImpact

Change-Id: I0285fe6a47df9e699ac20ae4a83b0bf23829e1e6
2014-08-13 15:43:22 +03:00
Thiago da Silva
048d46e609 fix to handle object-expiration headers in one method
There was some duplication of code in both POST and PUT
methods to handle object-expiration headers.
A method was created to remove this duplication,
which should help with maintainability of code.

Change-Id: I85cc4a7b0d688760c97598d80b9e9a39288c5f34
Signed-off-by: Thiago da Silva <thiago@redhat.com>
2014-08-11 13:30:24 -04:00
anc
a4f634bd89 Restrict keystone cross-tenant ACLs to IDs
The keystoneauth middleware supports cross-tenant access
control using the syntax <tenant>:<user> in container ACLs,
where <tenant> and <user> may currently be either a unique
id or a name. As a result of the keystone v3 API introducing
domains, names are no longer globally unique and are only
unique within a domain. The use of unqualified tenant and
user names in this ACL syntax is therefore not 'safe' in a
keystone v3 environment.

This patch modifies keystoneauth to restrict cross-tenant
ACL matching to use only ids for accounts that are not in
the default domain. For backwards compatibility,
names will still be matched in ACLs when both the requesting
user and tenant are known to be in the default domain AND the
account's tenant is also in the default domain (the default
domain being the domain to which existing tenants are
migrated).

Accounts existing prior to this patch are assumed to be for
tenants in the default domain. New accounts created using a
v2 token scoped on the tenant are also assumed to be in the
default domain. New accounts created using a v3 token scoped
on the tenant will learn their domain membership from the
token info. New accounts created using any unscoped token,
(i.e. with a reselleradmin role) will have unknown domain
membership and therefore be assumed to NOT be in the default
domain.

Despite this provision for backwards compatibility, names
must no longer be used when setting new ACLs in any account,
including new accounts in the default domain.

This change obviously impacts users accustomed to specifying
cross-tenant ACLs in terms of names, and further work will be
necessary to restore those use cases. Some ideas are
discussed under the bug report. With that caveat, this patch
removes the reported vulnerability when using
swift/keystoneauth with a keystone v3 API.

Note: to observe the new 'restricted' behaviour you will need
to setup keystone user(s) and tenant(s) in a non-default domain
and set auth_version = v3.0 in the auth_token middleware config
section of proxy-server.conf. You may also benefit from the
keystone v3 enabled swiftclient patch under review here:
https://review.openstack.org/#/c/91788/

DocImpact

blueprint keystone-v3-support

Closes-Bug:  #1299146

Change-Id: Ib32df093f7450f704127da77ff06b595f57615cb
2014-08-08 15:58:29 +01:00
Samuel Merritt
4bc600ff42 Fix "Handoff requested (N)" log spam
A long, long time ago, on a GET request, the proxy would go look on 3*
nodes for the requested thing. If one of the primary nodes was
error-limited, it'd look on two primaries and a handoff. Since this
indicated some failure somewhere, the proxy would emit a warning:
"Handoff requested (1)". If two primaries were down, there'd be a
second message "Handoff requested (2)", and so on.

Some StatsD messages were emitted too.

A somewhat shorter time ago (commit d79a67eb), the proxy started
looking into handoffs if it got 404s from the primaries. While this
was a good idea, it resulted lots of "Handoff requested (N)" log spam;
you'd see these messages on every single 404. Also, the StatsD
handoff_count and handoff_all_count metrics shot way up and turned
into noise.

This commit restores the original intent (and usefulness) of the log
messages and StatsD metrics: if the proxy only looks at the normal
number of handoff nodes, nothing is logged. However, if a primary is
down, then the message "Handoff requested (1)" will be logged,
indicating that the proxy looked at one more handoff than it normally
would, and this happened because a primary node was error-limited.

Closes-Bug: 1297214

* or whatever the replica count was

Change-Id: If1b77c18c880b096e8ab1df3008db40ce313835d
2014-07-25 19:06:56 -07:00
Gil Vernik
e5d76e9f05 Container PUT requests and max container per account
If container counter per account is equal or greater than
max_container_per_account, then all PUT requests are failed and
403 is returned.
This is correct behaviour if the request is to create a new
container, however if container already exists PUT should be
allowed, even the max_container_per_account condition has met.

This patch allows to process PUT requests for existing containers,
even if max_container_per_account > = container count.

It indirectly resolve the bug 1306711, since swift-client
uses internally PUT requests for container, prior it upload an
object there.

Change-Id: I2dcf20b6feb27e346111466a565695eba4b4b1da
2014-07-11 12:56:10 +03:00
Clay Gerrard
e114a628dc Remove fake _get_part and use the real thing
The get_part method is fast and stable given a consistent hash path
suffix/prefix, so there's no absolute requirement for the fake
implementation other than convenience.  OTOH, removing the fake
implementation and fixing the tests that were relying on it should make
it easier to write better tests going forward and harder to hide bugs
that don't show up when using the fakes.

There may be some overhead when writing new tests that use the ring if
you're making assertions on partitions or paths, but with a part power
of zero it's normally trivially obvious when a 1 needs to be a 0 or vice
versa.  Or you can just drop the assertions around the parts you were
faking anyway.

Change-Id: I8bfc388a04eff6491038991cdfd7686c9d961545
2014-07-08 15:57:43 +00:00
Paul Luse
9150fb383e Adjust timeout value in test_client_timeout
Per information filed in bug (and related bugs referenced in the
report) it appears that in TestObjectController.test_client_timeout
having a matching timeout value for both self.app.client_timeout
and SlowBody() appears to work however, as expected, a smaller
value in self.app.client_timeout also works where a larger
value fails.  With that short test combined with speculation and
related fixes, seems reasonable to merge what is suggested in
the bug report, drop the self.app.client_timeout to 0.05

Fixes Bug #1316716

Change-Id: Ib4c6d3bb275f6c50c62505c90656efa7ee566bc0
2014-07-04 06:08:53 -07:00
Jenkins
f43050d242 Merge "Fix exception raising in FakeConn" 2014-07-03 11:42:47 +00:00
Samuel Merritt
b823e1602e Fix exception raising in FakeConn
Timeout isn't an Exception, so Timeouts in tests weren't getting
raised. Instead, you'd sometimes have an HTTPResponse's .status be a
Timeout object, not an integer, which greatly confuses code that
expects an integer.

Also reorder the test that exposed the failure in the gate so it blows
up most times instead of sometimes do demonstrate the failure with out
this fix to FakeConn.

Change-Id: I76367a0575f84cad6b2f03e814f3f16bf96bc7d1
2014-07-02 15:09:01 -07:00
Clay Gerrard
620ff9b673 Fix order dependent test in proxy.test_server
TestObjectController.test_POST_backend_headers was being too picky about the
order of backend requests which when pushed through eventlet will not have a
stable order.  This change preserves the expectations and assertions while
removing the dependency on the order of the requests.

Change-Id: I7176ccb9223cd3dfc3c922b8b3d81eb514891d05
2014-07-02 12:20:05 -07:00
Clay Gerrard
0f0c0e5553 Fix KeyError on Storage Policy Container Sync
In the proxy, container_info can return a 'storage_policy' of None.  When
you set a header value on a swob.Request to None that effectively just
delete's the key.  One path through the proxy during container sync was
counting on the the 'X-Backend-Storage-Policy-Index' being set which isn't
the case if the cached container_info if for a pre-policies container.

Also clean up some test cruft, tighten up the interface on FakeConn, and add
some object controller tests to exercise more interesting failure and handoff
code paths.

Change-Id: Ic379fa62634c226cc8a5a4c049b154dad70696b3
2014-06-30 13:28:24 -07:00
Paul Luse
873c52e608 Replace POLICY and POLICY_INDEX with string literals
Replaced throughout code base &  tox'd. Functional as well
as probe tests pass with and without policies defined.

POLICY --> 'X-Storage-Policy'
POLICY_INDEX --> 'X-Backend-Storage-Policy-Index'

Change-Id: Iea3d06de80210e9e504e296d4572583d7ffabeac
2014-06-23 12:52:50 -07:00
Clay Gerrard
8bec50838c Extend interface on InternalClient
* add get_object
 * allow extra headers passthrough on HEAD/metadata reqeusts
 * expose (account|container|get_object)_ring properties

Pipeline propety access to the auto_create_account_prefix also allows us to
bypass the early exit on a container HEAD for auto_create_accounts if the
container-updater hasn't cycled yet.

Allow overriding of storage policy index.

This is something the reconciler will need so that it can GET from one
policy, PUT in another, and then DELETE from the first one again.

DocImpact
Implements: blueprint storage-policies
Change-Id: I9b287d15f2426022d669d1186c9e22dd8ca13fb9
2014-06-18 17:31:39 -07:00
Clay Gerrard
3824ff3df7 Add Storage Policy support to Object Server
Objects now have a storage policy index associated with them as well;
this is determined by their filesystem path. Like before, objects in
policy 0 are in /srv/node/$disk/objects; this provides compatibility
on upgrade. (Recall that policy 0 is given to all existing data when a
cluster is upgraded.) Objects in policy 1 are in
/srv/node/$disk/objects-1, objects in policy 2 are in
/srv/node/$disk/objects-2, and so on.

 * 'quarantined' dir already created 'objects' subdir so now there
   will also be objects-N created at the same level

This commit does not address replicators, auditors, or updaters except
where method signatures changed. They'll still work if your cluster
has only one storage policy, though.

DocImpact
Implements: blueprint storage-policies
Change-Id: I459f3ed97df516cb0c9294477c28729c30f48e09
2014-06-18 17:31:38 -07:00
Clay Gerrard
4321bb0af6 Add Storage Policy support to Containers
Containers now have a storage policy index associated with them,
stored in the container_stat table. This index is only settable at
container creation time (PUT request), and cannot be changed without
deleting and recreating the container. This is because a container's
policy index will apply to all its objects, so changing a container's
policy index would require moving large amounts of object data
around. If a user wants to change the policy for data in a container,
they must create a new container with the desired policy and move the
data over.

Keep status_changed_at up-to-date with status changes.

In particular during container recreation and replication.

When a container-server receives a PUT for a deleted database an extra UPDATE
is issued against the container_stat table to notate the x-timestamp of the
request.

During replication if merge_timestamps causes a container's status to change
(from DELETED to ACTIVE or vice-versa) the status_changed_at field is set to
the current time.

Accurate reporting of status_changed_at is useful for container replication
forensics and allows resolution of "set on create" attributes like the
upcoming storage_policy_index.

Expose Backend container info on deleted containers.

Include basic container info in backend headers on 404 responses from the
container server.  Default empty values are used as placeholders if the
database does not exist.

Specifically the X-Backend-Status-Changed-At, X-Backend-DELETE-Timestamp and
the X-Backend-Storage-Policy-Index value will be needed by the reconciler to
deal with reconciling out of order object writes in the face of recently
deleted containers.

 * Add "status_changed_at" key to the response from ContainerBroker.get_info.
 * Add "Status Timestamp" field to swift.cli.info.print_db_info_metadata.
 * Add "status_changed_at" key to the response from AccountBroker.get_info.

DocImpact
Implements: blueprint storage-policies
Change-Id: Ie6d388f067f5b096b0f96faef151120ba23c8748
2014-06-18 17:31:38 -07:00
Clay Gerrard
7624b198cf Update FakeRing and FakeLogger
FakeLogger gets better log level handling

Parameterize logger on some daemons which were previously
unparameterized and try and use the interface in tests.

FakeRing use more real code

The existing FakeRing mock's implementation bit me on some pretty subtle
character encoding issue by-passing the hash_path code that is normally
part of get_part_nodes.  This change tries to exercise more of the real
ring code paths when it makes sense and provide a better Fake for use in
testing.

Add write_fake_ring helper to test.unit for when you need a real ring.

DocImpact
Implements: blueprint storage-policies
Change-Id: Id2e3740b1dd569050f4e083617e7dd6a4249027e
2014-06-18 17:31:37 -07:00
Paul Luse
46c68aebd1 Add Storage Policy Support
The basic idea here is to replace the use of a single object ring in
the Application class with a collection of object rings. The
collection includes not only the Ring object itself but the policy
name associated with it, the filename for the .gz and any other
metadata associated with the policy that may be needed. When
containers are created, a policy (thus a specific obj ring) is
selected allowing apps to specify policy at container creation time
and leverage policies simply by using different containers for object
operations.

The policy collection is based off of info in the swift.conf file.
The format of the sections in the .conf file is as follows:

swift.conf format:

    [storage-policy:0]
    name = chicken

    [storage-policy:1]
    name = turkey
    default = yes

With the above format:

- Policy 0 will always be used for access to existing containers
  without the policy specified. The ring name for policy 0 is always
  'object', assuring backwards compatiblity. The parser will always
  create a policy 0 even if not specified

- The policy with 'default=yes' is the one used for new container
  creation. This allows the admin to specify which policy is used without
  forcing the application to add the metadata.

This commit simply introduces storage policies and the loading
thereof; nobody's using it yet. That will follow in subsequent
commits.

Expose storage policies in /info

DocImpact
Implements: blueprint storage-policies
Change-Id: Ica05f41ecf3adb3648cc9182f11f1c8c5c678985
2014-06-18 17:31:37 -07:00