2784 Commits

Author SHA1 Message Date
Chmouel Boudjnah
18a0813d9b Add documentation about flake8+hacking.
- Fixes bug 1201431.

Change-Id: If025a41caf3a629b9efb4d67c53c423796d37a91
2013-07-15 17:14:16 +02:00
Dirk Mueller
8aba2d602e Start using Hacking
Instead of blacklisting Hacking globally,
only blacklist those that currently occur frequently
(for a later followup patch), and fix the rest. In
detail:

H101  Use TODO(NAME)
H201  no 'except:' at least use 'except Exception:'
H231  octal number 022 should be written as 0o22
H401  docstring should not start with a space
H701  Empty localization string

Change-Id: Ib3b3d56b68d1cf15d3b67ac9749fcbdb876dc52a
2013-07-15 11:41:58 +02:00
Jenkins
cbc29e43be Merge "Extract quorum-size calculation to utility method." 2013-07-14 21:32:14 +00:00
Jenkins
eb43021e62 Merge "Make keystoneauth reseller_prefix check for _" 2013-07-12 22:26:08 +00:00
Greg Lange
e7b5e9d44f make internal client metadata methods return dicts with lower case keys
Change-Id: I19ced30b6e92133766310f8e41286202607c5b4c
2013-07-09 20:27:52 +00:00
Jenkins
72faf7b86d Merge "Revert "docfix apache2 now supports client chunked encodin"" 2013-07-09 19:40:33 +00:00
Chmouel Boudjnah
6f57b06c7c Make keystoneauth reseller_prefix check for _
- Automatically add underscore if there wasn't one already.
- This makes consistent with tempauth.
- Fixes bug 1198878.
- This should probably need to be mentioned in the release changelog for
  upgrades.

Change-Id: Ia82f14f82c380c7ed2ddd190116f1493e410cfbe
2013-07-08 13:32:28 +02:00
Samuel Merritt
151313ba8c Fix flaky test.
test_DELETE_x_container_headers_with_more_container_replicas() would
sometimes fail, and it's because a test helper method was defaulting
to sorting the captured headers by X-Container-Partition. Well, in
tests we use FakeRing, and the partition is always 1, so that sorting
didn't buy us much. Now the sorting is done by X-Container-Device,
which actually differs meaningfully.

Change-Id: Ibe5b2fcd3f23280ed2caaa703111a98861331866
2013-07-03 17:45:20 -07:00
Jenkins
7bb1d07cd0 Merge "Record some simple object stats in the object auditor" 2013-07-03 04:59:47 +00:00
David Hadas
fcfb8012cd Revert "docfix apache2 now supports client chunked encodin"
This reverts commit 68cb91097b75a92237bd90caffcd405c3e83cb53

Just so this is not get forgotten in the tree...

We are using daemon mode and chunked is not supported in this mode.
2013-07-02 22:18:32 +00:00
David Goetz
043bfb77f4 Record some simple object stats in the object auditor
Change-Id: I043a80c38091f59ce6707730363a4b43b29ae6ec
2013-07-02 13:41:18 -07:00
Jenkins
bc35717a61 Merge "small cleanup to associated projects page" 2013-07-02 00:13:24 +00:00
Samuel Merritt
885e02102d Extract quorum-size calculation to utility method.
The behavior's the same as before; this just makes the code (IMHO) a
little easier to read.

Change-Id: Ie54d836d81af7410057219f60b72b840c9ce51b9
2013-07-01 16:09:36 -07:00
John Dickinson
9b2ee07ee3 small cleanup to associated projects page
Change-Id: I5d6d6d6c32b6573474288897f6fa174b6f150183
2013-07-01 13:40:37 -07:00
David Goetz
a24c89e5bd adding copyright notice
Change-Id: Ic2c79d52770208c22b39cb2b00f2f8a6a86e8aa7
2013-07-01 12:04:26 -07:00
Jenkins
4a90414fc7 Merge "docfix apache2 now supports client chunked encodin" 2013-06-29 01:21:10 +00:00
Jenkins
1f7c86ec91 Merge "Update docs to use default XFS inode size" 2013-06-29 00:51:24 +00:00
Chuck Thier
581f7f5517 Update docs to use default XFS inode size
In past couple of years, the XFS team has greatly improved inode use in
xfs.  With more recent kernels, there is no performance penalty for
using the default inode size, and a smaller inode size gives us
improvements in other areas where disk access is involved.

DocImpact

Change-Id: Ie9da53a6e8bf43d1d02881befbb52595462c9f2e
2013-06-28 19:52:17 +00:00
Jenkins
e759dad84f Merge "Add test for swift.common.utils.lock_file" 2013-06-28 17:53:23 +00:00
Tom Fifield
68cb91097b docfix apache2 now supports client chunked encodin
As reported in the documentation bug, the apache deployment guide's
reference to apache2 mod_wsgi not supporting client chunked encoding
has become outdated. It now supports this feature, using an optional
parameter.

Updated the paragraph in question to reflect this

patchset 2 mentions the WSGIChunkedRequest variable and adds it
to the sample configs - On by default. Feedback welcome

fixes bug 1194935

Change-Id: I07c5c8506ac34e1e0e08fa6d961babde2f9b7367
2013-06-28 15:27:33 +10:00
Felipe Reyes
0e602979c9 Add test for swift.common.utils.lock_file
Change-Id: Id03f130ad7016b11b740050b7aadf879773d5a5f
2013-06-28 00:28:26 -04:00
Jenkins
c6e53721e0 Merge "Make formpost work with multiple tempurl keys." 2013-06-27 20:45:53 +00:00
Jenkins
2a91768911 Merge "TempAuth: don't return 401/403 if .r without .rlistings" 2013-06-26 22:25:46 +00:00
Jenkins
53908a1147 Merge "Allow SLOs to be made up of other SLOs" 2013-06-26 21:51:07 +00:00
Samuel Merritt
bd00f0c8c9 Make formpost work with multiple tempurl keys.
Formpost wouldn't look at X-Account-Meta-Temp-URL-Key-2 for signature
validation, which was confusing to users. Now it does, and there's a
utility function that formpost and tempurl both use to help prevent
divergence in the future.

Also removed formpost's custom cache-lookup logic. Formpost was
looking for tempurl keys in memcache, but it's been broken since
commit 21343ab (2013-05-02) when tempurl changed its memcache keys, so
the lookup would always miss. (Since tempurl started storing *lists*
of keys, formpost would have choked on the new data, so that oversight
was the only thing keeping formpost working at all.)

Now formpost just uses the same get_account_info() call as everyone
else does, so it gets all the benefits of the new caching framework.

Change-Id: I426d8f2882ebe3ce373de604b9ff18846f5999e3
2013-06-26 11:44:48 -07:00
David Goetz
9f942b1256 Allow SLOs to be made up of other SLOs
We've gone back and forth about this. In the initial commit, it couldn't
possibly work because you wouldn't be able to get the Etags to match. Then it
was expressly disallowed with a custom error message, and now its allowed. The
reason we're allowing it is that 1,000 segments isn't enough for some use cases
and we decided its better than just upping the number of allowed segments. The
code to make it work isn't all that complicated and it allows for virtually
unlimited SLO object size. There is also a new configurable limit on the
maximum connection time for both SLOs and DLOs defaulting to 1 day. This will
hopefully alleviate worries about infinite requests. Think I'll leave the
python-swift client support for nested SLOs to somebody else though :).

DocImpact

Change-Id: Id16187481b37e716d2bd09bdbab8cc87537e3ddd
2013-06-26 09:44:33 -07:00
Chuck Thier
b012fd998c Change ring partition size for SAIO
Making this smaller (10 instead of 18) can make some of the tests run
faster and makes rebuilding of the rings faster.

Change-Id: Ibe46011d8e6a6482d39b3a20ac9c091d9fbc6ef7
2013-06-26 15:52:18 +00:00
Jenkins
1620f4e99e Merge "Adding unit tests for direct client" 2013-06-26 15:42:40 +00:00
Kun Huang
8ff374de97 TempAuth: don't return 401/403 if .r without .rlistings
When checking referer (.r, .rlistings), tempauth return 401/403 if '.r'
exists without '.rlistings' or obj. Actually it should skip this kind of
authorizing and check the following authorizing.

And remove some unnecessary lines

Change-Id: Ib60e59b7c80331a699cb5c3496f948cf32135d0b
2013-06-26 23:35:16 +08:00
Jenkins
66a0817e99 Merge "Remove dead code and tests for staticweb." 2013-06-26 14:32:01 +00:00
Jenkins
882078dcf1 Merge "version bump to 1.9.1" 2013-06-26 06:46:12 +00:00
Jenkins
fefdf5f90b Merge "updated changelog and authors update for 1.9.0 release" 2013-06-26 06:43:13 +00:00
Samuel Merritt
8b4520a483 Remove dead code and tests for staticweb.
Dead since e499b91.

Change-Id: I3c571ae471eb4a0b95772b19eac90183cfc7fe22
2013-06-25 23:18:57 -07:00
John Dickinson
68fc163fdd version bump to 1.9.1
Change-Id: If04c88f7e686f4b4709c5b68351ebbbea97d12bd
2013-06-25 23:00:31 -07:00
David Hadas
8226761889 Deleted account respond as non existing accounts
Currently clients can not distinguish between non existing accounts
(which can be created) and accounts marked for deletion, which has
not yet been reaped and therefore cannot be re-created until reaped.

Following this patch, if an account is marked as deleted but hasn't
been reaped and is still on disk, responses will include a status
header:
    'X-Account-Status' = 'Deleted'

Fixes:Bug #1188609
Change-Id: Ibd39965ae3f5d45fd78f130e0e31f5a0141a8633
2013-06-26 08:33:23 +03:00
John Dickinson
abda080366 updated changelog and authors update for 1.9.0 release
Change-Id: I61ffbaba01aba86d61e4f3d988b5abeee8f07813
2013-06-25 21:41:28 -07:00
Jenkins
edf4068c8b Merge "Local write affinity for object PUT requests." 2013-06-26 04:39:53 +00:00
Jenkins
83bdd0b2e1 Merge "Refactor DiskFile instantiation in object server." 2013-06-25 20:29:25 +00:00
Jenkins
a42e87a464 Merge "Add sample rsyslog.conf." 2013-06-25 19:58:01 +00:00
Jenkins
9ac0b14a05 Merge "Fix swift-dispersion-report." 2013-06-25 19:43:59 +00:00
Kun Huang
9fc55ee2b4 Add sample rsyslog.conf.
Give users a sample rsyslog.conf to manage their logs easily based on
LOCAL0 swift used. In this patch I offer some choices to output logs
via commnet lines or uncomment.

Change-Id: I2fe150a6e3d164a989c3520c0b7f032897a71f18
2013-06-25 10:24:26 +08:00
Brian D. Burns
6768d5b4be Use object name from request in bulk Errors
This will allow users to more easily match the failed objects in Errors
for bulk delete requests to the object names they provided in the request.
For extract Errors, it reports the failed path from the tar archive.

DocImpact

Change-Id: I084057810fc4fb7fdac05494cc6fec2cbf81bb9d
2013-06-24 16:44:32 -04:00
Samuel Merritt
57d4e6e718 Fix swift-dispersion-report.
Fixes bug 1180680.

Change-Id: I1c0eaa6f934f73853e832a8e468e1b8369a21117
2013-06-24 10:58:22 -07:00
Samuel Merritt
d9f2a76973 Local write affinity for object PUT requests.
The proxy can now be configured to prefer local object servers for PUT
requests, where "local" is governed by the "write_affinity". The
"write_affinity_node_count" setting controls how many local object
servers to try before giving up and going on to remote ones.

I chose to simply re-order the object servers instead of filtering out
nonlocal ones so that, if all of the local ones are down, clients can
still get successful responses (just slower).

The goal is to trade availability for throughput. By writing to local
object servers across fast LAN links, clients get better throughput
than if the object servers were far away over slow WAN links. The
downside, of course, is that data availability (not durability) may
suffer when drives fail.

The default configuration has no write affinity in it, so the default
behavior is unchanged.

Added some words about these settings to the admin guide.

DocImpact

Change-Id: I09a0bd00524544ff627a3bccdcdc48f40720a86e
2013-06-23 22:04:56 -07:00
Jenkins
13347af64c Merge "Improve swift's keystoneauth ACL support" 2013-06-24 04:57:13 +00:00
Jenkins
054ffbe4a6 Merge "Stop getting useless bytes on manifest Range requests." 2013-06-23 17:48:49 +00:00
yuan-zhou
60388c9978 Adding unit tests for direct client
- Added simple unit tests for direct client
 - Implemented chunked put in direct_put_object

Change-Id: Icec1019b5157fae33e2fd36a7fc3b9c0cc15afec
2013-06-22 16:15:29 +08:00
Samuel Merritt
e8e6bffc95 Clean up some remnants of StaticWeb's and TempURL's custom caching.
As of e499b91, these config values and functions are unused in StaticWeb.

As of 2e155e5, the comment in TempURL is false.

Change-Id: I75b631ece8a9a6075c406765361629c549c449f6
2013-06-21 17:37:06 -07:00
Jenkins
75660a1e9e Merge "bulk response Content-Type does not match request Accept header" 2013-06-21 17:21:03 +00:00
Jenkins
461b88b7bd Merge "Added utils.FileLikeIter" 2013-06-21 15:34:41 +00:00