Commit 7fcbbebb changed the internal client a little bit and removed the
class variable self.attempts in SimpleClient. This is currently still
needed in swift-dispersion-populate to track the number of retries, thus
re-adding it again.
Closes-Bug: 1351323
Change-Id: I98ba6441607158b34708b2dbb2d1a63788681d63
Previously, this code was attempting to set up http proxying but it
wasn't working. We noticed after a while when we saw traffic going
through an alternate route instead of our set of http proxies with
container sync.
Additional work and testing by clayg; thanks!
Change-Id: I840b8e55a80c13ae85c65bf68de261d735685b27
The SimpleClient.base_request method need only to call urllib2.urlopen
once to get a response from the server. Remove the duplicate call.
* fixup tests that were poorly mocking urllib2
* mock eventlet sleep because we're all busy
Change-Id: I6da93c689200351e9dbd9416b9c4bf9544571f15
Discovered some tests that were coupling the code under test with the
storage policies configured in /etc/swift/swift.conf. There was some
tests that created fake rings in their tempdirs, but didn't reset or
patch the POLICIES global. So if your local config needed more rings
that the fake's were setting up (just 2) the tests would puke when they
loaded up an app that looked for rings. I think this probably started
happening when we added eager object ring loading back into the proxy.
* two TestCases in test_wsgi were missing @patch_policies
* fixed issue with patch_policies that could cause state to bleed
between tests
* patch_policies' legacy and default collections get a FakeRing by
default
* drive-by cleanup for test_loadapp_proxy() ring serialized path
handling
* drive-by cleanup for test_internal_client that was doing basically
the same thing as test_wsgi
Change-Id: Ia706000ba961ed24f2c22b81041e53a0c3f302fc
* add get_object
* allow extra headers passthrough on HEAD/metadata reqeusts
* expose (account|container|get_object)_ring properties
Pipeline propety access to the auto_create_account_prefix also allows us to
bypass the early exit on a container HEAD for auto_create_accounts if the
container-updater hasn't cycled yet.
Allow overriding of storage policy index.
This is something the reconciler will need so that it can GET from one
policy, PUT in another, and then DELETE from the first one again.
DocImpact
Implements: blueprint storage-policies
Change-Id: I9b287d15f2426022d669d1186c9e22dd8ca13fb9
Also fixed a bug where SimpleClient would send ?format=json for object
requests, which is not necessary.
Change-Id: If06a7dcebc9de2d7c8b28a046d60b902dae821c1
As seen on #1174809, changes use of mutable types as default
arguments and defaults them within the method. Otherwise, those
defaults can be unexpectedly persisted with the function between
invocations and erupt into mass hysteria on the streets.
There was indeed a test (TestSimpleClient.test_get_with_retries)
that was erroneously relying on this behavior. Since previous tests
had populated their own instantiations with a token, this test only
passed because the modified headers dict from previous tests was
being overridden. As expected, with the mutable defaults fix in
SimpleClient, this test begain to fail since it never specified any
token, yet it has always passed anyway. This change also now provides
the expected token.
Change-Id: If95f11d259008517dab511e88acfe9731e5a99b5
Related-Bug: #1174809
internal client.
Internal client uses 'chunked' transfer encoding in it's
upload_object method. If content-length is known in advance,
internal_client should allowed to use content length in the
upload_object method and not the chunked method.
Change-Id: I68d2ebde78e01fa16b7187a2f045ea20f4310722
One can argue that it makes sense for the client-facing proxy server
to have certain middlewares like gatekeeper in its pipeline, but that
is not desirable for InternalClient. In particular, it prevents you
from passing in sysmeta headers using InternalClient, and I found
myself wanting to do that earlier today.
Now InternalClient's proxy application gets exactly what's configured;
no more, no less. This will mean that the object expirer can read and
write sysmeta headers, but I think we can trust it to keep our
secrets.
Change-Id: I17b4a89c24e600754701ee1645b40406421fa6f3
Removes the requirement for swiftclient in swift-dispersion-report
and swift-dispersion-populate. To prevent a dependency on
keystoneclient and to avoid reinventing the wheel with an internal
keystoneclient, authentication with keystone is only supported if
swiftclient is available. If not, only auth v1 is supported.
The dependency in swift/container/sync.py has also been removed.
Implements: blueprint remove-swiftclient-dependency
Change-Id: I6ec3b3c85a67b9ab6eb04b90ffc16daf1600e8a7
Hopefuly this extra test case is not too inane and slows us
down for nothing. It is verified to fail with the old code.
Change-Id: I604eca09f7f9ae044a8ad75284cd82a37325f99c
except x,y: was deprected and is removed in Python 3.x.
Use "except x as y:" instead which works in any Python
version >= 2.6.
Change-Id: I7008c74b807340f3457d3a0c8bd0b83f23169d14
These bug fixes are lumped together because they all caused problems
with the object expirer doing its job.
There was a bug with the internal client doing listings that happened
to run across a Unicode object name for use as a marker.
There was a bug with the object expirer not utf8 encoding object
names it got from json listings, causing deletes to fail.
There was a bug with the object expirer url quoting object names when
calling the internal client's make_request, when make_request already
handles that.
Change-Id: I29fdd351fd60c8e63874b44d604c5fdff35169d4
_get_metadata() should only look for metadata on 200 family http status codes
in the path_parts() in the internal client test file should return distinct unicode strings
with spaces for account, container, and obj
Change-Id: I74e335d3b73a59092a0eec06539f4931420061dd
made internal client handle unicode path parts by adding make_path method
fixed pep8 problems in internal client and its test
moved internal client unit test file to correct directory
Change-Id: Id1c81c9cb0db05342e4e8a8393db93552fda4647