HTTP header values should be quoted. Since the WWW-Authenticate
header value contains user-supplied strings, it's important to
ensure it's properly quoted to ensure the integrity of the protocol.
Previous to this patch, the URL was unquoted and then the unquoted
value was returned in the header. This patch re-quotes the value
when it is set on the response.
This is filed as CVS-2014-3497
Fixes bug 1327414
Change-Id: If8bd8842f2ce821756e9b4461a18a8ac8d42fb8c
Just put SLO and DLO after any auth middleware. This works because when
the request goes through that middleware in the pipeline the
authentication takes place: validation of the token, setting up who the
user is, and setting the authorization call back. Each subrequest made
for the segments will be subjected to that authorization call back which
verifies the user has access to the individual segments.
To get this to work with keystone, the keystone identity is set up
during __call__ and applied to the authorize function using a
functools.partial. When the authorize function is later called from the
environ by the proxy server the idenity that was set up when the request
passed through the auth middleware is used, not what can be pulled out
of the possibly altered state of the request's environment.
DocImpact
fixes bug: 1315133
Change-Id: I7827dd2d9dfbb3c6424773fb2891355d47e372ba
Formpost middleware fails to upload files larger then 2 GiB due to
an Overflow error. The reason is that the readline() will use a
readline(int(content_length)) later on and fail if it is larger than
2GiB. Since it is not required to read the whole content into memory
to detect the boundary only read the amount of required bytes.
The underlying error is located in Python 2.7 and is related to
cStringIO: http://bugs.python.org/issue7358
Closes-Bug: #1326429
Change-Id: I196edda647921c2691d278cebd1cca80ebd360f2
There is a simple typo in the man page of proxy-server.conf,
"client_timeout" is written as "client_timeoutt".
This commit fixes it.
Closes-Bug: #1326237
Change-Id: I98777f523906e4ed625de8f20a96979ea627aa1f
Container server representation of data (received from backend) is isolated
into `create_listing()` call. GET() method will call `create_listing()`
when it's ready to create a serialized object listing. It will allow us to
customize that representation: get different listing, if backend supports
more interesting queries, for example.
Change-Id: I4fa03357d3d9e697bc3e1fb5da42d7572c6ae9b3
The tempurl functional tests would fail if you put tempurl.methods in
the proxy server's disallowed_sections. The test assumed that the
presence of the key 'tempurl' implied the presence of the subkey
'methods', but since 9cbf8a3 landed, operators can remove arbitrary
things from /info.
Normally I'd just change x['methods'] to x.get('methods', []), but it
turns out this was setting up a class variable that nobody ever looked
out again, so removing the code also works.
Change-Id: Ie899d146bc6fff81a5fae77815897244e8ec6bec
Also fixed a bug where SimpleClient would send ?format=json for object
requests, which is not necessary.
Change-Id: If06a7dcebc9de2d7c8b28a046d60b902dae821c1
We previously registered with no parameters, added parms so they
are displayed as follows (example):
"ratelimit": {"max_sleep_time_seconds": 60.0,
"container_listing_ratelimits": [[0, 100.0], [10, 50.0], [50, 20.0]],
"container_ratelimits": [[0, 100.0], [10, 50.0], [50, 20.0]],
"account_ratelimit": 1.0}
Note that not all parameters are exposed (intentionally) via /info
Change-Id: I36c7ef15af17e3eb8ebb93429035bd06d089a945
Closes-Bug: 1308989
Log lines can get quite large, as we previously noticed with rsync error
log lines. We added a setting to cap those, but it really looks like we
should have just done this overall limit. We noticed the issue when we
switched to UDP syslogging and it would occasionally blow past the 16436
lo MTU! This causes Python's logging code to get an error and hilarity
ensues.
Change-Id: I44bdbe68babd58da58c14360379e8fef8a6b75f7
Container sync had a bug where it'd send out the trailing
"; swift_bytes=xxx" part of the content-type header. That trailing part
is just for internal cluster usage by SLO. Since that needed to be
stripped in two places now, I separated it out to a function that both
spots call.
Change-Id: Ibd6035d7a6b78205344bcc9d98bc1b7a9d463427
Based on comments from deployers at the Juno OpenStack summit,
limiting the default logged token length (to, by default, prevent
tokens from being fully logged) is a good idea.
Change-Id: I58980e85329d99de41f1c08f75e85973452317b1
On Lucid at least, xprofile doesn't work. While others try to figure out the
underlying issue, this patch would just skip the tests if the profiling code
can not be imported.
Change-Id: I0fee3f729742cc727edc2adabb4ed782bea7798f
There are files containing string format arguments inside
logging messages. Using logging function parameters should
be preferred.
Change-Id: I1c2a4f43463932e85c75d576241d0c0455c38492
This allows an easier and more explicit way to tell swift-init to run on
specific servers. For example with an SAIO, this allows you to do
something like:
swift-init object-server.1 reload
to reload just the 1st object server. A more real world example is when
you are running separate servers for replication. In this example you
might have an object-server/public.conf and
object-server/replication.conf. With this change you can do something
like:
swift-init object-server.replication reload
to just reload the replication server.
DocImpact
Change-Id: I5c6046b5ee28e17dadfc5fc53d1d872d9bb8fe48
The value of the X-Trans-Id-Extra header on the request (if any) will
now be appended to the transaction ID. This lets users put their own
information into transaction IDs.
For example, Glance folks upload images as large objects, so they'd
like to be able to tie together all the segment PUTs and the manifest
PUT with some operation ID in the logs. This would let them pass in
that operation ID as X-Trans-Id-Extra, and then when things went
wrong, it'd be much easier to find all the requests in Swift's logs.
Also, this aids debuggability when requests fail to receive
responses. If a user is sending in their own X-Trans-Id-Extra strings,
then that gives operators something to search for in the logs. The
normal txid won't work since that's in the response, but the client
didn't receive one.
Swift will only use the first 32 characters of X-Trans-Id-Extra so
that its log lines stay a manageable length. Also, it's URL-quoted so
that users cannot inject double quotes into X-Trans-Id-Extra and screw
up log parsers.
DocImpact
Change-Id: I3c51d0c5ac55697ac230001840da219e73a03157
According to the OpenStack translation policy available at
https://wiki.openstack.org/wiki/LoggingStandards debug messages
should not be translated. Like mentioned in several changes in
Nova by garyk this is to help prioritize log translation.
Change-Id: I59486b1110f08510d83a4aec2a1666805c59d1cd
Closes-Bug: #1318333
As seen on #1174809, changes use of mutable types as default
arguments and defaults them within the method. Otherwise, those
defaults can be unexpectedly persisted with the function between
invocations and erupt into mass hysteria on the streets.
There was indeed a test (TestSimpleClient.test_get_with_retries)
that was erroneously relying on this behavior. Since previous tests
had populated their own instantiations with a token, this test only
passed because the modified headers dict from previous tests was
being overridden. As expected, with the mutable defaults fix in
SimpleClient, this test begain to fail since it never specified any
token, yet it has always passed anyway. This change also now provides
the expected token.
Change-Id: If95f11d259008517dab511e88acfe9731e5a99b5
Related-Bug: #1174809
When current code modifies the pipeline, it prints the entry point
names instead of the names used to construct the pipeline. This is
inconvenient because a sysadmin cannot copy and paste from the log.
We already save the pipeline name into contexts in most cases, so
the fix simply reuses that to provide friendly names.
Fixes bug: 1311802
Change-Id: Ic76baf1360cd521f140fa1980029ccbce58f1717
The profile middleware provide a tool to profile Swift
code on the fly and collect statistic data for performance
analysis. An native simple Web UI is also provided to help
query and visualize the data.
Change-Id: I6a1554b2f8dc22e9c8cd20cff6743513eb9acc05
Implements: blueprint profiling-middleware
John Dickinson
Jenkins
David Goetz
Christian Schwede
replay
Christian Berendt
Pete Zaitcev
Samuel Merritt
gholt
Paul Luse
Alex Gaynor
Mark Seger
Chuck Thier
Brian Cline
Morgan Fainberg
zhang-hare