On an object PUT with a non-ascii name, if we hit some kind of
exception speaking to only one object-server of the N we try to
connect to, we try to log it -- but this causes an exception when
interpolating the UTF-8 encoded path iff the message template is
unicode.
Since this is essentially an exception within an exception handler,
this fails the entire request with a 500 error -- even though the
other nodes may have been just fine. This occurs before it attempts
a handoff node.
The simplest way to reproduce this is by running func tests against
a small cluster where one of the object nodes is not running
N.B. The locale of the node does not matter because the message
template is interpolated with node/device data from the Ring which is
always unicode because of json.
This includes an update to the FakeRing used by unittest
infrastructure to ensure that the FakeRing devices make a round-trip
through json to ensure consistent typing with real Rings.
Change-Id: Icb7284eb5abc9869c1620ee6366817112d8e5587
Closes-bug: #1597210
Also, make request method calling in obj.server consistent with change
3944d8 to account & container
Change-Id: I893d77a06793a5eeafac203a45971e96425afb96
Related-Change: I2f7586f96b41a97e6ae254efc83218b3b5c6cc9e
Include a note in container-sync docs pointing to specific
configuration needed to be compatible with encryption.
Also remove the sample encryption root secret from
proxy-server.conf-sample and in-process test setup. Remove encryption
middleware from the default proxy pipeline.
Change-Id: Ibceac485813f3ac819a53e644995749735592a55
Adds encryption middlewares.
All object servers and proxy servers should be upgraded before
introducing encryption middleware.
Encryption middleware should be first introduced with the
encryption middleware disable_encryption option set to True.
Once all proxies have encryption middleware installed this
option may be set to False (the default).
Increases constraints.py:MAX_HEADER_COUNT by 4 to allow for
headers generated by encryption-related middleware.
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Co-Authored-By: Christian Cachin <cca@zurich.ibm.com>
Co-Authored-By: Mahati Chamarthy <mahati.chamarthy@gmail.com>
Co-Authored-By: Peter Chng <pchng@ca.ibm.com>
Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
Co-Authored-By: Jonathan Hinson <jlhinson@us.ibm.com>
Co-Authored-By: Hamdi Roumani <roumani@ca.ibm.com>
UpgradeImpact
Change-Id: Ie6db22697ceb1021baaa6bddcf8e41ae3acb5376
John seemed to have some misgivings about using elif.
Change-Id: I39962607cf2a8f90353020f67979e92b48959dd6
Related-Change: I7732f13b06c8826537b8f8230a2785607790b8e1
Adds a new form of system metadata for objects.
Sysmeta cannot be updated by an object POST because
that would cause all existing sysmeta to be deleted.
Crypto middleware will want to add 'system' metadata
to object metadata on PUTs and POSTs, but it is ok
for this metadata to be replaced en-masse on every
POST.
This patch introduces x-object-transient-sysmeta-*
that is persisted by object servers and returned
in GET and HEAD responses, just like user metadata,
without polluting the x-object-meta-* namespace.
All headers in this namespace will be filtered
inbound and outbound by the gatekeeper, so cannot
be set or read by clients.
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Janie Richling <jrichli@us.ibm.com>
Change-Id: I5075493329935ba6790543fc82ea6e039704811d
Whatever container update override etag is sent to the object server
with a PUT must be used in container updates for subsequent
POSTs. Unfortunately the current container update override headers
(x-backend-container-update-override-*) are not persisted with the
object metadata so are not available when handling a POST.
For EC there is an ugly hack in the object server to use the
x-object-sysmeta-ec-[etag,size] values when doing a container update
for a POST.
With crypto, the encryption middleware needs to override the etag
(possibly overriding the already overridden EC etag value) with an
encrypted etag value. We therefore have a similar problem that this
override value is not persisted at the object server.
This patch introduces a new namespace for container override headers,
x-object-sysmeta-container-update-override-*, which uses object
sysmeta so that override values are persisted. This allows a general
mechanism in the object server to apply the override values (if any
have been set) from object sysmeta when constructing a container
update for a PUT or a POST. Middleware should use the
x-object-sysmeta-container-update-override-* namespace when setting
container update overrides. Middleware should be aware that other
middleware may have already set container override headers, in which
case consideration should be given to whether any existing value should
take precedence.
For backwards compatibility the existing
x-backend-container-update-override-* style headers are still
supported in the object server for EC override values, and the ugly
hack for EC etag/size override in POST updates remains in the object
server. That allows an older proxy server to be used with an upgraded
object server. The proxy server continues to use the
x-backend-container-update-override-* style headers for EC values so
that an older object server will continue to work with an upgraded
proxy server.
x-object-sysmeta-container-update-override-* headers take precedence
over x-backend-container-update-override-* headers and the use of
x-backend-container-update-override-* headers by middleware is
deprecated. Existing third party middleware that is using
x-backend-container-update-override-* headers should be modified to
use x-object-sysmeta-container-update-override-* headers in order to
be compatible with other middleware such as encryption and to ensure
that container updates during POST requests carry correct values. If
targeting multiple versions of Swift object servers it may be
necessary to send headers from both namespaces. However, in general it
is recommended to upgrade all backend servers, then upgrade proxy
servers before finally upgrading third party middleware.
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
UpgradeImpact
Change-Id: Ib80b4db57dfc2d37ea8ed3745084a3981d082784
Before this patch, the proxy ObjectController supported sending
metadata from the proxy server to object servers in "footers" that
trail the body of HTTP PUT requests, but this support was for EC
policies only. The encryption feature requires that footers are sent
with both EC and replicated policy requests in order to persist
encryption specific sysmeta, and to override container update headers
with an encrypted Etag value.
This patch:
- Moves most of the functionality of ECPutter into a generic Putter
class that is used for replicated object PUTs without footers.
- Creates a MIMEPutter subclass to support multipart and multiphase
behaviour required for any replicated object PUT with footers and
all EC PUTs.
- Modifies ReplicatedObjectController to use Putter objects in place
of raw connection objects.
- Refactors the _get_put_connections method and _put_connect_node methods
so that more code is in the BaseObjectController class and therefore
shared by [EC|Replicated]ObjectController classes.
- Adds support to call a callback that middleware may have placed
in the environ, so the callback can set footers. The
x-object-sysmeta-ec- namespace is reserved and any footer values
set by middleware in that namespace will not be forwarded to
object servers.
In addition this patch enables more than one value to be added to the
X-Backend-Etag-Is-At header. This header is used to point to an
(optional) alternative sysmeta header whose value should be used when
evaluating conditional requests with If-[None-]Match headers. This is
already used with EC policies when the ECObjectController has
calculated the actual body Etag and sent it using a footer
(X-Object-Sysmeta-EC-Etag). X-Backend-Etag-Is-At is in that case set
to X-Object-Sysmeta-Ec-Etag so as to point to the actual body Etag
value rather than the EC fragment Etag.
Encryption will also need to add a pointer to an encrypted Etag value.
However, the referenced sysmeta may not exist, for example if the
object was created before encryption was enabled. The
X-Backend-Etag-Is-At value is therefore changed to support a list of
possible locations for alternate Etag values. Encryption will place
its expected alternative Etag location on this list, as will the
ECObjectController, and the object server will look for the first
object metadata to match an entry on the list when matching
conditional requests. That way, if the object was not encrypted then
the object server will fall through to using the EC Etag value, or in
the case of a replicated policy will fall through to using the normal
Etag metadata.
If your proxy has a third-party middleware that uses X-Backend-Etag-Is-At
and it upgrades before an object server it's talking to then conditional
requests may be broken.
UpgradeImpact
Co-Authored-By: Alistair Coles <alistair.coles@hpe.com>
Co-Authored-By: Thiago da Silva <thiago@redhat.com>
Co-Authored-By: Samuel Merritt <sam@swiftstack.com>
Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota@lab.ntt.co.jp>
Closes-Bug: #1594739
Change-Id: I12a6e41150f90de746ce03623032b83ed1987ee1
The api-ref document needs a newer sphinx version, allow a 1.2 Sphinx
version to be used - like it's used in global-requirements.txt.
Change-Id: I9183cc56753fbe7e41206c6a9081899df5c3919a
Needed-By: Ifebc65b188c4f2ba35b61c0deae5ec24401df7f9
While creating a probe test for the expirer daemon, I found
the following error scenario:
1. Introduce a new object server. Initially it doesn't have a tmp_dir.
2. Have the object-replicator replicate some objects, one of them
with an expiration (X-Delete-At).
3. Send a DELETE request for the expired object.
While beginning to process the DELETE request, the fresh
object server still doesn't have a tmp_dir created.
Since the object has an old expiration value, the object server
will first call "delete_at_update", before creating a tombstone.
delete_at_update then must create an async_pending,
which will lead to an IO error, since tmp_dir doesn't exist.
As said, I have witnessed this in practice in the probe test I wrote
at https://review.openstack.org/#/c/326903/.
This patch changes pickle_async_update behavior to create
tmp_dir, in case it doesn't exist.
Change-Id: I88b0e5f75a2a28d6880694ff327ac2763c816d24
Relocates some test infrastructure in preparation for
use with encryption tests, in particular moves the test
server setup code from test/unit/proxy/test_server.py
to a new helpers.py so that it can be re-used, and adds
ability to specify additional config options for the
test servers (used in encryption tests).
Adds unit test coverage for extract_swift_bytes and functional
test coverage for container listings. Adds a check on the content
and metadata of reconciled objects in probe tests.
Change-Id: I9bfbf4e47cb0eb370e7a74d18c78d67b6b9d6645
I always get tripped up when I'm editing code that catches
AttributeError and does something with it. I'll type "req.emthod" or
something, and next thing I know I'm getting 405s in all my unit
tests. This diff removes some places where we catch AttributeError
(sometimes, having deliberately thrown it only one line before) so
that typos can crash the way Guido intended.
Change-Id: I2f7586f96b41a97e6ae254efc83218b3b5c6cc9e
If one has an object.builder file in the current directory and runs
test_ringbuilder, it will fail with an irritating error. That's because
test_use_ringfile_as_builderfile doesn't use self.tmpfile, but
object.builder - and that one might exist in the local directory.
This patch changes this, using self.tmpfile as argument name.
Closes-Bug: 1590356
Change-Id: I4b3287a36e8a5e469eb037128427dc7867910e53
Added content-disposition header to HEAD tempurl request.
As per HTTP docs[1] HEAD response must be identical to GET
except return message-body response.
[1]https://tools.ietf.org/html/rfc2616#section-9.4
Change-Id: Ie60a6fb632613055da5279db5b128ce5ee5172ae
Closes-Bug:#1539805
Every time we call start_server, check is True.
Every time we call check_server, we use the default timeout.
Change-Id: Id38182f15bcbfbb145b57cee179a8fd47ec8e2b7
