1feaf6e289
Change-Id: I2bf91c755d2f779639c773fcfe2a3b54a584b4ae
1143 lines
39 KiB
Plaintext
1143 lines
39 KiB
Plaintext
swift (2.0.0)
|
|
|
|
* Storage policies
|
|
|
|
Storage policies allow deployers to configure multiple object rings
|
|
and expose them to end users on a per-container basis. Deployers
|
|
can create policies based on hardware performance, regions, or other
|
|
criteria and independently choose different replication factors on
|
|
them. A policy is set on a Swift container at container creation
|
|
time and cannot be changed.
|
|
|
|
Full docs are at http://swift.openstack.org/overview_policies.html
|
|
|
|
* Add profiling middleware in Swift
|
|
|
|
The profile middleware provides a tool to profile Swift
|
|
code on the fly and collects statistical data for performance
|
|
analysis. A native simple Web UI is also provided to help
|
|
query and visualize the data.
|
|
|
|
* Add --quoted option to swift-temp-url
|
|
|
|
* swift-recon now supports checking the md5sum of swift.conf, which
|
|
helps deployers verify configurations are consistent across a cluster.
|
|
|
|
* Users can now set the transaction id suffix by passing in
|
|
a value in the X-Trans-Id-Extra header.
|
|
|
|
* New log_max_line_length option caps the maximum length of a log line.
|
|
|
|
* Support If-[Un]Modified-Since for object HEAD
|
|
|
|
* Added missing constraints and ratelimit parameters to /info
|
|
|
|
* Add ability to remove subsections from /info
|
|
|
|
* Unify logging for account, container, and object server processes
|
|
to provide a consistent message format. This change reorders the
|
|
fields logged for the account server.
|
|
|
|
* Add targeted config loading to swift-init. This allows an easier
|
|
and more explicit way to tell swift-init to run specific server
|
|
process configurations.
|
|
|
|
* Properly quote www-authenticate (CVE-2014-3497)
|
|
|
|
* Fix logging issue when services stop on py26.
|
|
|
|
* Change the default logged length of the auth token to 16.
|
|
|
|
* Explicitly set permissions on generated ring files to 0644
|
|
|
|
* Fix file uploads larger than 2GiB in the formpost feature
|
|
|
|
* Fixed issue where large objects would fail to download if the
|
|
auth token expired partway through the download
|
|
|
|
* Various other minor bug fixes and improvements
|
|
|
|
swift (1.13.1)
|
|
|
|
* Change the behavior of CORS responses to better match the spec
|
|
|
|
A new proxy config variable (strict_cors_mode, default to True)
|
|
has been added. Setting it to False keeps the old behavior. For
|
|
an overview of old versus new behavior, please see
|
|
https://review.openstack.org/#/c/69419/
|
|
|
|
* Invert the responsibility of the two instances of proxy-logging in
|
|
the proxy pipeline
|
|
|
|
The first proxy_logging middleware instance to receive a request
|
|
in the pipeline marks that request as handling it. So now, the
|
|
left most proxy_logging middleware handles logging for all
|
|
client requests, and the right most proxy_logging middleware
|
|
handles all other requests initiated from within the pipeline to
|
|
its left. This fixes logging related to large object
|
|
requests not properly recording bandwidth.
|
|
|
|
* Added swift-container-info and swift-account-info tools
|
|
|
|
* Allow specification of object devices for audit
|
|
|
|
* Dynamic large object COPY requests with ?multipart-manifest=get
|
|
now work as expected
|
|
|
|
* When a client is downloading a large object and one of the segment
|
|
reads gets bad data, Swift will now immediately abort the request.
|
|
|
|
* Fix ring-builder crash when a ring partition was assigned to a
|
|
deleted device, zero-weighted device, and normal device
|
|
|
|
* Make probetests work with conf.d configs
|
|
|
|
* Various other minor bug fixes and improvements.
|
|
|
|
swift (1.13.0)
|
|
|
|
* Account-level ACLs and ACL format v2
|
|
|
|
Accounts now have a new privileged header to represent ACLs or
|
|
any other form of account-level access control. The value of
|
|
the header is a JSON dictionary string to be interpreted by the
|
|
auth system. A reference implementation is given in TempAuth.
|
|
Please see the full docs at
|
|
http://swift.openstack.org/overview_auth.html
|
|
|
|
* Added a WSGI environment flag to stop swob from always using
|
|
absolute location. This is useful if middleware needs to use
|
|
out-of-spec Location headers in a response.
|
|
|
|
* Container sync proxies now support simple load balancing
|
|
|
|
* Config option to lower the timeout for recoverable object GETs
|
|
|
|
* Add a way to ratelimit all writes to an account
|
|
|
|
* Allow multiple storage_domain values in cname_lookup middleware
|
|
|
|
* Moved all DLO functionality into middleware
|
|
|
|
The proxy will automatically insert the dlo middleware at an
|
|
appropriate place in the pipeline the same way it does with the
|
|
gatekeeper middleware. Clusters will still support DLOs after upgrade
|
|
even with an old config file that doesn't mention dlo at all.
|
|
|
|
* Remove python-swiftclient dependency
|
|
|
|
* Add secondary groups to process user during privilege escalation
|
|
|
|
* When logging request headers, it is now possible to specify
|
|
specifically which headers should be logged
|
|
|
|
* Added log_requests config parameter to account and container servers
|
|
to match the parameter in the object server. This allows a deployer
|
|
to turn off log messages for these processes.
|
|
|
|
* Ensure swift.source is set for DLO/SLO requests
|
|
|
|
* Fixed an issue where overwriting segments in a dynamic manifest
|
|
could cause issues on pipelined requests.
|
|
|
|
* Properly handle COPY verb in container quota middleware
|
|
|
|
* Improved StaticWeb 404 error message on web-listings and index
|
|
|
|
* Various other minor bug fixes and improvements.
|
|
|
|
swift (1.12.0)
|
|
|
|
* Several important pieces of information have been added to /info:
|
|
|
|
- Configured constraints are included and allow a client to discover
|
|
the limits on names and object sizes that the cluster supports.
|
|
|
|
- The supported tempurl methods are now included.
|
|
|
|
- Static large object constraints are now included.
|
|
|
|
* The Last-Modified header value returned will now be the object's
|
|
timestamp rounded up to the next second. This allows subsequent
|
|
requests with If-[un]modified-Since to use the Last-Modified
|
|
value as expected.
|
|
|
|
* Non-integer values for if-delete-at headers will now properly
|
|
report a 400 error instead of a 503.
|
|
|
|
* Fix object versioning with non-ASCII container names.
|
|
|
|
* Bulk delete with POST now works properly.
|
|
|
|
* Generic means for persisting system metadata
|
|
|
|
Swift now supports system-level metadata on accounts and
|
|
containers. System metadata provides a means to store internal
|
|
custom metadata with associated Swift resources in a safe and
|
|
secure fashion without actually having to plumb custom metadata
|
|
through the core swift servers. The new gatekeeper middleware
|
|
prevents this system metadata from leaking into the request or
|
|
being set by a client.
|
|
|
|
* catch_errors and gatekeeper middleware are now forced into the proxy
|
|
pipeline if not explicitly referenced.
|
|
|
|
* New container sync configuration option, separating the end user
|
|
from knowing the required end point and adding more secure
|
|
signed requests. See
|
|
http://swift.openstack.org/overview_container_sync.html for full
|
|
information.
|
|
|
|
* bulk middleware now can be configured to retry deleting containers.
|
|
|
|
* The default yield_frequency used to keep client connections alive
|
|
during slow bulk requests was reduced from 60 seconds to 10 seconds.
|
|
While this is a change to a default, it should not affect deployments
|
|
and there is no migration process needed.
|
|
|
|
* Swift processes will attempt to set RLIMIT_NPROC to 8192.
|
|
|
|
* Server processes will now exit with a non-zero error code on config
|
|
errors.
|
|
|
|
* Warn if read_affinity is configured but not enabled.
|
|
|
|
* Fix checkmount error parsing in swift-recon.
|
|
|
|
* Log at warn level when an object is quarantined.
|
|
|
|
* Fixed CVE-2014-0006 to avoid a potential timing attack with tempurl.
|
|
|
|
* Various other minor bug fixes and improvements.
|
|
|
|
|
|
swift (1.11.0)
|
|
|
|
* Added discoverable capabilities
|
|
|
|
A Swift proxy server now by default (although it can be turned off)
|
|
will respond to requests to /info. The response to these requests
|
|
include information about the cluster and can be used by clients to
|
|
determine which features are supported in the cluster.
|
|
|
|
* Object replication ssync (an rsync alternative)
|
|
|
|
A Swift storage node can now be configured to use Swift primitives
|
|
for replication transport instead of rsync. This is an experimental
|
|
feature that is not yet considered production ready.
|
|
|
|
* If a source times out on an object server read, try another one
|
|
of them with a modified range.
|
|
|
|
* The proxy now responds to many types of requests as soon as it
|
|
has a quorum. This can help speed up responses (without
|
|
changing the results), especially when one node is acting up.
|
|
There is a post_quorum_timeout config value that can tune how
|
|
long to wait for requests to finish after a quorum has been
|
|
established.
|
|
|
|
* Add accurate timestamps in proxy log lines for the start and
|
|
end of a request. These are added as new fields on the end of
|
|
the existing log lines, and therefore should not break
|
|
existing, well-behaved log processors.
|
|
|
|
* Add an "inline" query parameter to tempurl
|
|
|
|
By default, temporary URLs add a "Content-Disposition" header
|
|
that forces many clients to download the object. Now, temporary
|
|
URLs support an optional "inline" query parameter that will
|
|
force a "Content-Disposition: inline" header to be added to the
|
|
response, overriding the default.
|
|
|
|
* Use TCP_NODELAY for created sockets. This can dramatically
|
|
lower latency for small object workloads.
|
|
|
|
* DiskFile API, with reference implementation
|
|
|
|
The DiskFile abstraction for talking to data on disk has been
|
|
refactored to allow alternate implementations to be developed.
|
|
Included in the codebase is an in-memory reference
|
|
implementation. For full documentation, please see the developer
|
|
documentation. The DiskFile API is still a work in progress and
|
|
is not yet finalized.
|
|
|
|
* Removal of swift-bench
|
|
|
|
The included benchmarking tool swift-bench has been extracted
|
|
from the codebase and is now in its own repository at
|
|
https://github.com/openstack/swift-bench. New swift-bench
|
|
binaries and packages may be found on PyPI at
|
|
https://pypi.python.org/pypi/swift-bench
|
|
|
|
* Bulk delete now also supports the POST verb, in addition to DELETE
|
|
|
|
* Added functionality to the swift-ring-builder to support
|
|
limited recreation of ring builder files from the ring file itself.
|
|
|
|
* HEAD on account now returns 410 if account was deleted and
|
|
not yet reaped. The old behavior was to return a 404.
|
|
|
|
* Fixed a bug introduced since the 1.10.0 release that
|
|
prevented expired objects from being removed from the system.
|
|
This resulted in orphaned expired objects taking up space on
|
|
the system but inaccessible to the API. This regression and
|
|
fix are only important if you have deployed code since the
|
|
1.10.0 release. For a full discussion, including a script that
|
|
can be used to clean up orphaned objects, see
|
|
https://bugs.launchpad.net/swift/+bug/1257330
|
|
|
|
* Tie socket write buffer size to server chunk size parameter. This
|
|
pairs the underlying network buffer size with the size of data
|
|
that Swift attempts to read from the connection, thereby
|
|
improving efficiency and throughput on connections.
|
|
|
|
* Fix 500 from account-quota middleware. If a user had set
|
|
X-Account-Meta-Quota-Bytes to something non-integer prior to
|
|
the installation of the account-quota middleware, then the
|
|
quota check would choke on it. Now a non-integer value is
|
|
treated as "no quota".
|
|
|
|
* Quarantine objects with busted metadata. Before, if you
|
|
encountered an object with corrupt or missing xattrs, the
|
|
object server would return a 500 on GET, and wouldn't quarantine
|
|
anything. Now the object server returns a 404 for that GET and
|
|
the corrupted file is quarantined, thus giving replication a
|
|
chance to fix it.
|
|
|
|
* Fix quarantine and error counts in audit logs
|
|
|
|
* Report transaction ID in failure exception logs
|
|
|
|
* Make pbr a build-time only dependency
|
|
|
|
* Worked around a bug in eventlet 0.9.16 where the size of the
|
|
memcache connection pools would grow unbounded.
|
|
|
|
* Tempurl keys are now properly stored as utf8
|
|
|
|
* Fixed an issue where concurrent PUT requests to accounts or
|
|
containers may result in errors due to locked databases.
|
|
|
|
* Handle copy requests in account and container quota middleware
|
|
|
|
* Now ensure that a WWW-Authenticate header is on all 401 responses
|
|
|
|
* Various other bug fixes and improvements
|
|
|
|
swift (1.10.0)
|
|
|
|
* Added support for pooling memcache connections
|
|
|
|
* Added support to replicating handoff partitions first in object
|
|
replication. Can also configure how many remote nodes a storage node
|
|
must talk to before removing a local handoff partition.
|
|
|
|
* Fixed bug where memcache entries would not expire
|
|
|
|
* Much faster calculation for choosing handoff nodes
|
|
|
|
* Added container listing ratelimiting
|
|
|
|
* Fixed issue where the proxy would continue to read from a storage
|
|
server even after a client had disconnected
|
|
|
|
* Added support for headers that are only visible to the owner of a Swift
|
|
account
|
|
|
|
* Fixed ranged GET with If-None-Match
|
|
|
|
* Fixed an issue where rings may not be balanced after initial creation
|
|
|
|
* Fixed internationalization support
|
|
|
|
* Return the correct etag for a static large object on the PUT response
|
|
|
|
* Allow users to extract archives to containers with ACLs set
|
|
|
|
* Fix support for range requests against static large objects
|
|
|
|
* Now logs x-copy-from header in a useful place
|
|
|
|
* Reverted back to old XML output of account and container listings to
|
|
ensure older clients do not break
|
|
|
|
* Account quotas now appropriately handle copy requests
|
|
|
|
* Fix issue with UTF-8 handling in versioned writes
|
|
|
|
* Various other bug fixes and improvements, including support for running
|
|
Swift under Pypy and continuing work to support storage policies
|
|
|
|
swift (1.9.1)
|
|
|
|
* Disallow PUT, POST, and DELETE requests from creating older tombstone
|
|
files, preventing the possibility of filling up the disk and removing
|
|
unnecessary container updates.
|
|
|
|
* Set default wsgi workers to cpu_count
|
|
|
|
Change the default value of wsgi workers from 1 to auto. The new
|
|
default value for workers in the proxy, container, account & object
|
|
wsgi servers will spawn as many workers per process as you have cpu
|
|
cores. This will not be ideal for some configurations, but it's much
|
|
more likely to produce a successful out of the box deployment.
|
|
|
|
* Added reveal_sensitive_prefix config setting to filter the auth token
|
|
logged by the proxy server.
|
|
|
|
* Ensure Keystone's reseller prefix ends with an underscore. Previously
|
|
this was a recommendation--now it is enforced.
|
|
|
|
* Added log_file_pattern config to swift-drive-audit for drive errors
|
|
|
|
* Add support for telling Swift to detect a content type on a request.
|
|
|
|
* Additional object stats are now logged in the object auditor
|
|
|
|
* Moved the DiskFile interface into its own module
|
|
|
|
* Ensure the SQLite cursors are closed when creating functions
|
|
|
|
* Better support for valid Accept headers
|
|
|
|
* In Keystone, don't allow users to delete their own account
|
|
|
|
* Return a UTC timezone designator in container listings
|
|
|
|
* Ensure that users can't remove their account quotas
|
|
|
|
* Allow floating point value for dispersion coverage
|
|
|
|
* Fix incorrect error page handling in staticweb
|
|
|
|
* Add utf-8 charset to multipart-manifest=get response.
|
|
|
|
* Allow dispersion tools to use keystone server with insecure certificate
|
|
|
|
* Ensure that files are always closed in tests
|
|
|
|
* Use OpenStack's "Hacking" guidelines for code formatting
|
|
|
|
* Various other minor bug fixes and improvements
|
|
|
|
swift (1.9.0)
|
|
|
|
* Global clusters support
|
|
|
|
The "region" concept introduced in Swift 1.8.0 has been augmented with
|
|
support for using a separate replication network and configuring read
|
|
and write affinity. These features combine to offer support for a single
|
|
Swift cluster spanning wide geographic area.
|
|
|
|
* Disk performance
|
|
|
|
The object server now can be configured to use threadpools to increase
|
|
performance and smooth out latency throughout the system. Also, many
|
|
disk operations were reordered to increase reliability and improve
|
|
performance.
|
|
|
|
* Added config file conf.d support
|
|
|
|
Allow Swift daemons and servers to optionally accept a directory as the
|
|
configuration parameter. This allows different parts of the config file
|
|
to be managed separately, eg each middleware could use a separate file
|
|
for its particular config settings.
|
|
|
|
* Allow two TempURL keys per account
|
|
|
|
By adding a second key, a user can safely rotate keys and prevent URLs
|
|
already in use from becoming invalid. TempURL middlware has also been
|
|
updated to allow a configuable set of allowed methods and to prevent a
|
|
bugrelated to content-disposition names.
|
|
|
|
* Added crossdomain.xml middleware. See
|
|
http://docs.openstack.org/developer/swift/crossdomain.html for details
|
|
|
|
* Added rsync bandwidth limit setting for object replicator
|
|
|
|
* Transaction ID updated to include the time and an optional suffix
|
|
|
|
* Added x-remove-versions-location header to disable versioned writes
|
|
|
|
* Improvements to support for Keystone ACLs
|
|
|
|
* Added parallelism to object expirer daemon
|
|
|
|
* Added support for ring hash prefix in addition to the existing suffix
|
|
|
|
* Allow all headers requested for CORS
|
|
|
|
* Stop getting useless bytes on manifest Range requests
|
|
|
|
* Improved container-sync resiliency
|
|
|
|
* Added example Apache config files. See
|
|
http://docs.openstack.org/developer/swift/apache_deployment_guide.html
|
|
for more info
|
|
|
|
* If an account is marked as deleted but hasn't been reaped and is still
|
|
on disk, responses will include an "X-Account-Status" header
|
|
|
|
* Fix 503 on account/container HEAD with invalid format
|
|
|
|
* Added extra safety on account-level DELETE when using bulk deletes
|
|
|
|
* Made colons quote-safe in logs (mainly for IPv6)
|
|
|
|
* Fixed bug with bulk delete max items
|
|
|
|
* Fixed static large object manifest range requests
|
|
|
|
* Prevent static large objects from containing other static large objects
|
|
|
|
* Fixed issue with use of delimiter in container queries where some
|
|
objects would not be listed
|
|
|
|
* Various other minor bug fixes and improvements
|
|
|
|
swift (1.8.0)
|
|
|
|
* Make rings' replica count adjustable
|
|
|
|
* Added a region tier to the ring above zones
|
|
|
|
* Added timing-based sorting of object servers on read requests
|
|
|
|
* Added support for auto-extract archive uploads
|
|
|
|
* Added support for bulk delete requests
|
|
|
|
* Added support for large objects with static manifests
|
|
|
|
* Added list_endpoints middleware to provide an API for determining where
|
|
the ring places data
|
|
|
|
* proxy-logging middleware can now handle logging for other middleware
|
|
|
|
proxy-logging should be used twice in the proxy pipeline. The first
|
|
handles middleware logs for requests that never made it all the way
|
|
to the server. The last handles requests that do make it to the server.
|
|
|
|
This is a change that may require an update to your proxy server
|
|
config file or custom middleware that you may be using. See the full
|
|
docs at http://docs.openstack.org/developer/swift/misc.html#module-swift.common.middleware.proxy_logging.
|
|
|
|
* Changed the default sample rate for a few high-traffic requests.
|
|
|
|
Added log_statsd_sample_rate_factor to globally tune the StatsD
|
|
sample rate. This tunable can be used to reduce StatsD traffic
|
|
proportionally for all metrics and is intended to replace
|
|
log_statsd_default_sample_rate, which is left alone for
|
|
backward-compatibility, should anyone be using it.
|
|
|
|
* Added swift_hash_path_prefix option to swift.conf
|
|
|
|
New deployments are advised to set this value to a random secret
|
|
to protect against hash collisions
|
|
|
|
* Added user-managed container quotas
|
|
|
|
* Added support for account-level quotas managed by an auth reseller
|
|
|
|
* Added --run-dir option to swift-init
|
|
|
|
* Added more options to swift-bench
|
|
|
|
* Added support for CORS "actual requests"
|
|
|
|
* Added fallocate_reserve option to protect against full drives
|
|
|
|
* Allow ring rebalance to take a seed
|
|
|
|
* Ring serialization will now produce the same gzip file (Py2.7)
|
|
|
|
* Added support to swift-drive-audit for handling rotated logs
|
|
|
|
* Added first-byte latency timings for GET requests
|
|
|
|
* Added per disk PUT timing monitoring support
|
|
|
|
* Added speed limit options for DB auditor
|
|
|
|
* Force log entries to be one line
|
|
|
|
* Ensure that fsync is used and not just fdatasync
|
|
|
|
* Improved handoff node selection
|
|
|
|
* Deprecated keystone is_admin feature
|
|
|
|
* Fix large objects with unicode in the segment names
|
|
|
|
* Update Swift's MemcacheRing to provide API compatibility with
|
|
standard Python memcache libraries
|
|
|
|
* Various other minor bug fixes and improvements
|
|
|
|
swift (1.7.6)
|
|
|
|
* Better tempauth storage URL guessing
|
|
|
|
* Added --top option to swift-recon -d
|
|
|
|
* Allow optional, temporary healthcheck failure
|
|
|
|
* keystoneauth middleware now supports cross-tenant ACLs
|
|
|
|
* Add dispersion report flags to limit reports
|
|
|
|
* Add config option to turn eventlet debug on/off
|
|
|
|
* Added override option for swift-init's KILL_WAIT
|
|
|
|
* Added oldest and most recent replication pass to swift-recon
|
|
|
|
* Fixed 500 error response when GETing a many-segment manifest
|
|
|
|
* Memcached keys now use a delta timeout when possible
|
|
|
|
* Refactor DiskFile to hide temp file names and exts
|
|
|
|
* Remove IP-based container-sync ACLs from auth middlewares
|
|
|
|
* Fixed bug in deleting memcached account info data
|
|
|
|
* Fixed lazy-listing of object manifest segments
|
|
|
|
* Fixed bug where a ? in the object name caused an error
|
|
|
|
* Swift now returns 406 if it can't satisfy Accept
|
|
|
|
* Fix infinite recursion bug in object replicator
|
|
|
|
* Swift will now reject names with NULL characters
|
|
|
|
* Fixed object-auditor logging to use a minimum of unix sockets
|
|
|
|
* Various other minor bug fixes and improvements
|
|
|
|
swift (1.7.5)
|
|
|
|
* Support OPTIONS verb, including CORS preflight requests
|
|
|
|
* Added support for custom log handlers
|
|
|
|
* Range support is extended to support GET requests with multiple ranges.
|
|
Multi-range GETs are not yet supported against large-object manifests.
|
|
|
|
* Cluster constraints are now settable by config
|
|
|
|
* Replicators can now run against specific devices or partitions
|
|
|
|
* swift-bench now supports running on multiple cores and multiple servers
|
|
|
|
* Added partition option to swift-get-nodes
|
|
|
|
* Allow underscores in account and user in tempauth via base64 encodings
|
|
|
|
* New option to the dispersion report to output the missing partitions
|
|
|
|
* Changed storage server StatsD metrics to report timings instead of
|
|
counts for errors. See the admin guide for the updated metric names.
|
|
|
|
* Removed a dependency on WebOb and replaced it with an internal module
|
|
|
|
* Fixed config parsing in swift-bench -x
|
|
|
|
* Fixed sample_rate in StatsD logging
|
|
|
|
* Track unlinks of async_pendings with StatsD
|
|
|
|
* Remove double GET on range requests
|
|
|
|
* Allow unsetting of X-Container-Sync-To and ACL headers
|
|
|
|
* DB reclamation now removes empty suffix directories
|
|
|
|
* Fix non-standard 100-continue behavior
|
|
|
|
* Allow object-expirer to delete the last copy of a versioned object
|
|
|
|
* Only set TCP_KEEPIDLE on systems where it is supported
|
|
|
|
* Fix stdin flush and fdatasync issues on BSD platforms
|
|
|
|
* Allow object-expirer to delete the last version of an object
|
|
|
|
* Various other minor bug fixes and improvements
|
|
|
|
swift (1.7.4)
|
|
|
|
* Fix issue where early client disconnects may have caused a memory leak
|
|
|
|
swift (1.7.2)
|
|
|
|
* Fix issue where memcache serialization was not properly loading
|
|
the config value
|
|
|
|
swift (1.7.0)
|
|
|
|
* Use custom encoding for ring data instead of pickle
|
|
|
|
Serialize RingData in a versioned, custom format which is a combination
|
|
of a JSON-encoded header and .tostring() dumps of the
|
|
replica2part2dev_id arrays. This format deserializes hundreds of times
|
|
faster than rings serialized with Python 2.7's pickle (a significant
|
|
performance regression for ring loading between Python 2.6 and Python
|
|
2.7). Fixes bug 1031954.
|
|
|
|
The new implementation is backward-compatible; if a ring
|
|
does not begin with a new-style magic string, it is assumed to be an
|
|
old-style pickle-dumped ring and is handled as before. So new Swift
|
|
code can read old rings, but old Swift code will not be able to read
|
|
newly-serialized rings.
|
|
|
|
* Do not use pickle for serialization in memcache, but JSON
|
|
|
|
To avoid issues on upgrades (unability to read pickled values, and cache
|
|
poisoning for old servers not understanding JSON), we add a
|
|
memcache_serialization_support configuration option, with the following
|
|
values:
|
|
|
|
0 = older, insecure pickle serialization
|
|
1 = json serialization but pickles can still be read (still insecure)
|
|
2 = json serialization only (secure and the default)
|
|
|
|
To avoid an instant full cache flush, existing installations should
|
|
upgrade with 0, then set to 1 and reload, then after some time (24
|
|
hours) set to 2 and reload. Support for 0 and 1 will be removed in
|
|
future versions.
|
|
|
|
* Update proxy-server StatsD logging. This is a significant change to the
|
|
existing StatsD intigration. Docs for this feature can be found in
|
|
doc/source/admin_guide.rst.
|
|
|
|
* Improved swift-bench to allow random object sizes and better usability
|
|
|
|
* Updated probe tests
|
|
|
|
* Replicator removal metrics are now generated on a per-device basis
|
|
|
|
* Made object replicator locking more optimistic
|
|
|
|
* Split proxy-server code into separate modules
|
|
|
|
* Fixed bug where swift-recon would not report all unmounted drives
|
|
|
|
* Fixed issue where a LockTimeout may have caused a file descriptor to
|
|
not be closed properly
|
|
|
|
* Fixed a bug where an error may have caused the proxy to stop returning
|
|
data to a client
|
|
|
|
* Fixed bug where expirer would get confused by odd deletion times
|
|
|
|
* Fixed a bug where auto-creating accounts would return an error if they
|
|
were recreated after being deleted
|
|
|
|
* Fix when rate_limit_after_segment kicks in
|
|
|
|
* fallocate() failures properly return HTTPInsufficientStorage from
|
|
object-server before reading from wsgi.input, allowing the proxy
|
|
server to quickly error_limit that node
|
|
|
|
* Fixed error with large object manifests and x-newest headers on GET
|
|
|
|
* Various other minor bug fixes and improvements
|
|
|
|
swift (1.6.0)
|
|
|
|
* Removed bin/swift and swift/common/client.py from the swift repo. These
|
|
tools are now managed in the python-swiftclient project. The
|
|
python-swiftclient project is a second deliverable of the openstack
|
|
swift project.
|
|
|
|
* Moved swift_auth (openstack keystone) middleware from keystone project
|
|
into swift project
|
|
|
|
* Made dispersion report work with any replica count other than 3. This
|
|
substantially affects the JSON output of the dispersion report, and any
|
|
tools written to consume this output will need to be updated.
|
|
|
|
* Added Solaris (Illumos) compatibility
|
|
|
|
* Added -a option to swift-get-nodes to show all handoffs
|
|
|
|
* Add UDP protocol support for logger
|
|
|
|
* Added config options for rate limiting of large object downloads.
|
|
|
|
* Added config option `log_handoffs` (defaults to True) to proxy server
|
|
to log and update statsd with information about when a handoff node is
|
|
used. This is helpful to track the health of the cluster.
|
|
|
|
* swift-bench can now use auth 2.0
|
|
|
|
* Support forbidding substrings based on a regexp in name_filter
|
|
middleware
|
|
|
|
* Hardened internal server processes so only authorized methods can be
|
|
called.
|
|
|
|
* Made ranged requests on large objects work correctly when size of
|
|
manifest file is not 0 byte
|
|
|
|
* Added option to dispersion report to print 404s to stdout
|
|
|
|
* Fix object replication on older rsync versions when using ipv4
|
|
|
|
* Fixed bug with container reclaim/report race
|
|
|
|
* Make object server's caching more configurable.
|
|
|
|
* Check disk failure before syncing for each partition
|
|
|
|
* Allow special characters to be referenced by manifest objects
|
|
|
|
* Validate devices and partitions to avoid directory traversals
|
|
|
|
* Support WebOb 1.2
|
|
|
|
* Ensure that accessing the ring devs reloads the ring if necessary.
|
|
Specifically, this allows replication to work when it has been started
|
|
with an empty ring.
|
|
|
|
* Various other minor bug fixes and improvements
|
|
|
|
swift (1.5.0)
|
|
|
|
* New option to toggle SQLite database preallocation with account
|
|
and container servers.
|
|
|
|
IMPORTANT:
|
|
The default for database preallocation is now off when before
|
|
it was always on. This will affect performance on clusters that
|
|
use standard drives with shared account, container, object
|
|
servers. Such deployments will need to update their
|
|
configurations to turn database preallocation back on (see
|
|
account-server.conf-sample and container-server.conf.sample
|
|
files).
|
|
|
|
If you are using dedicated account and container servers with
|
|
SSDs, you should defragment your file systems after upgrade and
|
|
should notice dramatically less disk usage.
|
|
|
|
* swift3 middleware removed and moved to http://github.com/fujita/swift3.
|
|
This will require a config change in the proxy server and adds a new
|
|
dependency for deployers using this middleware.
|
|
|
|
* Moved proxy server logging to middleware. This requires a config change
|
|
in the proxy server.
|
|
|
|
* Added object versioning feature. (See docs for full description)
|
|
|
|
* Add statsd logging throughout the system (beta, some event names may
|
|
change)
|
|
|
|
* Expanded swift-recon middleware support
|
|
|
|
* The ring builder now supports as-unique-as-possible partition
|
|
placement, unified balancing methods, and can work on more than one
|
|
device at a time.
|
|
|
|
* Numerous bug fixes to StaticWeb (previously unusable at scale).
|
|
|
|
* Bug fixes to all middleware to allow passthrough requests under various
|
|
conditions and to share pre-authed request code (which previously had
|
|
differing behaviors and interaction bugs).
|
|
|
|
* Bug fix to object expirer that could cause infinite looping.
|
|
|
|
* Added optional delay to account reaping.
|
|
|
|
* Async-pending write optimization.
|
|
|
|
* Dispersion tools now support multiple auth versions
|
|
|
|
* Updated man pages
|
|
|
|
* Proxy server can now deny requests to particular hostnames
|
|
|
|
* Updated docs for domain remap middleware
|
|
|
|
* Updated docs for cname lookup middleware
|
|
|
|
* Made swift CLI binary easier to wrap
|
|
|
|
* Proxy will now also return X-Timestamp header
|
|
|
|
* Added associated projects doc as a place to track ecosystem projects
|
|
|
|
* end_marker made consistent across both object and container listings
|
|
|
|
* Various other minor bug fixes and improvements
|
|
|
|
swift (1.4.8)
|
|
|
|
* Added optional max_containers_per_account restriction
|
|
|
|
* Added alternate metadata header removal method
|
|
|
|
* Added optional name_check middleware filter
|
|
|
|
* Added support for venv-based test runs with tox
|
|
|
|
* StaticWeb behavior change with X-Web-Mode: true and
|
|
non-StaticWeb-enabled containers (immediately 404s instead of passing
|
|
the request on down the WSGI pipeline).
|
|
|
|
* Fixed typo in swift-dispersion-report JSON output.
|
|
|
|
* Swift-Recon-related fix to create temporary files on the same disk as
|
|
their final destinations.
|
|
|
|
* Updated return codes in swift3 middleware
|
|
|
|
* Fixed swift3 middleware to allow Content-Range header in response
|
|
|
|
* Updated swift.common.client and swift CLI tool with auth 2.0 changes
|
|
|
|
* Swift CLI tool now supports common openstack auth args
|
|
|
|
* Body of HTTP responses now included in error messages of swift CLI tool
|
|
|
|
* Refactored some ring building functions for clarity and simplicity
|
|
|
|
swift (1.4.7)
|
|
|
|
* Improvements to account and container replication.
|
|
|
|
* Fix for account servers allowing .pending to exist before .db.
|
|
|
|
* Fixed possible key-guessing exploit in formpost.
|
|
|
|
* Fixed bug in ring builder when removing a large percentage of devices.
|
|
|
|
* Swift CLI tool now supports openstack-standard CLI flags.
|
|
|
|
* New JSON output option for swift-dispersion-report.
|
|
|
|
* Removed old stats tools.
|
|
|
|
* Other bug fixes and documentation updates.
|
|
|
|
swift (1.4.6)
|
|
|
|
* TempURL and FormPost middleware added
|
|
|
|
* Added memcache.conf option
|
|
|
|
* Dropped eval-based json parser fallback
|
|
|
|
* Properly lose all groups when dropping privileges
|
|
|
|
* Fix permissions when creating files
|
|
|
|
* Fixed bug regarding negative Content-Length in requests
|
|
|
|
* Consistent formatting on Last-Modified response header
|
|
|
|
* Added timeout option to swift-recon
|
|
|
|
* Allow arguments to be passed to nosetest
|
|
|
|
* Removed tools/rfc.sh
|
|
|
|
* Other minor bug fixes
|
|
|
|
swift (1.4.5)
|
|
|
|
* New swift-orphans and swift-oldies command line tools to detect
|
|
orphaned Swift processes and long running processes.
|
|
|
|
* Command line tool "swift" now supports marker queries.
|
|
|
|
* StaticWeb middleware improved to save an extra request when
|
|
possible.
|
|
|
|
* Updated swift-init to support swift-object-expirer.
|
|
|
|
* Fixed object replicator timeout handling [bug 814263].
|
|
|
|
* Fixed accept header 503 vs. 400 [bug 891247].
|
|
|
|
* More exception handling for auditors.
|
|
|
|
* Doc updates for PPA [bug 905608].
|
|
|
|
* Doc updates to explain replication more clearly [bug 906976].
|
|
|
|
* Updated SAIO instructions to no longer mention ~/swift/trunk.
|
|
|
|
* Fixed docstrings in the ring code.
|
|
|
|
* PEP8 Updates.
|
|
|
|
swift (1.4.4)
|
|
|
|
* Fixes to prevent socket hoarding (memory leak)
|
|
|
|
* Add sockstat info to recon.
|
|
|
|
* Fixed leak from SegmentedIterable.
|
|
|
|
* Fixed bufferedhttp to deref socks and fps.
|
|
|
|
* Add support for OS Auth API version 2.
|
|
|
|
* Make Eventlet's WSGI server log differently.
|
|
|
|
* Updated TimeoutError and except Exception refs.
|
|
|
|
* Fixed time-sensitive tests.
|
|
|
|
* Fixed object manifest etags.
|
|
|
|
* Fixes for swift-recon disk usage distribution graph.
|
|
|
|
* Adding new manpages for configuration files.
|
|
|
|
* Change bzr to swift in getting_started doc.
|
|
|
|
* Fixes the HTTPConflict import.
|
|
|
|
* Expiring Objects Support.
|
|
|
|
* Fixing bug with x-trans-id.
|
|
|
|
* Requote the source when doing a COPY.
|
|
|
|
* Add documentation for Swift Recon.
|
|
|
|
* Make drive audit regexes detect 4-letter drives.
|
|
|
|
* Adding what acc/cont/obj into the ratelimit error messages.
|
|
|
|
* Query only specific zone via swift-recon.
|
|
|
|
swift (1.4.3)
|
|
|
|
* Additional quarantine catching code.
|
|
|
|
* Added client_ip to all proxy log lines not otherwise containing it.
|
|
|
|
* Content-Type is now application/xml for "GET services/bucket" swift3
|
|
middleware requests.
|
|
|
|
* Alpha release of the Swift Recon Experiment
|
|
|
|
* Fix last modified date for swift3 middleware.
|
|
|
|
* Fix to clear account/container metadata on account/container deletion.
|
|
|
|
* Fix for corner case regarding X-Newest.
|
|
|
|
* Fix for object auditor running out of file descriptors.
|
|
|
|
* Fix to return all proper headers for manifest objects.
|
|
|
|
* Fix to the swift tool to strip any leading slashes on file names when
|
|
uploading.
|
|
|
|
swift (1.4.2)
|
|
|
|
* Removed stats/logging code from Swift [now in separate slogging project].
|
|
|
|
* Container Synchronization Feature - First Edition
|
|
|
|
* Fix swift3 authentication bug about the Date and X-Amz-Date handling.
|
|
|
|
* Changing ratelimiting so that it only limits PUTs/DELETEs.
|
|
|
|
* Object POSTs are implemented as COPYs now by default (you can revert to
|
|
previous implementation with conf object_post_as_copy = false)
|
|
|
|
* You can specify X-Newest: true on GETs and HEADs to indicate you want
|
|
Swift to query all backend copies and return the newest version
|
|
retrieved.
|
|
|
|
* Object COPY requests now always copy the newest object they can find.
|
|
|
|
* Account and container GETs and HEADs now shuffle the nodes they use to
|
|
balance load.
|
|
|
|
* Fixed the infinite charset: utf-8 bug
|
|
|
|
* This fixes the bug that drop_buffer_cache() doesn't work on systems where
|
|
off_t isn't 64 bits.
|
|
|
|
swift (1.4.1)
|
|
|
|
* st renamed to swift
|
|
|
|
* swauth was separated froms swift. It is now its own project and can be
|
|
found at https://github.com/gholt/swauth.
|
|
|
|
* tempauth middleware added as an extremely limited auth system for dev
|
|
work.
|
|
|
|
* Account and container listings now properly labeled UTF-8 (previously the
|
|
label was "utf8").
|
|
|
|
* Accounts are auto-created if an auth token is valid when the
|
|
account_autocreate proxy config parameter is set to true.
|
|
|
|
swift (1.4.0)
|
|
|
|
* swift-bench now cleans up containers it creates.
|
|
|
|
* WSGI servers now load WSGI filters and applications after forking for
|
|
better plugin support.
|
|
|
|
* swauth-cleanup-tokens now handles 404s on token containers and tokens
|
|
better.
|
|
|
|
* Proxy logs the remote IP address as the client IP in the absence of
|
|
X-Forwarded-For and X-Cluster-Client-IP headers instead of - like it did
|
|
before.
|
|
|
|
* Swift3 WSGI middleware added support for param-signed URLs.
|
|
|
|
* swauth- scripts now exit with proper exit codes.
|
|
|
|
* Fixed a bug where allowed_headers weren't honored for HEAD requests.
|
|
|
|
* Double quarantining of corrupted sqlite3 databases now works.
|
|
|
|
* Fix for Object replicator breaking when running object replicator with no
|
|
objects on the server.
|
|
|
|
* Added the Accept-Ranges header to GET and HEAD requests.
|
|
|
|
* When a single object has multiple async pending updates on a single
|
|
device, only latest async pending is now sent.
|
|
|
|
* Fixed issue of Swift3 WSGI middleware not working correctly with '/' in
|
|
object names.
|
|
|
|
* Renamed swift-stats-* to swift-dispersion-* to avoid confusion with log
|
|
stats stuff.
|
|
|
|
* Added X-Trans-Id transaction id header to every response.
|
|
|
|
* Fixed a Python 2.7 compatibility problem.
|
|
|
|
* Now using bracketed notation for ip literals in rsync calls, so
|
|
compressed ipv6 literals work.
|
|
|
|
* Added a container stats collector and refactoring some of the stats code.
|
|
|
|
* Changed subdir nodes in XML formatted object listings to align with
|
|
object nodes. Now: <subdir name="foo"><name>foo</name></subdir> Before:
|
|
<subdir name="foo" />.
|
|
|
|
* Fixed bug in Swauth to support for multiple swauth instances.
|
|
|
|
* swift-ring-builder: Added list_parts command which shows common
|
|
partitions for a given list of devices.
|
|
|
|
* Object auditor now shows better statistics updates in the logs.
|
|
|
|
* Stats uploaders now allow overrides for source_filename_pattern and
|
|
new_log_cutoff values.
|