In this change, there are proposed improvement to Swift documentation aimed at
first-time contributors. They include a simplification of the Getting
Started page and a new page with some basic instructions/commands that the
first-time contributor should know. In addition, it shows some common errors
that the first-time contributor may find when executing git rebase and
information on how to track your changes.
Change-Id: I704202955093736b2f3b4102a649690a0392c6b0
After discussion https://review.openstack.org/#/c/129384/ moving
to the doc directory in swift repo.
This lets us eliminate the object-api repo along with all the <service>-
api repos and move content to audience-centric locations.
Change-Id: Ia0d9973847f7409a02dcc1a0e19400a3c3ecdf32
Add overview and example information for using Storage Policies.
DocImpact
Implements: blueprint storage-policies
Change-Id: I6f11f7a1bdaa6f3defb3baa56a820050e5f727f1
The purpose of GateKeeper mostly relates to the development of new swift code,
so I threw together a guide for development_middleware that covers some basics
with a eye towards metadata handling in-particular.
I also fixed up some missing autodoc's, split out middleware autodoc and added
some ref's here and about so I could link to them from the
development_middleware guide.
DocImpact
Change-Id: I20dd942ea8df9e33c3e794cb49669ffa1332c63e
Refactor on-disk knowledge out of the object server by pushing the
async update pickle creation to the new DiskFileManager class (name is
not the best, so suggestions welcome), along with the REPLICATOR
method logic. We also move the mount checking and thread pool storage
to the new ondisk.Devices object, which then also becomes the new home
of the audit_location_generator method.
For the object server, a new setup() method is now called at the end
of the controller's construction, and the _diskfile() method has been
renamed to get_diskfile(), to allow implementation specific behavior.
We then hide the need for the REST API layer to know how and where
quarantining needs to be performed. There are now two places it is
checked internally, on open() where we verify the content-length,
name, and x-timestamp metadata, and in the reader on close where the
etag metadata is checked if the entire file was read.
We add a reader class to allow implementations to isolate the WSGI
handling code for that specific environment (it is used no-where else
in the REST APIs). This simplifies the caller's code to just use a
"with" statement once open to avoid multiple points where close needs
to be called.
For a full historical comparison, including the usage patterns see:
https://gist.github.com/portante/5488238
(as of master, 2b639f5, Merge
"Fix 500 from account-quota This Commit
middleware")
--------------------------------+------------------------------------
DiskFileManager(conf)
Methods:
.pickle_async_update()
.get_diskfile()
.get_hashes()
Attributes:
.devices
.logger
.disk_chunk_size
.keep_cache_size
.bytes_per_sync
DiskFile(a,c,o,keep_data_fp=) DiskFile(a,c,o)
Methods: Methods:
*.__iter__()
.close(verify_file=)
.is_deleted()
.is_expired()
.quarantine()
.get_data_file_size()
.open()
.read_metadata()
.create() .create()
.write_metadata()
.delete() .delete()
Attributes: Attributes:
.quarantined_dir
.keep_cache
.metadata
*DiskFileReader()
Methods:
.__iter__()
.close()
Attributes:
+.was_quarantined
DiskWriter() DiskFileWriter()
Methods: Methods:
.write() .write()
.put() .put()
* Note that the DiskFile class * Note that the DiskReader() object
implements all the methods returned by the
necessary for a WSGI app DiskFileOpened.reader() method
iterator implements all the methods
necessary for a WSGI app iterator
+ Note that if the auditor is
refactored to not use the DiskFile
class, see
https://review.openstack.org/44787
then we don't need the
was_quarantined attribute
A reference "in-memory" object server implementation of a backend
DiskFile class in swift/obj/mem_server.py and
swift/obj/mem_diskfile.py.
One can also reference
https://github.com/portante/gluster-swift/commits/diskfile for the
proposed integration with the gluster-swift code based on these
changes.
Change-Id: I44e153fdb405a5743e9c05349008f94136764916
Signed-off-by: Peter Portante <peter.portante@redhat.com>
The main purpose of this patch is to lay the groundwork for allowing
the container and account servers to optionally use pluggable backend
implementations. The backend.py files will eventually be the module
where the backend APIs are defined via docstrings of this reference
implementation. The swift/common/db.py module will remain an internal
module used by the reference implementation.
We have a raft of changes to docstrings staged for later, but this
patch takes care to relocate ContainerBroker and AccountBroker into
their new home intact.
Change-Id: Ibab5c7605860ab768c8aa5a3161a705705689b04
The crossdomain doc was named *.xml instead of *.rst causing it to not
get built or included in the toctree where it was supposed to.
The apache deployment guide wasn't linked to from anywhere, so I added
it under the normal deployment guide.
Change-Id: I817a1f2ca1ed7913e8ea5155cc1fac07caf0b637
Allows client-side technologies such as Flash, Java and Silverlight running
on web pages served elsewhere to interact with the Swift API.
Bug #1159960
Change-Id: I7d0533a0aaf189ac452abbd983469acb064fdca4
Support separate replication ip address:
- Added new function in utils. This function provides ability
to select separate IP address for replication service.
- Db_replicator and object replicators were changed.
Replication process uses new function now.
Replication network parameters:
- Replication network fields (replication_ip, replication_port)
support was added to device dictionary in swift-ring-builder script.
- Changes were made to support new fields in search, show and set_info
functions.
Implementation of replication servers:
- Separate replication servers use the same code as normal replication
servers, but with replication_server parameter = True. When using a
separate replication network, the non-replication servers set
replication_server = False. When there is no separate replication
network (the default case), replication_server is not included in the config.
DocImpact
Change-Id: Ie9af5bdcdf9241c355e36053ca4adfe49dc35bd0
Implements: blueprint dedicated-replication-network
Fix for bug 1095130
* Added a wrapper function around public methods to handle
CORS actual requests. These requests need to return some
extra headers to be valid responses to a CORS request.
Access-Control-Expose-Headers and Access-Control-Allow-Origin.
* Added support for the CORS header Access-Control-Expose-Headers.
* Some refactoring of the OPTIONS method so the
"is_origin_allowed" logic can be reused.
* Added a little extra detail to the CORS documentation.
DocImpact
Change-Id: I68538e472a900775427f21a8a59e738a83dcc8bc
Removed sidebar with broken (static) links referencing out-of-date docs.
Added an external link to the Swift API docs
fixes bug #1025099
Change-Id: I7f3106175b84b1063f74aa6c5693ab1e422cdb59
There are 3 sections in there, all useless.
Section 1 tells you how to install Swift packages from the swift-core
PPA. However, the latest version there is ancient.
Section 2 tells you how to build your own Swift packages. However, it
talks about getting the source code from the "debian" branch in bzr,
which is obviously really old.
Section 3 tells you how to take the packages from section 2 and
install them. This isn't too out-of-date, but since section 2 doesn't
work any more, section 3 is useless.
Since stale docs are worse than no docs, there's no current
information in this document, and bringing it up-to-date requires a
whole pile of work, I've chosen to delete it entirely.
Also pulled out a couple references to the PPA elsewhere.
Fixes bug 917385.
Fixes bug 1026145.
Change-Id: I510bd8619531fe110419e5488bd20d3602868d66
Rate Limit middleware is now at
http://dpgoetz.github.com/swift-ratelimit/
For current users of Rate Limit, this will require installing the new
package and changing the "use" line of the ratelimit conf section to:
[filter:ratelimit]
use = egg:swiftratelimit#middleware
And then 'swift-init proxy reload'.
Change-Id: I2ab774e9cee9fba4103c1be4bea6d52d1adb29f7
In the interest of keeping the core Swift code repository less
complex, but still offering a quick way to find associated projects
that enhance or use Swift, I've added this new Associated Projects
page prominently to the Swift documentation.
This will allow much less barrier to entry on enhancing Swift as
other projects can work independently and will only need to wait on
the core Swift project for approval of minimal tweaks to the core
Swift code base.
It will also allow an easy central place to find cool associated
projects that otherwise might go unnoticed or even duplicated.
The plan is to move non-essential projects that are currently
embedded in the Swift repository out into their own projects with
links to them on this new page. This would include items such as
(just what I can think of right now): bin/swift command line tool and
clients, swift-bench, swift-dispersion, TempURL, FormPost, StaticWeb,
Rate Limiting, Swift3, Domain Remap, and CNAME Lookup.
After all that is done, those projects will be able to move forward
much more quickly and new developers for Swift itself will have much
less to learn and get confused about.
Change-Id: Ib8447d8bd480f0a3d8f0413137ccdba73a11dd91
In the side bar, links to past versions and docs.openstack.org
On the index.rst, links to wiki and docs.openstack.org
Change-Id: Icf33c6f396e1ab016fd86a56e61df3e063a1bae2
Object versioning in swift is implemented by setting a flag on the container
to tell swift to version all objects in the container. The flag is the
``X-Versions-Location`` header on the container, and its value is the
container where the versions are stored.
When data is ``PUT`` into a versioned container (a container with the
versioning flag turned on), the existing data in the file is redirected to a
new object and the data in the ``PUT`` request is saved as the data for the
versioned object. The new object name (for the previous version) is
``<versions_container>/<object_name>/<timestamp>``, where the timestamp is
generated by converting the ``Last-Modified`` header value of the current
version to a unix timestamp.
A ``GET`` to a versioned object will return the current version of the object
without having to do any request redirects or metadata lookups.
Change-Id: I4fcd723145e02bbb2ec1d3ad356713f5dea43b8b
