646 Commits

Author SHA1 Message Date
Jenkins
d6467d3385 Merge "Add multiple reseller prefixes and composite tokens" 2015-02-24 16:12:01 +00:00
Donagh McCabe
89397c5b67 Add multiple reseller prefixes and composite tokens
This change is in support of Composite Tokens and Service Accounts
(see http://specs.openstack.org/openstack/swift-specs/specs/in_progress/
service_token.html)

During coding, minor changes were made compared to the original
specification. See https://review.openstack.org/138771 for these changes.

DocImpact

Change-Id: I6072b4efb3a479a8e0cc2d9c11ffda5764b55e30
2015-02-23 15:57:20 +00:00
Alistair Coles
4ca08cc395 Update guest VM OS recommendation in SAIO doc
The target development platform has changed to Ubuntu 14.04 [1].
This patch makes the suggested SAIO platform the same.

Also, remove pointer to wiki page for other platform install
instructions that either redirects back to this SAIO doc or
to another wike page and then a dead link.

[1] I0a96bcf692bb240f3ab5aab7fefd294a07735a83

DocImpact

Change-Id: I9f96104b5437c1f1f28f924c048ef83cf03338f4
2015-02-19 17:46:10 +00:00
John Dickinson
949804eda4 update the getting started doc
Change-Id: I0a96bcf692bb240f3ab5aab7fefd294a07735a83
2015-02-17 13:24:33 -08:00
John Dickinson
7acc291129 added swift_source to ratelimit info calls
Change-Id: I2b4ccb809c0f4505098e261f546f53c27440dd43
2015-02-15 17:14:31 -08:00
Jenkins
f2e2326b5f Merge "Update TempURL docs to include containers" 2015-02-12 22:32:07 +00:00
Richard Hawkins
f5b533fb2f Update TempURL docs to include containers
Change-Id: Ifb7c26b23ec81e3bde96b6e3bad0e950c0ca9408
2015-02-12 11:27:31 -06:00
Clay Gerrard
ca90be414e Add developer tools section to associated projects
vagrant-swift-all-in-one is being used and maintained by a number of swift
developers, it has an open source license.

The ansible playbook project serves a similar goal but it's based on a Fedora
distribution and includes Swift-on-File support.

Drive-by fix for the Swift-on-File link which has migrated to stackforge.

Change-Id: Id7478d58adcead57cf56ac4e1d05c6556c8c9b7b
2015-02-11 16:43:44 -08:00
Donagh McCabe
843ce7e301 API document update for container-level tempurl/formpost
Tempurl and Formpost now support secret keys at the container
level [1]. Adds these headers:

    X-Container-Meta-Temp-Url-Key
    X-Container-Meta-Temp-Url-Key-2

DocImpact -- because headers need to be added to [2]

[1] https://review.openstack.org/#/c/154293
[2] http://developer.openstack.org/api-ref-objectstorage-v1.html

Change-Id: I157cd4a540dcc4a9b45bc3d806df152ab56e2354
2015-02-11 12:02:38 +00:00
Jenkins
0186194e96 Merge "Output logs of policy index" 2015-02-10 22:21:13 +00:00
Jenkins
bc7c496f71 Merge "Allow hostnames for nodes in Rings" 2015-02-10 04:32:38 +00:00
John Dickinson
b45b83fb00 Correct the config default for delay_auth_decision
Updated proxy-server.conf-sample with the correct default. Also
updated the note on the overview-auth doc page.

Change-Id: I5cd62a7a118a28f7b58f47b8d8d4d963f6bc7347
2015-02-05 11:52:41 -08:00
Jenkins
e7e49f6b37 Merge "Update auth_token section in documentation" 2015-02-05 15:30:51 +00:00
Nicolas Trangez
b5f8c594f9 Add swift-scality-backend to associated projects
Change-Id: I7fd56c7cf5b7634224b8a2876258cf1f6be447f1
2015-02-04 16:54:46 +01:00
John Dickinson
2eba998a7c added swift-ui browser to associated projects
Change-Id: I23abd014b21d1a968fe7352f3915f02e3d4d47cf
2015-02-03 22:12:03 -08:00
Alistair Coles
5a0f8f1402 Update auth_token section in documentation
Bring docs in line with changes to auth_token config
defaults made in I7076fa03ab531cbb1114918f75113620b65590dc

Change-Id: Ia21685ebd1f3ed7bdba9de2ebac9fdcce8495949
2015-02-03 11:57:06 +00:00
Hisashi Osanai
efb39a5665 Allow hostnames for nodes in Rings
This change modifies the swift-ring-builder and introduces new format
of sub-commands (search, list_parts, set_weight, set_info and remove)
in addition to add sub-command so that hostnames can be used in place
of an ip-address for the sub-commands.
The account reaper, container synchronizer, and replicators were also
updated so that they still have a way to identify a particular device
as being "local".

Previously this was Change-Id:
Ie471902413002872fc6755bacd36af3b9c613b74

Change-Id: Ieff583ffb932133e3820744a3f8f9f491686b08d
Co-Authored-By: Alex Pecoraro <alex.pecoraro@emc.com>
Implements: blueprint allow-hostnames-for-nodes-in-rings
2015-02-02 05:06:03 +09:00
Jiangmiao Gao
77e9267916 Fix typing mistake for doc
Change-Id: If3d4e4d44676ef987f584db8cb462ecaed03a019
2015-01-28 22:41:41 +08:00
Jenkins
ecf4d057b3 Merge "Add dispersion command to swift-ring-builder" 2015-01-27 16:12:10 +00:00
Daisuke Morita
afdbf73f12 Output logs of policy index
To make it easier for Swift operators to specify problematic devices,
a policy index will be recorded in log files of proxy and storage servers
for each user request which is related to storage policy.

This patch simply adds 'storage_policy_index' field in a log format.
If there is no specified policy index, '-' is output in this field.

Extra fix: Doc about the log line of storage nodes now properly reflects
           'server_pid' field.

DocImpact

Change-Id: I7286ae85bcbcec73b5377dc115cbdb0f57d1b025
Implements: blueprint logging-policy-number
2015-01-23 10:48:38 +09:00
Prashanth Pai
a0977ab8ca dlo: Update doc about manifest containing data
Current behavior:
* If data/body is present in manifest file PUT request, the data/body gets
  saved onto disk, just like for a normal object.
* Generally, this data in manifest file is never served on a GET response.
  However, when the manifest object path itself is part of prefix, GET
  response would contain data present in manifest file as well.
* The query param multipart-manifest=get meant to retrieve SLO manifest
  also works in case of DLO manifest. Hence a COPY request with the
  multipart-manifest=get query param would actually copy DLO manifest.

How things should have been:
* The DLO manifest object is supposed to have no content and only have
  X-Object-Manifest metadata header.
* Query param multipart-manifest=get is SLO specific and shouldn't have
  any role in DLO.

This change intends to only document current behaviour and not change it,
assuming there are users who have previously saved some content in DLO
manifest file and/or have been using multipart-manifest=get to fetch
and/or COPY the DLO manifest file with it's content.

Change-Id: I0f6e175ad7752169ecf94df949336e0665928df7
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-01-16 10:24:14 +05:30
Jenkins
0e660fade3 Merge "Change black/white-listing to use sysmeta." 2015-01-10 00:21:41 +00:00
Jenkins
64b28d615a Merge "Add notion of overload to swift-ring-builder" 2015-01-09 20:59:04 +00:00
David Goetz
172a9b369f Change black/white-listing to use sysmeta.
The way we do this now involves a conf change and a proxy
reload which is a pain. You can now just set these:

X-Account-Sysmeta-Global-Write-Ratelimit: WHITELIST

or

X-Account-Sysmeta-Global-Write-Ratelimit: BLACKLIST

NOTE:
The existing proxy config settings: account_whitelist
and account_blacklist will continue to work.

Change-Id: I532663f1d2c75d03170c5fdb9b330416822fbc88
2015-01-09 08:35:50 -08:00
Clay Gerrard
a8bd2f737c Add dispersion command to swift-ring-builder
Output a dispersion report that shows how many parts have each replica count
at each tier along with some additional context.  Also the max_dispersion is a
good canary for what a reasonable overload might be.

Also display a warning on rebalance if the ring's dispersion is sub-optimal.

The primitive form of the dispersion graph is cached on the builder, but the
dispersion command will build it on the fly if you have a ring that was last
rebalanced before the change.

Also add --force option to rebalance to make it write a ring even if less than
1% of parts moved.

Try to clarify some dispersion and balance a little bit in the ring section of
the architectural overview.

Co-Authored-By: Christian Schwede <christian.schwede@enovance.com>
Co-Authored-By: Darrell Bishop <darrell@swiftstack.com>

Change-Id: I7696df25d092fac56588080722e0a4167ed2c824
2015-01-08 18:40:27 -08:00
Samuel Merritt
bcf26f5209 Add notion of overload to swift-ring-builder
The ring builder's placement algorithm has two goals: first, to ensure
that each partition has its replicas as far apart as possible, and
second, to ensure that partitions are fairly distributed according to
device weight. In many cases, it succeeds in both, but sometimes those
goals conflict. When that happens, operators may want to relax the
rules a little bit in order to reach a compromise solution.

Imagine a cluster of 3 nodes (A, B, C), each with 20 identical disks,
and using 3 replicas. The ring builder will place 1 replica of each
partition on each node, as you'd expect.

Now imagine that one disk fails in node C and is removed from the
ring. The operator would probably be okay with remaining at 1 replica
per node (unless their disks are really close to full), but to
accomplish that, they have to multiply the weights of the other disks
in node C by 20/19 to make C's total weight stay the same. Otherwise,
the ring builder will move partitions around such that some partitions
have replicas only on nodes A and B.

If 14 more disks failed in node C, the operator would probably be okay
with some data not living on C, as a 4x increase in storage
requirements is likely to fill disks.

This commit introduces the notion of "overload": how much extra
partition space can be placed on each disk *over* what the weight
dictates.

For example, an overload of 0.1 means that a device can take up to 10%
more partitions than its weight would imply in order to make the
replica dispersion better.

Overload only has an effect when replica-dispersion and device weights
come into conflict.

The overload is a single floating-point value for the builder
file. Existing builders get an overload of 0.0, so there will be no
behavior change on existing rings.

In the example above, imagine the operator sets an overload of 0.112
on his rings. If node C loses a drive, each other drive can take on up
to 11.2% more data. Splitting the dead drive's partitions among the
remaining 19 results in a 5.26% increase, so everything that was on
node C stays on node C. If another disk dies, then we're up to an
11.1% increase, and so everything still stays on node C. If a third
disk dies, then we've reached the limits of the overload, so some
partitions will begin to reside solely on nodes A and B.

DocImpact

Change-Id: I3593a1defcd63b6ed8eae9c1c66b9d3428b33864
2015-01-07 14:16:08 -08:00
Dhriti Shikhar
5b99ba1c8a Substituted object storage paragraph with simple definition
Change-Id: I32711fd10dfb1b84cbea9d05638b9ee002588104
Closes-bug: #1373925
2015-01-07 00:15:36 +05:30
Jenkins
18acb03f43 Merge "Fix typo in apache_deployment doc" 2014-11-25 04:14:15 +00:00
Jenkins
351126f8eb Merge "Fix typos in overview_large_objects and versioning doc" 2014-11-25 04:14:07 +00:00
Shilla Saebi
98de48d898 Fix typo in apache_deployment doc
Change-Id: I42d76f544290dbda62633de90608d41caadac084
2014-11-24 15:40:21 +00:00
Shilla Saebi
a1872b0498 Fix 2 typos in admin_guide file
Change-Id: Ibf1e5dbf6ff4747c7f23f6638321ab41bba3021b
2014-11-24 15:38:25 +00:00
Shilla Saebi
0dc4b0a7b7 Fix typos in overview_large_objects and versioning doc
Change-Id: I1a919ad1b0298d5817f9eb2caf5e3bd7b3243c2c
2014-11-24 15:35:06 +00:00
Jenkins
a590e84a53 Merge "Adds v1 API documentation to doc/source/api" 2014-11-19 23:39:06 +00:00
Anne Gentle
6f9ca6122e Adds v1 API documentation to doc/source/api
After discussion https://review.openstack.org/#/c/129384/ moving
to the doc directory in swift repo.

This lets us eliminate the object-api repo along with all the <service>-
api repos and move content to audience-centric locations.

Change-Id: Ia0d9973847f7409a02dcc1a0e19400a3c3ecdf32
2014-11-17 16:19:07 -06:00
Thiago da Silva
11a72a4a50 move slo, dlo after tempauth in pipeline
Noticed that slo and dlo middleware were placed before
tempauth, they should be placed after

DocImpact

Change-Id: Ia931e2280125d846f248b23e219aebad14c66210
Signed-off-by: Thiago da Silva <thiago@redhat.com>
2014-11-17 16:41:09 +00:00
Jenkins
eb528f75d2 Merge "Make resetswift customizable via environment" 2014-11-11 16:53:06 +00:00
Samuel Merritt
466403723c Make resetswift customizable via environment
Instead of recommending to edit resetswift to replace "/dev/sdb1" with
"/srv/swift-disk", use an environment variable instead. This way I can
set SAIO_BLOCK_DEVICE=/srv/swift-disk in my .bashrc, and then when I'm
testing out changes to resetswift, I don't need to remember to edit
the modified script, nor do I end up submitting changes with the wrong
default in there.

The variable defaults to /dev/sdb1, so if you use the script unmodified
and don't set SAIO_BLOCK_DEVICE, nothing changes for you.

Change-Id: I741a8c91c2c54a4f32bc391cd794ef4206402753
2014-11-10 15:07:44 -08:00
Jenkins
fc7af8c893 Merge "Rename Swiftbrowser in associated projects" 2014-11-07 02:04:42 +00:00
Christian Schwede
ecc946b4ff Rename Swiftbrowser in associated projects
Let's use the full project name to avoid confusion with the recently added
Swiftbrowser based on AngularJS.

Change-Id: Ib07338268a1593bc2882908b49c1fb4a130ff43d
2014-11-06 15:44:29 +01:00
Martin Geisler
dff981a03e Add Swift Browser as an associated project
This is a JavaScript based browser for Swift.

Change-Id: I2e304d4a0623c715f8712a358fef5067abc8935b
2014-11-06 14:52:04 +01:00
Christian Schwede
83030b921d Update admin guide on handling drive failures
Simply replacing a failed disk requires a very long time if the ring is not
changed, because all data will be replicated to a single new disk. This extends
the time to recover from missing replicas, and becomes even more important with
bigger disks.

This patch updates the doc to include a faster alternative by setting the weight
of a failed disk to 0.  In this case the partitions from the failed disk are
distributed and replicated to the remaining disks in the cluster, and because
each disk gets only a fraction of the partitions it's also much faster.

Change-Id: I16617756359771ad89ca5d4690b58a014f481d9b
2014-10-29 10:39:01 +00:00
Samuel Merritt
dd3564a587 Mention Python 2.7 in Getting Started guide
Change-Id: I2992d1227f294d4cb1c6a8676137df44e231bd80
2014-10-24 15:34:45 -07:00
Gregory Haynes
047215afeb Update Getting Started requirements versions
These docs currently say we target ubuntu 10.04 and an eventlet version
that our requirements file does not allow. Update these versions.

Change-Id: I052b6561f88ec90f865454e426032f1baf4586c0
2014-10-24 15:23:33 -07:00
Christian Schwede
6d5206325a Add a reference to the OpenStack security guide
Change-Id: I85b9591a17572eb57f315d7b8e6741455aef5817
2014-10-07 21:54:17 +02:00
Tom Fifield
c207c9b9da Refer multi node install to docs.openstack.org
The multi-node install was so horribly outdated it still
refered to ubuntu 10.04. The install guide at docs.openstack.org
has a usable swift installation guide - link to it until such
time as this page can be fixed.

Change-Id: I29fa334d9ffc9b63c8f31c664e7509b2f2577574
2014-10-07 09:07:43 -07:00
Jenkins
7d66b2f2f0 Merge "Added instructions to create a label or UUID to the XFS volume and mount using it." 2014-09-25 18:58:22 +00:00
Mahati Chamarthy
b68258a322 Added instructions to create a label or UUID to the XFS volume and mount using it.
Change-Id: Idcaf16a278d6c34770af9b1f17d69bdd94fb86b7
2014-09-23 14:55:16 +05:30
Jenkins
71a35b5543 Merge "Update documentation for using keystone auth" 2014-09-22 15:44:29 +00:00
Jenkins
607e2ed2e3 Merge "Add "--no-overlap" option to swift-dispersion populate" 2014-09-22 15:44:14 +00:00
Lorcan
4faf170270 Add "--no-overlap" option to swift-dispersion populate
This change allows the user to use a "--no-overlap" parameter when
running the tool multiple times. It will increase the coverage by
whatever is specified in the dispersion_coverage field of the conf
file in a manner where existing container/objects are left in place
and no partition is populated more than once.

Related-Bug: #1233045

Change-Id: I139fed2f4c967ba18d073b7ecd1e946ed4da1271
2014-09-19 10:44:30 +01:00