After discussion https://review.openstack.org/#/c/129384/ moving
to the doc directory in swift repo.
This lets us eliminate the object-api repo along with all the <service>-
api repos and move content to audience-centric locations.
Change-Id: Ia0d9973847f7409a02dcc1a0e19400a3c3ecdf32
Noticed that slo and dlo middleware were placed before
tempauth, they should be placed after
DocImpact
Change-Id: Ia931e2280125d846f248b23e219aebad14c66210
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Instead of recommending to edit resetswift to replace "/dev/sdb1" with
"/srv/swift-disk", use an environment variable instead. This way I can
set SAIO_BLOCK_DEVICE=/srv/swift-disk in my .bashrc, and then when I'm
testing out changes to resetswift, I don't need to remember to edit
the modified script, nor do I end up submitting changes with the wrong
default in there.
The variable defaults to /dev/sdb1, so if you use the script unmodified
and don't set SAIO_BLOCK_DEVICE, nothing changes for you.
Change-Id: I741a8c91c2c54a4f32bc391cd794ef4206402753
Let's use the full project name to avoid confusion with the recently added
Swiftbrowser based on AngularJS.
Change-Id: Ib07338268a1593bc2882908b49c1fb4a130ff43d
Simply replacing a failed disk requires a very long time if the ring is not
changed, because all data will be replicated to a single new disk. This extends
the time to recover from missing replicas, and becomes even more important with
bigger disks.
This patch updates the doc to include a faster alternative by setting the weight
of a failed disk to 0. In this case the partitions from the failed disk are
distributed and replicated to the remaining disks in the cluster, and because
each disk gets only a fraction of the partitions it's also much faster.
Change-Id: I16617756359771ad89ca5d4690b58a014f481d9b
These docs currently say we target ubuntu 10.04 and an eventlet version
that our requirements file does not allow. Update these versions.
Change-Id: I052b6561f88ec90f865454e426032f1baf4586c0
The multi-node install was so horribly outdated it still
refered to ubuntu 10.04. The install guide at docs.openstack.org
has a usable swift installation guide - link to it until such
time as this page can be fixed.
Change-Id: I29fa334d9ffc9b63c8f31c664e7509b2f2577574
This change allows the user to use a "--no-overlap" parameter when
running the tool multiple times. It will increase the coverage by
whatever is specified in the dispersion_coverage field of the conf
file in a manner where existing container/objects are left in place
and no partition is populated more than once.
Related-Bug: #1233045
Change-Id: I139fed2f4c967ba18d073b7ecd1e946ed4da1271
Cleanup and add clarification to the documentation
for using Keystone auth.
Update to refer to auth_token middleware being
distributed as part of the keystomemiddelware project
rather than keystone.
Include capabilities (/info) in the list of reasons
why delay_auth_decision might need to be set in
auth_token middleware config.
Add description of the project_id:user_id format
for container ACLs and emphasize that ids rather than
names should be used since this patch has now merged:
https://review.openstack.org/#/c/86430
DocImpact
blueprint keystone-v3-support
Change-Id: Idda4a3dcf8240474f1d2d163016ca2d40ec2d589
Remove intersphinx from the docs build as it triggers network calls that
occasionally fail, and we don't really use intersphinx (links other
sphinx documents out on the internet)
This also removes the requirement for internet access during docs build.
This can cause docs jobs to fail if the project errors out on
warnings.
Change-Id: I71e941e2a639641a662a163c682eb86d51de42fb
Related-Bug: #1368910
The SAIO configs have no default bind_ip setting configured
which causes them to listen on all available IP addresses.
This can be dangerous on a test machine with public interfaces,
especially with the default passwords set. Its reasonable to
choose a more restrictive setup, especially in SAIO which uses
127.0.0.1 throughout ring-builder, example commands and the
probe tests.
Change-Id: I471c49705ce09e07ec7acc07ee42a1e220529b82
The keystoneauth middleware supports cross-tenant access
control using the syntax <tenant>:<user> in container ACLs,
where <tenant> and <user> may currently be either a unique
id or a name. As a result of the keystone v3 API introducing
domains, names are no longer globally unique and are only
unique within a domain. The use of unqualified tenant and
user names in this ACL syntax is therefore not 'safe' in a
keystone v3 environment.
This patch modifies keystoneauth to restrict cross-tenant
ACL matching to use only ids for accounts that are not in
the default domain. For backwards compatibility,
names will still be matched in ACLs when both the requesting
user and tenant are known to be in the default domain AND the
account's tenant is also in the default domain (the default
domain being the domain to which existing tenants are
migrated).
Accounts existing prior to this patch are assumed to be for
tenants in the default domain. New accounts created using a
v2 token scoped on the tenant are also assumed to be in the
default domain. New accounts created using a v3 token scoped
on the tenant will learn their domain membership from the
token info. New accounts created using any unscoped token,
(i.e. with a reselleradmin role) will have unknown domain
membership and therefore be assumed to NOT be in the default
domain.
Despite this provision for backwards compatibility, names
must no longer be used when setting new ACLs in any account,
including new accounts in the default domain.
This change obviously impacts users accustomed to specifying
cross-tenant ACLs in terms of names, and further work will be
necessary to restore those use cases. Some ideas are
discussed under the bug report. With that caveat, this patch
removes the reported vulnerability when using
swift/keystoneauth with a keystone v3 API.
Note: to observe the new 'restricted' behaviour you will need
to setup keystone user(s) and tenant(s) in a non-default domain
and set auth_version = v3.0 in the auth_token middleware config
section of proxy-server.conf. You may also benefit from the
keystone v3 enabled swiftclient patch under review here:
https://review.openstack.org/#/c/91788/
DocImpact
blueprint keystone-v3-support
Closes-Bug: #1299146
Change-Id: Ib32df093f7450f704127da77ff06b595f57615cb
Currently the theme used by the swift developer docs are out of
date, it should be using oslosphinx, to provide a similar look
and feel between all openstack related projects.
Change-Id: Id7c226cdc13c6c4f3b5082b1ef4dfe09966b21ec
Add an environment variable to enable the use of the in-memory object
server during in-process functional test runs.
It might be worth-while to just run under both object servers in-tree,
but this at least enables it, without having to figure out how to make
two test runs in two different environments.
DocImpact
Change-Id: Id76b008e1f273c639ae61550affddc32c5d7c419
Signed-off-by: Thiago da Silva <thiago@redhat.com>
auth_token middleware in python-keystoneclient is deprecated and has
been moved to the keystonemiddleware repo.
Change-Id: Ia04aa83348e0776cb3239cb5420ee1450a990d5b
Closes-Bug: #1342274
May not be obvious, but existing code will let you change the
disk_chunk_size just for the auditor so this just points that
out in the docs. In one short test I ran with a 4 node cluster
with 18GB of 4MB objects on it, changint he auditor chunk size
from the default of 64K to 1MB creased the auditor CPU time from
10% to 4%.
Also added test code to make sure this overridden value is
actually used and checked other auditWorker conf values as
well.
Change-Id: Ia12e1c6127877dc2124b60cd963cd0b6d5f3d6ef
It does not appear that, aside from the user-agent string, the strings
"obj-server", "obj-updater", or "obj-replicator" (or "obj-<anything>"*)
appear in the swift code base, aside from the directory containing the
object services code being named "obj".
Furthermore, the container, account, and proxy services construct their
user-agent string, as reported in the logs, using their full name. In
addition, this full name also shows up as the name of the process via
"ps" or "top", etc., which can make it easier for admins to match log
entries with other tools.
For consistency, we update the object services to use an "object-"
prefix rather than "obj-" in its user agent string.
* obj-etag does appear in a unit test, but not part of the regular
code.
Change-Id: I914fc189514207df2535731eda10cb4b3d30cc6c
The version of setup tools in Ubuntu 12.04 is too old and is causing
the SAIO instructions to fail when installing python-swiftclient.
The work around is to install python-swiftclient's dependencies
before running running:
python setup.py develop
This change adds a note to users of Ubuntu 12.04 to replace step 2
of "Getting the code" with:
cd $HOME/python-swiftclient; sudo pip install -r requirements.txt; \
python setup.py develop; cd -
Change-Id: I63f57bbf1f1158f8740f6137ad55ff49f12a316c
Closes-Bug: #1217288
We are soon going to put servers with a high ratio of disk to CPU
into production as object servers. One of our concerns with this
configuration is that the object auditor would take too long to
complete its audit cycle. Therefore we decided to parallelise
the auditor.
The auditor already uses fork(), so we decided to use the parallel
model from the replicator. Concurrency is set by the concurrency
parameter in the auditor stanza, which sets the number of parallel
checksum auditors. The actual number of parallel auditing processes
is concurrency + 1 if zero_byte_fps is non-zero.
Only one ZBF process is forked, and a new ZBF process is forked as
soon as the current ZBF process finishes. Thus the last process
running will always be a ZBF process.
Both forever and once modes are parallelised.
Each checksum auditor process submits a nested dictionary with keys
{'object_auditor_stats_ALL': {'diskn': {..}}} to dump_recon_cache
so that the object_auditor_stats_ALL dict in recon cache consists
of individual sub-dicts for each of the object disks on the server.
The recon cache is no different to before when the checksum auditor
is run in serial mode. When swift-recon is run, it sums the stats
for the individual disks.
DocImpact
Change-Id: I0ce3db57a43e482d4be351cc522fc9060af6e2d3
Fixed some grammar issues, formatting issues, clarrified some
wording now that its easier to read fully rendered.
Change-Id: Ie803dd1a84d50de7663a7099c32d81807701e188
Add overview and example information for using Storage Policies.
DocImpact
Implements: blueprint storage-policies
Change-Id: I6f11f7a1bdaa6f3defb3baa56a820050e5f727f1
There is a simple typo in the man page of proxy-server.conf,
"client_timeout" is written as "client_timeoutt".
This commit fixes it.
Closes-Bug: #1326237
Change-Id: I98777f523906e4ed625de8f20a96979ea627aa1f
Log lines can get quite large, as we previously noticed with rsync error
log lines. We added a setting to cap those, but it really looks like we
should have just done this overall limit. We noticed the issue when we
switched to UDP syslogging and it would occasionally blow past the 16436
lo MTU! This causes Python's logging code to get an error and hilarity
ensues.
Change-Id: I44bdbe68babd58da58c14360379e8fef8a6b75f7
