Fix missing *-replicator conf sections in deployment guide
The doc for these sections was missing because of an rst error - the source is there in rst file but didn't make it into the html output. Add doc for per_diff and max_diffs in account and container doc sections. Also, fix a bunch of other sphinx build errors and most of the warnings. Change-Id: If9ed2619b2f92c6c65a94f41d8819db8726d3893
This commit is contained in:
parent
4a13dcc4a8
commit
1a2b54fc0a
@ -177,7 +177,7 @@ Logging level. The default is INFO.
|
||||
.IP \fBlog_address\fR
|
||||
Logging address. The default is /dev/log.
|
||||
.IP \fBper_diff\fR
|
||||
The default is 1000.
|
||||
Maximum number of database rows that will be sync'd in a single HTTP replication request. The default is 1000.
|
||||
.IP \fBmax_diffs\fR
|
||||
This caps how long the replicator will spend trying to sync a given database per pass so the other databases don't get starved. The default is 100.
|
||||
.IP \fBconcurrency\fR
|
||||
|
@ -183,7 +183,7 @@ Logging level. The default is INFO.
|
||||
.IP \fBlog_address\fR
|
||||
Logging address. The default is /dev/log.
|
||||
.IP \fBer_diff\fR
|
||||
The default is 1000.
|
||||
Maximum number of database rows that will be sync'd in a single HTTP replication request. The default is 1000.
|
||||
.IP \fBmax_diffs\fR
|
||||
This caps how long the replicator will spend trying to sync a given database per pass so the other databases don't get starved. The default is 100.
|
||||
.IP \fBconcurrency\fR
|
||||
|
@ -1,5 +1,3 @@
|
||||
.. _formpost:
|
||||
|
||||
====================
|
||||
Form POST middleware
|
||||
====================
|
||||
|
@ -293,10 +293,12 @@ a manifest object but a normal object with content same as what you would
|
||||
get on a **GET** request to original manifest object.
|
||||
|
||||
To duplicate a manifest object:
|
||||
|
||||
* Use the **GET** operation to read the value of ``X-Object-Manifest`` and
|
||||
use this value in the ``X-Object-Manifest`` request header in a **PUT**
|
||||
operation.
|
||||
* Alternatively, you can include *``?multipart-manifest=get``* query
|
||||
string in the **COPY** request.
|
||||
|
||||
This creates a new manifest object that shares the same set of segment
|
||||
objects as the original manifest object.
|
||||
|
@ -559,53 +559,60 @@ replication_failure_ratio 1.0 If the value of failures /
|
||||
|
||||
[object-replicator]
|
||||
|
||||
================== ================= =======================================
|
||||
Option Default Description
|
||||
------------------ ----------------- ---------------------------------------
|
||||
log_name object-replicator Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Logging level
|
||||
daemonize yes Whether or not to run replication as a
|
||||
daemon
|
||||
interval 30 Time in seconds to wait between
|
||||
replication passes
|
||||
concurrency 1 Number of replication workers to spawn
|
||||
timeout 5 Timeout value sent to rsync --timeout
|
||||
and --contimeout options
|
||||
stats_interval 3600 Interval in seconds between logging
|
||||
replication statistics
|
||||
reclaim_age 604800 Time elapsed in seconds before an
|
||||
object can be reclaimed
|
||||
handoffs_first false If set to True, partitions that are
|
||||
not supposed to be on the node will be
|
||||
replicated first. The default setting
|
||||
should not be changed, except for
|
||||
extreme situations.
|
||||
handoff_delete auto By default handoff partitions will be
|
||||
removed when it has successfully
|
||||
replicated to all the canonical nodes.
|
||||
If set to an integer n, it will remove
|
||||
the partition if it is successfully
|
||||
replicated to n nodes. The default
|
||||
setting should not be changed, except
|
||||
for extreme situations.
|
||||
node_timeout DEFAULT or 10 Request timeout to external services.
|
||||
This uses what's set here, or what's set
|
||||
in the DEFAULT section, or 10 (though
|
||||
other sections use 3 as the final
|
||||
default).
|
||||
rsync_module {replication_ip}::object
|
||||
Format of the rsync module where the
|
||||
replicator will send data. The
|
||||
configuration value can include some
|
||||
variables that will be extracted from
|
||||
the ring. Variables must follow the
|
||||
format {NAME} where NAME is one of:
|
||||
ip, port, replication_ip,
|
||||
replication_port, region, zone, device,
|
||||
meta. See etc/rsyncd.conf-sample for
|
||||
some examples.
|
||||
================== ================= =======================================
|
||||
================== ======================== ================================
|
||||
Option Default Description
|
||||
------------------ ------------------------ --------------------------------
|
||||
log_name object-replicator Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Logging level
|
||||
daemonize yes Whether or not to run replication
|
||||
as a daemon
|
||||
interval 30 Time in seconds to wait between
|
||||
replication passes
|
||||
concurrency 1 Number of replication workers to
|
||||
spawn
|
||||
timeout 5 Timeout value sent to rsync
|
||||
--timeout and --contimeout
|
||||
options
|
||||
stats_interval 3600 Interval in seconds between
|
||||
logging replication statistics
|
||||
reclaim_age 604800 Time elapsed in seconds before an
|
||||
object can be reclaimed
|
||||
handoffs_first false If set to True, partitions that
|
||||
are not supposed to be on the
|
||||
node will be replicated first.
|
||||
The default setting should not be
|
||||
changed, except for extreme
|
||||
situations.
|
||||
handoff_delete auto By default handoff partitions
|
||||
will be removed when it has
|
||||
successfully replicated to all
|
||||
the canonical nodes. If set to an
|
||||
integer n, it will remove the
|
||||
partition if it is successfully
|
||||
replicated to n nodes. The
|
||||
default setting should not be
|
||||
changed, except for extreme
|
||||
situations.
|
||||
node_timeout DEFAULT or 10 Request timeout to external
|
||||
services. This uses what's set
|
||||
here, or what's set in the
|
||||
DEFAULT section, or 10 (though
|
||||
other sections use 3 as the final
|
||||
default).
|
||||
rsync_module {replication_ip}::object Format of the rsync module where
|
||||
the replicator will send data.
|
||||
The configuration value can
|
||||
include some variables that will
|
||||
be extracted from the ring.
|
||||
Variables must follow the format
|
||||
{NAME} where NAME is one of: ip,
|
||||
port, replication_ip,
|
||||
replication_port, region, zone,
|
||||
device, meta. See
|
||||
etc/rsyncd.conf-sample for some
|
||||
examples.
|
||||
================== ======================== ================================
|
||||
|
||||
[object-updater]
|
||||
|
||||
@ -718,35 +725,53 @@ allow_versions false Enable/Disable object versioning feature
|
||||
|
||||
[container-replicator]
|
||||
|
||||
================== ==================== ====================================
|
||||
Option Default Description
|
||||
------------------ -------------------- ------------------------------------
|
||||
log_name container-replicator Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Logging level
|
||||
per_diff 1000
|
||||
concurrency 8 Number of replication workers to
|
||||
spawn
|
||||
interval 30 Time in seconds to wait between
|
||||
replication passes
|
||||
node_timeout 10 Request timeout to external services
|
||||
conn_timeout 0.5 Connection timeout to external
|
||||
services
|
||||
reclaim_age 604800 Time elapsed in seconds before a
|
||||
container can be reclaimed
|
||||
rsync_module {replication_ip}::container
|
||||
Format of the rsync module where the
|
||||
replicator will send data. The
|
||||
configuration value can include some
|
||||
variables that will be extracted from
|
||||
the ring. Variables must follow the
|
||||
format {NAME} where NAME is one of:
|
||||
ip, port, replication_ip,
|
||||
replication_port, region, zone,
|
||||
device, meta. See
|
||||
etc/rsyncd.conf-sample for some
|
||||
examples.
|
||||
================== ==================== ====================================
|
||||
================== =========================== =============================
|
||||
Option Default Description
|
||||
------------------ --------------------------- -----------------------------
|
||||
log_name container-replicator Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Logging level
|
||||
per_diff 1000 Maximum number of database
|
||||
rows that will be sync'd in a
|
||||
single HTTP replication
|
||||
request. Databases with less
|
||||
than or equal to this number
|
||||
of differing rows will always
|
||||
be sync'd using an HTTP
|
||||
replication request rather
|
||||
than using rsync.
|
||||
max_diffs 100 Maximum number of HTTP
|
||||
replication requests attempted
|
||||
on each replication pass for
|
||||
any one container. This caps
|
||||
how long the replicator will
|
||||
spend trying to sync a given
|
||||
database per pass so the other
|
||||
databases don't get starved.
|
||||
concurrency 8 Number of replication workers
|
||||
to spawn
|
||||
interval 30 Time in seconds to wait
|
||||
between replication passes
|
||||
node_timeout 10 Request timeout to external
|
||||
services
|
||||
conn_timeout 0.5 Connection timeout to external
|
||||
services
|
||||
reclaim_age 604800 Time elapsed in seconds before
|
||||
a container can be reclaimed
|
||||
rsync_module {replication_ip}::container Format of the rsync module
|
||||
where the replicator will send
|
||||
data. The configuration value
|
||||
can include some variables
|
||||
that will be extracted from
|
||||
the ring. Variables must
|
||||
follow the format {NAME} where
|
||||
NAME is one of: ip, port,
|
||||
replication_ip,
|
||||
replication_port, region,
|
||||
zone, device, meta. See
|
||||
etc/rsyncd.conf-sample for
|
||||
some examples.
|
||||
================== =========================== =============================
|
||||
|
||||
[container-updater]
|
||||
|
||||
@ -859,33 +884,51 @@ set log_level INFO Logging level
|
||||
|
||||
[account-replicator]
|
||||
|
||||
================== ================== ======================================
|
||||
Option Default Description
|
||||
------------------ ------------------ --------------------------------------
|
||||
log_name account-replicator Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Logging level
|
||||
per_diff 1000
|
||||
concurrency 8 Number of replication workers to spawn
|
||||
interval 30 Time in seconds to wait between
|
||||
replication passes
|
||||
node_timeout 10 Request timeout to external services
|
||||
conn_timeout 0.5 Connection timeout to external services
|
||||
reclaim_age 604800 Time elapsed in seconds before an
|
||||
account can be reclaimed
|
||||
rsync_module {replication_ip}::account
|
||||
Format of the rsync module where the
|
||||
replicator will send data. The
|
||||
configuration value can include some
|
||||
variables that will be extracted from
|
||||
the ring. Variables must follow the
|
||||
format {NAME} where NAME is one of:
|
||||
ip, port, replication_ip,
|
||||
replication_port, region, zone,
|
||||
device, meta. See
|
||||
etc/rsyncd.conf-sample for some
|
||||
examples.
|
||||
================== ================== ======================================
|
||||
================== ========================= ===============================
|
||||
Option Default Description
|
||||
------------------ ------------------------- -------------------------------
|
||||
log_name account-replicator Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Logging level
|
||||
per_diff 1000 Maximum number of database rows
|
||||
that will be sync'd in a single
|
||||
HTTP replication request.
|
||||
Databases with less than or
|
||||
equal to this number of
|
||||
differing rows will always be
|
||||
sync'd using an HTTP replication
|
||||
request rather than using rsync.
|
||||
max_diffs 100 Maximum number of HTTP
|
||||
replication requests attempted
|
||||
on each replication pass for any
|
||||
one container. This caps how
|
||||
long the replicator will spend
|
||||
trying to sync a given database
|
||||
per pass so the other databases
|
||||
don't get starved.
|
||||
concurrency 8 Number of replication workers
|
||||
to spawn
|
||||
interval 30 Time in seconds to wait between
|
||||
replication passes
|
||||
node_timeout 10 Request timeout to external
|
||||
services
|
||||
conn_timeout 0.5 Connection timeout to external
|
||||
services
|
||||
reclaim_age 604800 Time elapsed in seconds before
|
||||
an account can be reclaimed
|
||||
rsync_module {replication_ip}::account Format of the rsync module where
|
||||
the replicator will send data.
|
||||
The configuration value can
|
||||
include some variables that will
|
||||
be extracted from the ring.
|
||||
Variables must follow the format
|
||||
{NAME} where NAME is one of: ip,
|
||||
port, replication_ip,
|
||||
replication_port, region, zone,
|
||||
device, meta. See
|
||||
etc/rsyncd.conf-sample for some
|
||||
examples.
|
||||
================== ========================= ===============================
|
||||
|
||||
[account-auditor]
|
||||
|
||||
|
@ -51,16 +51,16 @@ To execute the unit tests:
|
||||
|
||||
.. note::
|
||||
As of tox version 2.0.0, most environment variables are not automatically
|
||||
passed to the test environment. Swift's tox.ini overrides this default
|
||||
behavior so that variable names matching SWIFT_* and *_proxy will be passed,
|
||||
but you may need to run tox --recreate for this to take effect after
|
||||
upgrading from tox<2.0.0.
|
||||
passed to the test environment. Swift's `tox.ini` overrides this default
|
||||
behavior so that variable names matching ``SWIFT_*`` and ``*_proxy`` will be
|
||||
passed, but you may need to run `tox --recreate` for this to take effect
|
||||
after upgrading from tox<2.0.0.
|
||||
|
||||
Conversely, if you do not want those environment variables to be passed to
|
||||
the test environment then you will need to unset them before calling tox.
|
||||
|
||||
Also, if you ever encounter DistributionNotFound, try to use `tox --recreate`
|
||||
or remove the .tox directory to force tox to recreate the dependency list.
|
||||
or remove the `.tox` directory to force tox to recreate the dependency list.
|
||||
|
||||
The functional tests may be executed against a :doc:`development_saio` or
|
||||
other running Swift cluster using the command:
|
||||
|
@ -254,9 +254,11 @@ This configuration works as follows:
|
||||
``admin`` or ``swiftoperator`` role(s). When validated, the service token
|
||||
gives the ``service`` role.
|
||||
* Swift interprets the above configuration as follows:
|
||||
|
||||
* Did the user token provide one of the roles listed in operator_roles?
|
||||
* Did the service token have the ``service`` role as described by the
|
||||
``SERVICE_service_roles`` options.
|
||||
|
||||
* If both conditions are met, the request is granted. Otherwise, Swift
|
||||
rejects the request.
|
||||
|
||||
|
@ -171,6 +171,7 @@ The sequence of events and actions are as follows:
|
||||
a copy of the <user-token>. In the X-Service-Token header, place your
|
||||
Service's token. If you use python-swiftclient you can achieve this
|
||||
by:
|
||||
|
||||
* Putting the URL in the ``preauthurl`` parameter
|
||||
* Putting the <user-token> in ``preauthtoken`` paramater
|
||||
* Adding the X-Service-Token to the ``headers`` parameter
|
||||
@ -251,7 +252,7 @@ However, if one Service is compromised, that Service can access
|
||||
data created by another Service. To prevent this, multiple Service Prefixes may
|
||||
be used. This also requires that the operator configure multiple service
|
||||
roles. For example, in a system that has Glance and Cinder, the following
|
||||
Swift configuration could be used:
|
||||
Swift configuration could be used::
|
||||
|
||||
[keystoneauth]
|
||||
reseller_prefix = AUTH_, IMAGE_, BLOCK_
|
||||
|
@ -90,8 +90,19 @@ use = egg:swift#recon
|
||||
# log_level = INFO
|
||||
# log_address = /dev/log
|
||||
#
|
||||
# Maximum number of database rows that will be sync'd in a single HTTP
|
||||
# replication request. Databases with less than or equal to this number of
|
||||
# differing rows will always be sync'd using an HTTP replication request rather
|
||||
# than using rsync.
|
||||
# per_diff = 1000
|
||||
#
|
||||
# Maximum number of HTTP replication requests attempted on each replication
|
||||
# pass for any one container. This caps how long the replicator will spend
|
||||
# trying to sync a given database per pass so the other databases don't get
|
||||
# starved.
|
||||
# max_diffs = 100
|
||||
#
|
||||
# Number of replication workers to spawn.
|
||||
# concurrency = 8
|
||||
#
|
||||
# Time in seconds to wait between replication passes
|
||||
|
@ -99,8 +99,19 @@ use = egg:swift#recon
|
||||
# log_level = INFO
|
||||
# log_address = /dev/log
|
||||
#
|
||||
# Maximum number of database rows that will be sync'd in a single HTTP
|
||||
# replication request. Databases with less than or equal to this number of
|
||||
# differing rows will always be sync'd using an HTTP replication request rather
|
||||
# than using rsync.
|
||||
# per_diff = 1000
|
||||
#
|
||||
# Maximum number of HTTP replication requests attempted on each replication
|
||||
# pass for any one container. This caps how long the replicator will spend
|
||||
# trying to sync a given database per pass so the other databases don't get
|
||||
# starved.
|
||||
# max_diffs = 100
|
||||
#
|
||||
# Number of replication workers to spawn.
|
||||
# concurrency = 8
|
||||
#
|
||||
# Time in seconds to wait between replication passes
|
||||
|
@ -71,16 +71,16 @@ class TempAuth(object):
|
||||
|
||||
The reseller prefix specifies which parts of the account namespace this
|
||||
middleware is responsible for managing authentication and authorization.
|
||||
By default, the prefix is AUTH so accounts and tokens are prefixed
|
||||
by AUTH_. When a request's token and/or path start with AUTH_, this
|
||||
By default, the prefix is 'AUTH' so accounts and tokens are prefixed
|
||||
by 'AUTH\_'. When a request's token and/or path start with 'AUTH\_', this
|
||||
middleware knows it is responsible.
|
||||
|
||||
We allow the reseller prefix to be a list. In tempauth, the first item
|
||||
in the list is used as the prefix for tokens and user groups. The
|
||||
other prefixes provide alternate accounts that user's can access. For
|
||||
example if the reseller prefix list is 'AUTH, OTHER', a user with
|
||||
admin access to AUTH_account also has admin access to
|
||||
OTHER_account.
|
||||
admin access to 'AUTH_account' also has admin access to
|
||||
'OTHER_account'.
|
||||
|
||||
Required Group:
|
||||
|
||||
@ -98,7 +98,7 @@ class TempAuth(object):
|
||||
is not processed.
|
||||
|
||||
The X-Service-Token is useful when combined with multiple reseller prefix
|
||||
items. In the following configuration, accounts prefixed SERVICE_
|
||||
items. In the following configuration, accounts prefixed 'SERVICE\_'
|
||||
are only accessible if X-Auth-Token is from the end-user and
|
||||
X-Service-Token is from the ``glance`` user::
|
||||
|
||||
|
@ -1739,7 +1739,7 @@ def expand_ipv6(address):
|
||||
def whataremyips(bind_ip=None):
|
||||
"""
|
||||
Get "our" IP addresses ("us" being the set of services configured by
|
||||
one *.conf file). If our REST listens on a specific address, return it.
|
||||
one `*.conf` file). If our REST listens on a specific address, return it.
|
||||
Otherwise, if listen on '0.0.0.0' or '::' return all addresses, including
|
||||
the loopback.
|
||||
|
||||
@ -3078,15 +3078,15 @@ class ThreadPool(object):
|
||||
|
||||
def run_in_thread(self, func, *args, **kwargs):
|
||||
"""
|
||||
Runs func(*args, **kwargs) in a thread. Blocks the current greenlet
|
||||
Runs ``func(*args, **kwargs)`` in a thread. Blocks the current greenlet
|
||||
until results are available.
|
||||
|
||||
Exceptions thrown will be reraised in the calling thread.
|
||||
|
||||
If the threadpool was initialized with nthreads=0, it invokes
|
||||
func(*args, **kwargs) directly, followed by eventlet.sleep() to ensure
|
||||
the eventlet hub has a chance to execute. It is more likely the hub
|
||||
will be invoked when queuing operations to an external thread.
|
||||
``func(*args, **kwargs)`` directly, followed by eventlet.sleep() to
|
||||
ensure the eventlet hub has a chance to execute. It is more likely the
|
||||
hub will be invoked when queuing operations to an external thread.
|
||||
|
||||
:returns: result of calling func
|
||||
:raises: whatever func raises
|
||||
@ -3126,7 +3126,7 @@ class ThreadPool(object):
|
||||
|
||||
def force_run_in_thread(self, func, *args, **kwargs):
|
||||
"""
|
||||
Runs func(*args, **kwargs) in a thread. Blocks the current greenlet
|
||||
Runs ``func(*args, **kwargs)`` in a thread. Blocks the current greenlet
|
||||
until results are available.
|
||||
|
||||
Exceptions thrown will be reraised in the calling thread.
|
||||
|
@ -597,6 +597,8 @@ class PortPidState(object):
|
||||
|
||||
def port_index_pairs(self):
|
||||
"""
|
||||
Returns current (port, server index) pairs.
|
||||
|
||||
:returns: A set of (port, server_idx) tuples for currently-tracked
|
||||
ports, sockets, and PIDs.
|
||||
"""
|
||||
@ -711,6 +713,8 @@ class ServersPerPortStrategy(object):
|
||||
|
||||
def loop_timeout(self):
|
||||
"""
|
||||
Return timeout before checking for reloaded rings.
|
||||
|
||||
:returns: The time to wait for a child to exit before checking for
|
||||
reloaded rings (new ports).
|
||||
"""
|
||||
|
@ -447,7 +447,7 @@ class BaseDiskFileManager(object):
|
||||
Parse an on disk file name.
|
||||
|
||||
:param filename: the data file name including extension
|
||||
:returns: a dict, with keys for timestamp, and ext::
|
||||
:returns: a dict, with keys for timestamp, and ext:
|
||||
|
||||
* timestamp is a :class:`~swift.common.utils.Timestamp`
|
||||
* ext is a string, the file extension including the leading dot or
|
||||
@ -895,8 +895,10 @@ class BaseDiskFileManager(object):
|
||||
be yielded.
|
||||
|
||||
timestamps is a dict which may contain items mapping:
|
||||
|
||||
ts_data -> timestamp of data or tombstone file,
|
||||
ts_meta -> timestamp of meta file, if one exists
|
||||
|
||||
where timestamps are instances of
|
||||
:class:`~swift.common.utils.Timestamp`
|
||||
"""
|
||||
@ -1961,7 +1963,7 @@ class DiskFileManager(BaseDiskFileManager):
|
||||
Returns the timestamp extracted .data file name.
|
||||
|
||||
:param filename: the data file name including extension
|
||||
:returns: a dict, with keys for timestamp, and ext::
|
||||
:returns: a dict, with keys for timestamp, and ext:
|
||||
|
||||
* timestamp is a :class:`~swift.common.utils.Timestamp`
|
||||
* ext is a string, the file extension including the leading dot or
|
||||
@ -2241,12 +2243,12 @@ class ECDiskFileManager(BaseDiskFileManager):
|
||||
be stripped off to retrieve the timestamp.
|
||||
|
||||
:param filename: the data file name including extension
|
||||
:returns: a dict, with keys for timestamp, frag_index, and ext::
|
||||
:returns: a dict, with keys for timestamp, frag_index, and ext:
|
||||
|
||||
* timestamp is a :class:`~swift.common.utils.Timestamp`
|
||||
* frag_index is an int or None
|
||||
* ext is a string, the file extension including the leading dot or
|
||||
the empty string if the filename has no extenstion.
|
||||
the empty string if the filename has no extension.
|
||||
|
||||
:raises DiskFileError: if any part of the filename is not able to be
|
||||
validated.
|
||||
|
@ -317,6 +317,7 @@ def get_account_info(env, app, swift_source=None):
|
||||
|
||||
This call bypasses auth. Success does not imply that the request has
|
||||
authorization to the account.
|
||||
|
||||
:raises ValueError: when path can't be split(path, 2, 4)
|
||||
"""
|
||||
(version, account, _junk, _junk) = \
|
||||
|
Loading…
Reference in New Issue
Block a user