The swift-dispersion-populate and swift-dispersion-report tools now
accept a --insecure option.
Also, dispersion.conf now has a keystone_api_insecure option.
Default is obviously to use the secure path.
DocImpact
Change-Id: I4000352e547d9ce5b08ade54e0c886281caff891
Making it possible for one to overwrite the default set of regexes
used to search for device block errors in the log file. Also making
the log file naming pattern configurable by setting them in the
drive-audit.conf file.
Updating "Detecting Failed Drives" section on the admin guide as well.
Change-Id: I7bd3acffed196da3e09db4c9dcbb48a20bdd1cf0
'devices' is set in object-server.conf on each node, not in ring data,
and the things printed here is just for watching not for running, so
just leave a note here. (this https://review.openstack.org/#/c/23951/
is used for running, so just a note is not enough)
mark this commit as bug fixing is because this script is the last place
using /srv/node but not from conf as Chmouel said.
fixes import change on read_metadata
fixes bug #885006
Change-Id: I727ec2d01c093af61fd3895e5701d87ef67cd9ff
Instead of blacklisting Hacking globally,
only blacklist those that currently occur frequently
(for a later followup patch), and fix the rest. In
detail:
H101 Use TODO(NAME)
H201 no 'except:' at least use 'except Exception:'
H231 octal number 022 should be written as 0o22
H401 docstring should not start with a space
H701 Empty localization string
Change-Id: Ib3b3d56b68d1cf15d3b67ac9749fcbdb876dc52a
For systems with very large numbers of partitions, 1% dispersion
coverage may simply be too much/take too long. This fix allows <1
values to be used for dispersion_coverage.
DocImpact
Change-Id: I5ed35b69754d55a410e66e658b3854de57c7666b
The old format is still present and works just like it did before, so
your existing scripts won't break.
New format pros:
* it's readable even for Swift newcomers
* it's easy to extend
* it's familiar to anyone who's used a Unix command line
* we don't have to maintain the parser
New format cons:
* you can't add multiple devices in one go
Old format pros:
* you can add many devices with one command
* it's compact
Old format cons:
* it confuses newcomers
* "wait, is that zone dash IP colon port slash device, or zone slash
IP dash port colon meta underscore device?" Just try walking
someone through adding a device over voice chat.
* it's annoying to add new fields
Note that this only affects the command "swift-ring-builder
<builderfile> add". Other swift-ring-builder commands are unchanged.
DocImpact
Change-Id: I034b7f79eb6f4d81a5c4da193e1358741441c5b5
Two types of parallelism are added:
- concurrency to speed up what a single process does
- a way to run multiple daemons to work on different parts of the work
DocImpact
Change-Id: I48997f68eb2fd8de19a5ee8b9fcdf76dde2ba0ab
When added a new devices into builder the add_dev function returns it
unique id.
blueprint argparse-in-swift-ring-builder
Change-Id: I57080bb625e812f6cea71199df907a44b332b552
Dramatic part of RingBuilder.search_devs which parse a complex format
of a search device string moved to the swift-ring-builder script.
Instead, the search_devs has a simple interface to search devices.
blueprint argparse-in-swift-ring-builder
Change-Id: If3dd77b297b474fb9a058e4693fef2dfb11fca3d
Instances of the RingBuilder class can store its data to a disk file by
the save method and load it by the load method.
blueprint argparse-in-swift-ring-builder
Change-Id: I69fdf0693ca9f520d235a795ecdd2da310dcd5d3
When swift-bench is run in direct mode, don't try to delete the
containers which weren't created.
Fixes bug 1177960.
Change-Id: Ice07e8729bb776e2b215894cf95fb80b64167a8d
Allow Swift daemons and servers to optionally accept a directory as the
configuration parameter. Directory based configuration leverages
ConfigParser's native multi-file support. Files ending in '.conf' in the
given directory are parsed in lexicographical order. Filenames starting with
'.' are ignored. A mixture of file and directory configuration paths is not
supported - if the configuration path is a file behavior is unchanged.
* update swift-init to search for conf.d paths when building servers
(e.g. /etc/swift/proxy-server.conf.d/)
* new script swift-config can be used to inspect the cumulative configuration
* pull a little bit of code out of run_wsgi and test separately
* fix example config bug for the proxy servers client_disconnect option
* added section on directory based configuration to deployment guide
DocImpact
Implements: blueprint confd
Change-Id: I89b0f48e538117f28590cf6698401f74ef58003b
If we set device's weight to zero, currently balance will be set
special value(999.99) until zero weighted device return all
its partitions. So we cannot check balance has changed.
Thus we need to check balance or last_balance is special value.
Change-Id: I5b7db8b8e48db0c4771c51a764bda689869817d5
Fixes: bug #1171731
Support separate replication ip address:
- Added new function in utils. This function provides ability
to select separate IP address for replication service.
- Db_replicator and object replicators were changed.
Replication process uses new function now.
Replication network parameters:
- Replication network fields (replication_ip, replication_port)
support was added to device dictionary in swift-ring-builder script.
- Changes were made to support new fields in search, show and set_info
functions.
Implementation of replication servers:
- Separate replication servers use the same code as normal replication
servers, but with replication_server parameter = True. When using a
separate replication network, the non-replication servers set
replication_server = False. When there is no separate replication
network (the default case), replication_server is not included in the config.
DocImpact
Change-Id: Ie9af5bdcdf9241c355e36053ca4adfe49dc35bd0
Implements: blueprint dedicated-replication-network
* Algorithm format_device was changed for simplicity extension new
ip addresses parameters.
* Some prints outputs was replacement by function format_device.
Change-Id: I8565d42fcdb62eeb398c4432bb6f499c27c05cf6
Indent swift-ring-builder output was changed according
with old style (before implementation of region tier).
Change-Id: I0d1cc7acdc5baf86f343745aea6fc2120838fd36
Folks have actually been asking for this. I think they're sending a
DELETE TempURL to someone way ahead of time and the someone issues it
when they're ready. Honestly, I'm not entirely sure of the use case,
but having the set of methods configurable wouldn't hurt.
Change-Id: Ibdb48f8a72077b045eeedddfae4c0a1f56098d7a
swift-ring-builder uses outdated unbounded class method to handle
not supported command. It worked for python 2.6 or older but not
for python 2.7 or newer. This patch fixes the problem.
Change-Id: I7dbc681ef6be44f6d79ff93189ccca13c51eab74
Fixes: bug #1154882
Using root logger will make public module log failed by unknow keyword.
Just change the logger name to module itself could dismiss this impact.
And disable logger.propagate could prevent double outputing.
Change-Id: I18696d124ebac9ca970d502558972e51de759097
Fixes: bug #1105133
The region is one level above the zone; it is intended to represent a
chunk of machines that is distant from others with respect to
bandwidth and latency.
Old rings will default to having all their devices in region 1. Since
everything is in the same region by default, the ring builder will
simply distribute across zones as it did before, so your partition
assignment won't move because of this change. If you start adding
devices in other regions, of course, the assignment will change to
take that into account.
swift-ring-builder still accepts the same syntax as before, but will
default added devices to region 1 if no region is specified.
Examples:
$ swift-ring-builder foo.builder add r2z1-1.2.3.4:555/sda
$ swift-ring-builder foo.builder add r1z3-1.2.3.4:555/sda
$ swift-ring-builder foo.builder add z3-1.2.3.4:555/sda
Also, some updates to ring-overview doc.
Change-Id: Ifefbb839cdcf033e6c9201fadca95224c7303a29
Hard coding of '/srv/node' maybe drive to some errors. But it seems
unfixable (please see talks in bug report). To avoid erros as we can,
showing a friendly notification to let user set their own device
location works better than hard coding directly.
Change-Id: I45d749dd52af8d3b051dfe98dbb9d71a5a374f97
Fixes: bug #907908
Sometimes there is no date at the beginning of a line in kern.log.
Although it does not happen often, there should be a check ensuring
the program doesn't crash in case it happens.
Added try-except block surrounding parsing string to date format.
Change-Id: I44a101266582eea2199189a006afa1037a9bd4ea
Fixes: bug #1152658
1.Correct the usage of initializing a Ring object.
2.Fix typo.
3.Align lines of print_status part. (keep origin author mind)
note: Adam Kijak has commited it at early time, but didn't put it
in Gerrit. I import his commit and send it in right Workflow.
Fixes: bug #1131901
Change-Id: If56316cc424045f7df662d60088382f7fd40e8b1
Example:
$ swift-ring-builder account.builder set_replicas 4
$ swift-ring-builder rebalance
This is a prerequisite for supporting globally-distributed clusters,
as operators of such clusters will probably want at least as many
replicas as they have regions. Therefore, adding a region requires
adding a replica. Similarly, removing a region lets an operator remove
a replica and save some money on disks.
In order to not hose clusters with lots of data, swift-ring-builder
now allows for setting of fractional replicas. Thus, one can gradually
increase the replica count at a rate that does not adversely affect
cluster performance.
Example:
$ swift-ring-builder object.builder set_replicas 3.01
$ swift-ring-builder object.builder rebalance
<distribute rings and wait>
$ swift-ring-builder object.builder set_replicas 3.02
$ swift-ring-builder object.builder rebalance
<distribute rings and wait>...
Obviously, fractional replicas are nonsensical for a single
partition. A fractional replica count is for the whole ring, not for
any individual partition, and indicates the average number of replicas
of each partition. For example, a replica count of 3.2 means that 20%
of partitions have 4 replicas and 80% have 3 replicas.
Changes do not take effect until after the ring is rebalanced. Thus,
if you mean to go from 3 replicas to 3.01 but you accidentally type
2.01, no data is lost.
Additionally, 'swift-ring-builder X.builder create' can now take a
decimal argument for the number of replicas.
DocImpact
Change-Id: I12b34dacf60350a297a46be493d5d171580243ff
Change supports kern.log rotation in order to avoid loss
of significant information.
There is a year change functionality added as kern.log
does not keep record of year.
There is also backwards function added which allows
reading logs from the back to the front, speeding up the
execution along with the unit test for it
Fixes Bug 1080682
Change-Id: I93436c405aff5625396514000cab774b66022dd0
