5067 Commits

Author SHA1 Message Date
Paul Dardeau
7f4139bc26 Added unit tests for ringbuilder command-line utility
Added new unit tests:

test_add_device_old_missing_region
test_create_ring_number_of_arguments
test_add_duplicate_devices
test_rebalance_with_seed
test_set_overload_number_of_arguments
test_main_no_arguments
test_main_single_argument
test_main_with_safe

Modified existing unit tests to create sample ring at start of test.
This change was needed to have unit tests run correctly and demonstrate
code coverage.

test_unknown
test_search_device_number_of_arguments
test_list_parts_number_of_arguments
test_set_weight_number_of_arguments
test_set_info_number_of_arguments
test_remove_device_number_of_arguments
test_set_min_part_hours_number_of_arguments
test_set_replicas_number_of_arguments
test_set_replicas_invalid_value

Updates to handled nested mocks.
Updates to handle no exception case when SystemExit is expected.
PEP8 corrections.

Moved new tests from try blocks to use of assertRaises or call to
run_srb using exp_results with specified exit codes.

Updated run_srb to accept a dictionary of expected results. Specifically,
look for 'valid_exit_codes' to test, default to (0,1).

Change-Id: I4cf3f5f055a9babba140c68a9c7ff90b9c50ea62
2015-11-11 15:09:40 +00:00
Jenkins
e75dc8028b Merge "EC: Use best available ec_type in unittests" 2015-11-11 04:34:57 +00:00
Jenkins
f20346d96e Merge "suppress warning output in a unit test" 2015-11-11 04:28:21 +00:00
Tushar Gohad
2d85a3f699 EC: Use best available ec_type in unittests
To minimize external library dependencies for Swift unit
tests and SAIO, PyECLib 1.1.1 introduces a native backend
'liberasurecode_rs_vand.'  This patch is to migrate over
the unit tests to the new ec_type when available.

This change will work with current pyeclib requirements
(==1.0.7) and also future requirements (>=1.0.7).

When we're able to raise *our* requirements to >=1.1.1 we
should remove jerasure from the list of preferred backends.
Related SAIO doc and example config changes should be
included with that patch.

Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: Idf657f0acf0479bc8158972e568a29dbc08eaf3b
2015-11-10 12:18:50 -08:00
Jenkins
58fcc82428 Merge "Compare Swift config checksum in swift-recon --all" 2015-11-10 19:34:17 +00:00
venkatamahesh
705642db4c Change stackforge repo to openstack repo
The projects which are moved to openstack
are corrected by replacing stackforge with openstack

Change-Id: I65b794a7f10df617bc2a4caf2c4010477a82dbc2
2015-11-10 19:46:31 +05:30
John Dickinson
d755f5b520 suppress warning output in a unit test
test_write_builder_after_device_removal() wasn't setting a
default min_part_hours so a warnign was printed. Explicitly
adding a min_part_hours suppresses the warning

Change-Id: I6f234b72c34e066abb91f28e6eacf50e29be8842
2015-11-09 22:03:37 -08:00
Jenkins
166c34383a Merge "DiskFileNoSpace check during object delete" 2015-11-10 04:40:54 +00:00
Jenkins
85aaf50c21 Merge "Add unit tests to cover print_item_locations" 2015-11-09 20:53:37 +00:00
Jenkins
a5ebd97052 Merge "Add missing docs for ring.builder.rebalance" 2015-11-07 00:12:04 +00:00
Jenkins
510bcc1761 Merge "Re-organise ssync tests" 2015-11-06 23:43:34 +00:00
Jenkins
14b30e56ac Merge "bug/1270114. Missing documentation for Bulk Middleware." 2015-11-06 21:57:35 +00:00
Jenkins
f4e1f9fbf1 Merge "On py3, use dnspython3 dependency, not dnspython" 2015-11-06 19:42:20 +00:00
Jenkins
f26af37bc2 Merge "Read the response body, if response status is greater than 300." 2015-11-06 08:44:47 +00:00
Pradeep Kumar Singh
37337d5fcb Read the response body, if response status is greater than 300.
internal_client was not reading response if response status is not 200.
So proxy server treats this as client disconnect and logs 499 in log file.
This patch fixes that by reading response if response status is greater
than or equal to 300 and in acceptable statuses.

Closes-Bug: #1364752

Change-Id: I0512a25895da583956f76031e3c5de5c970bce01
2015-11-05 20:07:15 -08:00
Jenkins
e66b9284bc Merge "Add unit tests for utils.validate_hash_path" 2015-11-06 04:00:21 +00:00
Jenkins
12d08d6a6c Merge "Device marked to be removed in info about the ring." 2015-11-06 03:00:40 +00:00
Kota Tsuyuzaki
c81f202f71 Add missing docs for ring.builder.rebalance
commit 71993d84e88cc1d5f7742182905cace21c7e88cb added
a new 'remove_dev' column to the
swift.common.ring.builder.rebalance return value.

This patch adds the docs for that and clean up a bit to
the variable name to be easy to read.

Change-Id: Idfd46e47b9f6894cbafc8b7701a4c7414212f79f
2015-11-05 17:38:08 -08:00
Jenkins
22e294c418 Merge "Optimize the code performance" 2015-11-05 22:39:41 +00:00
Paul Dardeau
331382e192 Add unit tests to cover print_item_locations
Add unit tests to cover all code paths in print_item_locations
function in cli/info.py.

Update comment to match what's tested for invalid/missing policy.

Update tests to verify output of print_item_locations

Corrected PEP8 compliance violations.

Change-Id: I84958cb70205ee8d7ea246826dd56201fa642da9
2015-11-05 20:58:34 +00:00
Ondřej Nový
e0430fc74a Compare Swift config checksum in swift-recon --all
Change-Id: I796fe0895f4e5ddeb04c0d79a73579ce8bb9aa40
2015-11-05 21:21:21 +01:00
Victor Stinner
609095b23f On py3, use dnspython3 dependency, not dnspython
Update requirements to install dnspython3 on Python 3 and dnspython on
Python 2. dnspython is not compatible with Python 3, and dnspython3
is not compatible with Python 2.

Add a requirement on pbr to ensure that pbr 1.6 or newer is
installed. pbr 1.0 or newer is required to support environment
markers in requirements which are now used to select dnspython or
dnspython3 depending on the Python major version.

Note: pbr 1.0 is enough, but OpenStack global requirements asks to
use "pbr>=1.6".

Change-Id: Ie5d437cd396972d4143ffffb96ec3c289e221b08
2015-11-05 15:56:24 +01:00
Alistair Coles
6858510b59 Re-organise ssync tests
We have some tests that exercise both the sender and receiver,
but are spread across test_ssync_sender.py and test_ssync_receiver.py.
This creates a new module test_ssync.py and moves the end-to-end tests
into there.

Change-Id: Iea3e9932734924453f7241432afda90abbc75c06
2015-11-05 14:50:28 +00:00
Jenkins
ec9095bf58 Merge "Get rid of contextlib.nested() for py3" 2015-11-05 12:20:06 +00:00
Venkateswarlu Pallamala
48421f902e Optimize the code performance
Usually, the number of characters in FORBIDDEN_CHARS are fewer than
the number of characters in the req.path - so it's better to use
smaller count for the outer loop.

Change-Id: I78b7e0ff4cae1cc90e8d29d7bddcec7fe0ba4bd4
2015-11-04 06:06:10 -08:00
Jenkins
7242c4a006 Merge "Fix missing *-replicator conf sections in deployment guide" 2015-11-03 22:51:24 +00:00
Jenkins
e1910dff17 Merge "swift-ring-builder can't remove a device with zero weight" 2015-11-02 14:37:33 +00:00
Jenkins
5f23d0617f Merge "Add unit tests for swift.proxy.controllers.base" 2015-10-30 05:44:42 +00:00
Jenkins
0453730f93 Merge "Update admin guide with region." 2015-10-29 02:21:17 +00:00
Jenkins
9a7795086c Merge "Add verification socket close causes StopIteration" 2015-10-28 09:50:29 +00:00
Jenkins
410b48dfa5 Merge "SLO docs(tring): manifest params in table, code block; copyedits" 2015-10-28 03:28:20 +00:00
Bill Huber
faef717cd3 Add unit tests for swift.proxy.controllers.base
This patch adds more unit tests to diminish missing pieces
of the coverage in the proxy_controllers_base unit test.

Change-Id: I85ba1955c681cc9d5b2a70ac31155678d2e5b6fd
2015-10-27 09:26:46 -05:00
Peter Lisák
febdd6c1b4 Device marked to be removed in info about the ring.
Showing devices with 'DEL' mark if will be removed next rebalance.

Change-Id: I171aa8658b1c4ac1689ab9532fe65d114567baa7
2015-10-26 13:26:21 +01:00
Jenkins
246e134275 Merge "Minor improvement to updater unit tests" 2015-10-26 09:29:48 +00:00
Kota Tsuyuzaki
a4a5fcad8f Add unit tests for utils.validate_hash_path
This patch add some unit tests to prevent regression and to describe
validate_hash_path behavior, I found in review with

https://review.openstack.org/#/c/231864/

*bonus*
- Fix test_hash_path to use "with" syntax instead of "try/finally"
  for assigning a testing value into a global variable.

Change-Id: I948999a8fb8addb9a378dbf8bee853b205aeafad
2015-10-23 23:00:54 -07:00
Samuel Merritt
e31ecb24b6 Get rid of contextlib.nested() for py3
contextlib.nested() is missing completely in Python 3.

Since 2.7, we can use multiple context managers in a 'with' statement,
like so:

    with thing1() as t1, thing2() as t2:
        do_stuff()

Now, if we had some code that needed to nest an arbitrary number of
context managers, there's stuff we could do with contextlib.ExitStack
and such... but we don't. We only use contextlib.nested() in tests to
set up bunches of mocks without crazy-deep indentation, and all that
stuff fits perfectly into multiple-context-manager 'with' statements.

Change-Id: Id472958b007948f05dbd4c7fb8cf3ffab58e2681
2015-10-23 11:44:54 -07:00
Paul Dardeau
73e032049f Update admin guide with region.
Added region prefix to example commands for adding devices to ring.
Also updates description to include region prefix.

Change-Id: Ie6d6485b497cea973e37909b5b19b44946c8aa89
2015-10-23 18:20:25 +00:00
Jenkins
a094560f0c Merge "node_timeout as float in configs" 2015-10-23 16:49:03 +00:00
Alistair Coles
5d8d2e98ee Minor improvement to updater unit tests
Follow up for change I7c57bba053711a27d3802efe6f2a0bf53483a54f

The current tests won't prevent this kind of regression in the
updaters' __init__ methods:

  self.node_timeout = int(float(conf.get('node_timeout', 10)))

so this makes them a little tighter.

Change-Id: I60bd79ac392deb1e8a108357d48474dddd9028c1
2015-10-23 16:38:24 +01:00
Alistair Coles
1a2b54fc0a Fix missing *-replicator conf sections in deployment guide
The doc for these sections was missing because of an rst error - the
source is there in rst file but didn't make it into the html output.

Add doc for per_diff and max_diffs in account and container doc sections.

Also, fix a bunch of other sphinx build errors and most of the warnings.

Change-Id: If9ed2619b2f92c6c65a94f41d8819db8726d3893
2015-10-23 14:58:38 +01:00
OpenStack Proposal Bot
8f78c1a2b0 Imported Translations from Zanata
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: Ifebb8f1b50ac4c2c0f43df48c2414ce0f3742e42
2015-10-23 06:34:15 +00:00
Zack M. Davis
59849a9923 SLO docs(tring): manifest params in table, code block; copyedits
It's æsthetically pleasing to display the segment description parameters
in a table (using the reStructuredText "simple" table markup) and to use
a code block (which receives syntax hightlighting in the rendered
document!) for the example JSON fragment. Also, a few very minor copy
changes are made (a comma after an introductory phrase clarifies; JSON
and ETag receieve their canonical capitalizations; the elements of the
manifest JSON aren't "files" exactly; the inclusive nature of the range
is noted for the benefit of harried readers who won't read the "Range
Specification" section later in the document).

Change-Id: I5296fc04c6fe5410ad14bba7de0a7f8ab060b126
2015-10-22 17:00:40 -07:00
Ganesh Maharaj Mahalingam
b3a0acd5c7 DiskFileNoSpace check during object delete
On a full disk, a call to delete an object will fail when it tries to
write tombstones. Handling DiskFileNoSpace exception raised by
swift.common.utils.

Change-Id: I8f0cfcc4159ee154fcd3e7ca90c422aa5aadf0b3
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
Closes-Bug: 1491675
2015-10-22 13:55:01 +00:00
Jenkins
e283718ed0 Merge "Add test to verify staticweb uses auth'd subrequests" 2015-10-22 10:40:19 +00:00
Jenkins
76c83cd501 Merge "Add backend subrequest tests for staticweb" 2015-10-22 10:40:11 +00:00
Jenkins
a148a00cdb Merge "Do not use pre-authenticated requests in staticweb" 2015-10-22 10:38:19 +00:00
Jenkins
732c09982f Merge "Documenting the ability to specify ranges for SLO segments" 2015-10-22 01:43:20 +00:00
Kota Tsuyuzaki
b376e9dc34 Add verification socket close causes StopIteration
This patch add a expositive test that closing connection
(fd and socket) causes StopIteration on reader at proxy server
on PUT object case.

And then, if the connection closed and proxy detects a lack of
data length, it will result in HTTPClientDisconnected.

The only reason why I leave test_client_disconnect as current
is Bill is now working for the change at

https://review.openstack.org/#/c/236007/

Change-Id: If68f752a2b0dd6cfad4d007ca5d155e6ffb378e5
2015-10-21 05:44:46 -07:00
Jenkins
e0e8d8a255 Merge "Port swift.common.utils.backward() to Python 3" 2015-10-21 01:25:25 +00:00
Lisak, Peter
71993d84e8 swift-ring-builder can't remove a device with zero weight
If a device with 0 weight is tried to remove, the following rebalance
does not write changes into builder file.

Scenario:
$ swift-ring-builder object.builder set_weight --id 1 0.00
$ swift-ring-builder object.builder rebalance
Wait for moving files out of the device id=1.
$ swift-ring-builder object.builder remove --id 1
$ swift-ring-builder object.builder rebalance
In fact, the device id=1 is not removed after rebalance (must be --force used).

Change-Id: Iad5a444023eae9882a3addd7f119ff4d18559ddd
2015-10-20 16:13:28 +02:00