a1c32702, 736cf54a, and 38787d0f remove uses of `simplejson` from
various parts of Swift in favor of the standard libary `json`
module (introduced in Python 2.6). This commit performs the remaining
`simplejson` to `json` replacements, removes two comments highlighting
quirks of simplejson with respect to Unicode, and removes the references
to it in setup documentation and requirements.txt.
There were a lot of places where we were importing json from
swift.common.utils, which is less intuitive than a direct `import json`,
so that replacement is made as well.
(And in two more tiny drive-bys, we add some pretty-indenting to an XML
fragment and use `super` rather than naming a base class explicitly.)
Change-Id: I769e88dda7f76ce15cf7ce930dc1874d24f9498a
Follow up for https://review.openstack.org/#/c/236007
This fixes following minor items:
- Fix a 'raise Exception class' syntax to 'raise Exception instance'
- Use original eventlet.Timeout instead of swift.exceptions.Timeout
imported from eventlet.Timeout
- Change Timeout to initiate w/o args (1st arguments should be timeout
second and we don't have to set None if we don't want to set the sec)
- Add a message argument to some Exception instances
Change-Id: Iab608cd8a1f4d3f5b4963c26b94ab0501837ffe1
Added new unit tests:
test_add_device_old_missing_region
test_create_ring_number_of_arguments
test_add_duplicate_devices
test_rebalance_with_seed
test_set_overload_number_of_arguments
test_main_no_arguments
test_main_single_argument
test_main_with_safe
Modified existing unit tests to create sample ring at start of test.
This change was needed to have unit tests run correctly and demonstrate
code coverage.
test_unknown
test_search_device_number_of_arguments
test_list_parts_number_of_arguments
test_set_weight_number_of_arguments
test_set_info_number_of_arguments
test_remove_device_number_of_arguments
test_set_min_part_hours_number_of_arguments
test_set_replicas_number_of_arguments
test_set_replicas_invalid_value
Updates to handled nested mocks.
Updates to handle no exception case when SystemExit is expected.
PEP8 corrections.
Moved new tests from try blocks to use of assertRaises or call to
run_srb using exp_results with specified exit codes.
Updated run_srb to accept a dictionary of expected results. Specifically,
look for 'valid_exit_codes' to test, default to (0,1).
Change-Id: I4cf3f5f055a9babba140c68a9c7ff90b9c50ea62
Added unit test cases to cover all code paths in REPLICATE and __call__
functions in account/server.py and container/server.py
Change-Id: Ia335e9a6668821d3e34b12fc3a133a707880e87f
To minimize external library dependencies for Swift unit
tests and SAIO, PyECLib 1.1.1 introduces a native backend
'liberasurecode_rs_vand.' This patch is to migrate over
the unit tests to the new ec_type when available.
This change will work with current pyeclib requirements
(==1.0.7) and also future requirements (>=1.0.7).
When we're able to raise *our* requirements to >=1.1.1 we
should remove jerasure from the list of preferred backends.
Related SAIO doc and example config changes should be
included with that patch.
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: Idf657f0acf0479bc8158972e568a29dbc08eaf3b
In the _transfer_data method, we translate all (Exception, Timeout)
into a 499 whereas we should consider translating them to 500 on
particular returning error scenarios.
This affects both ReplicatedObjectController and ECObjectControllear.
Change-Id: I571bbc5b1451243907b094a5718c8735fd824268
Closes-Bug: 1504299
Add tests for direct_delete_account, direct_get_object, retry
Tests to validate possible argument calls of direct_put_object
Extend FakeConn to support chunks in response
Change-Id: I00cbc6c535ed8b3fb29bc07fdc51fdbb1220ff10
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
test_write_builder_after_device_removal() wasn't setting a
default min_part_hours so a warnign was printed. Explicitly
adding a min_part_hours suppresses the warning
Change-Id: I6f234b72c34e066abb91f28e6eacf50e29be8842
internal_client was not reading response if response status is not 200.
So proxy server treats this as client disconnect and logs 499 in log file.
This patch fixes that by reading response if response status is greater
than or equal to 300 and in acceptable statuses.
Closes-Bug: #1364752
Change-Id: I0512a25895da583956f76031e3c5de5c970bce01
Add unit tests to cover all code paths in print_item_locations
function in cli/info.py.
Update comment to match what's tested for invalid/missing policy.
Update tests to verify output of print_item_locations
Corrected PEP8 compliance violations.
Change-Id: I84958cb70205ee8d7ea246826dd56201fa642da9
We have some tests that exercise both the sender and receiver,
but are spread across test_ssync_sender.py and test_ssync_receiver.py.
This creates a new module test_ssync.py and moves the end-to-end tests
into there.
Change-Id: Iea3e9932734924453f7241432afda90abbc75c06
keystoneclient uses threading.Lock(), but swift doesn't
monkeypatch threading, this result in lockup when two
greenthreads try to acquire a non green lock.
This change fixes that.
Change-Id: I9b44284a5eb598a6978364819f253e031f4eaeef
Closes-bug: #1508424
This patch adds more unit tests to diminish missing pieces
of the coverage in the proxy_controllers_base unit test.
Change-Id: I85ba1955c681cc9d5b2a70ac31155678d2e5b6fd
This patch add some unit tests to prevent regression and to describe
validate_hash_path behavior, I found in review with
https://review.openstack.org/#/c/231864/
*bonus*
- Fix test_hash_path to use "with" syntax instead of "try/finally"
for assigning a testing value into a global variable.
Change-Id: I948999a8fb8addb9a378dbf8bee853b205aeafad
contextlib.nested() is missing completely in Python 3.
Since 2.7, we can use multiple context managers in a 'with' statement,
like so:
with thing1() as t1, thing2() as t2:
do_stuff()
Now, if we had some code that needed to nest an arbitrary number of
context managers, there's stuff we could do with contextlib.ExitStack
and such... but we don't. We only use contextlib.nested() in tests to
set up bunches of mocks without crazy-deep indentation, and all that
stuff fits perfectly into multiple-context-manager 'with' statements.
Change-Id: Id472958b007948f05dbd4c7fb8cf3ffab58e2681
Follow up for change I7c57bba053711a27d3802efe6f2a0bf53483a54f
The current tests won't prevent this kind of regression in the
updaters' __init__ methods:
self.node_timeout = int(float(conf.get('node_timeout', 10)))
so this makes them a little tighter.
Change-Id: I60bd79ac392deb1e8a108357d48474dddd9028c1
On a full disk, a call to delete an object will fail when it tries to
write tombstones. Handling DiskFileNoSpace exception raised by
swift.common.utils.
Change-Id: I8f0cfcc4159ee154fcd3e7ca90c422aa5aadf0b3
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
Closes-Bug: 1491675
This patch add a expositive test that closing connection
(fd and socket) causes StopIteration on reader at proxy server
on PUT object case.
And then, if the connection closed and proxy detects a lack of
data length, it will result in HTTPClientDisconnected.
The only reason why I leave test_client_disconnect as current
is Bill is now working for the change at
https://review.openstack.org/#/c/236007/
Change-Id: If68f752a2b0dd6cfad4d007ca5d155e6ffb378e5
If a device with 0 weight is tried to remove, the following rebalance
does not write changes into builder file.
Scenario:
$ swift-ring-builder object.builder set_weight --id 1 0.00
$ swift-ring-builder object.builder rebalance
Wait for moving files out of the device id=1.
$ swift-ring-builder object.builder remove --id 1
$ swift-ring-builder object.builder rebalance
In fact, the device id=1 is not removed after rebalance (must be --force used).
Change-Id: Iad5a444023eae9882a3addd7f119ff4d18559ddd
backward() is written to handle binary files:
* Replace literal native strings to literal byte strings: add b'...'
prefix
* Update unit tests: use byte strings
* TemporaryFile(): use the default mode 'w+b', instead of using 'r+w'
('r+w' mode creates a Unicode file on Python 3).
Change-Id: Ic91f7e6c605db0b888763080d49f0f501029837f
The current rule inside the db_replicator is to rsync+merge
containers during replication if the difference between rowids
differ by more than 50%:
# if the difference in rowids between the two differs by
# more than 50%, rsync then do a remote merge.
if rinfo['max_row'] / float(info['max_row']) < 0.5:
This mean on smaller containers, that only have few rows, and differ
by a small number still rsync+merge rather then copying rows.
This change adds a new condition, the difference in the rowids must
be greater than the defined per_diff otherwise usync will be used:
# if the difference in rowids between the two differs by
# more than 50% and the difference is greater than per_diff,
# rsync then do a remote merge.
# NOTE: difference > per_diff stops us from dropping to rsync
# on smaller containers, who have only a few rows to sync.
if rinfo['max_row'] / float(info['max_row']) < 0.5 and \
info['max_row'] - rinfo['max_row'] > self.per_diff:
Change-Id: I9e779f71bf37714919a525404565dd075762b0d4
Closes-bug: #1019712
