8423 Commits

Author SHA1 Message Date
Zuul
4aa33a2cc2 Merge "py3: start porting for unit/proxy/test_server.py" 2019-05-06 20:36:35 +00:00
zhufl
6d63297f02 [Trivial fix]Remove unnecessary slash
This is to remove the unncessary slash when the line doesn't
exceed the length of 79, to make the code more readable.

Change-Id: I31070067913560d6147f95d0fdfba4f9f9ce0936
2019-05-06 14:23:05 +08:00
Tim Burke
767872210f py3: port unit/proxy/test_sysmeta.py
Change-Id: I378705a8d9ad51dbfa85287d44e6313aa7680c23
2019-05-05 05:06:54 +00:00
Tim Burke
fed0204e86 py3: Make StoragePolicy objects hashable
Change-Id: I2e233afff26878ba94e268c71f6779bcca7b641c
2019-05-04 21:01:29 -07:00
Tim Burke
b8284538be py3: start porting for unit/proxy/test_server.py
Mostly this ammounts to

    Exception.message -> Exception.args[0]
    '...' -> b'...'
    StringIO -> BytesIO
    makefile() -> makefile('rwb')
    iter.next() -> next(iter)
    bytes[n] -> bytes[n:n + 1]
    integer division

Note that the versioning tests are mostly untouched; they seemed to get
a little hairy.

Change-Id: I167b5375e7ed39d4abecf0653f84834ea7dac635
2019-05-04 20:35:05 -07:00
Clay Gerrard
34bd4f7fa3 Clarify usage of dequeue_from_legacy option
Change-Id: Iae9aa7a91b9afc19cb8613b5bc31de463b853dde
2019-05-05 03:20:34 +00:00
Zuul
fd73353bb1 Merge "Enable to configure object-expirer in object-server.conf" 2019-05-05 00:25:11 +00:00
Zuul
506d0fa821 Merge "Unify auth_host/port/ssl/prefix func test options into auth_uri" 2019-05-05 00:04:08 +00:00
Tim Burke
9290f29e1c Wait longer for log lines in unit test
We've seen a noteworthy (literally -- there's a comment about it) number
of failures for test_multiphase_put_drains_extra_commit_junk_disconnect;
I'd rather waste an extra few hundredths of a second every run than have
to recheck most patches we want merged.

Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: Iebec6c18866e4f8ef40faab539b173f017d658e3
2019-05-04 15:56:55 -07:00
Zuul
310416abf3 Merge "py3: port s3api" 2019-05-04 22:49:38 +00:00
Zuul
de7c7f1007 Merge "Remove py35 unit test job" 2019-05-04 22:38:47 +00:00
Zuul
2f05025255 Merge "versioned_writes: stop trying to catch HTTPPreconditionFailed" 2019-05-04 18:35:50 +00:00
Zuul
89f2f7e8cd Merge "Fix wrong tox env name from func-post-ec to func-ec" 2019-05-04 17:49:22 +00:00
Kazuhiro MIYAHARA
443f029a58 Enable to configure object-expirer in object-server.conf
To prepare for object-expirer's general task queue feature [1],
this patch enables to configure object-expirer in object-server.conf.
Object-expirer.conf can be used in the same manner as before, but deprecated.

If both of object-server.conf with "object-expirer" section and
object-expirer.conf are in a node, only object-server.conf is used.
Object-expirer.conf is used only if all object-server.conf doesn't have
"object-expirer" section.

There are two differences between "object-expirer.conf" style and
"object-server.conf" style.

The first difference is `dequeue_from_legacy` default value.
`dequeue_from_legacy` defines task queue mode. In "object-expirer.conf"
style, the default mode is legacy queue. In "object-server.conf" style,
the default mode is general queue. But general mode means no-op mode
for now, because general task queue is not implemented yet.

The second difference is internal client config. In "object-expirer.conf"
style, config file of internal client is the object-expirer.conf itself.
In "object-server.conf" style, config file of internal client is
another file.

[1]: https://review.openstack.org/#/c/517389/

Co-Authored-By: Matthew Oliver <matt@oliver.net.au>

Change-Id: Ib21568f9b9d8547da87a99d65ae73a550e9c3230
2019-05-04 15:45:02 +00:00
Zuul
b8824f052b Merge "Fix ostestr invocation" 2019-05-04 15:35:32 +00:00
Tim Burke
1e4cdca339 Fix ostestr invocation
More-recent versions of stestr are rather picky about wanting an integer
for concurrency [1], but ostestr prior to [2] defaults it to None. As a
result, all our func tests would fail with:

   TypeError: int() argument must be a string or a number, not 'NoneType'

Also, move the rolling upgrade job to non-voting since it's going to
pull in the old version but try to run with new deps.

[1] https://github.com/mtreinish/stestr/commit/c93bc81
[2] https://review.openstack.org/#/c/651275/

Change-Id: Icba6c46bcff0e0516107e0f6742d32c92488ae20
2019-05-03 22:48:22 -07:00
Zuul
1c899f78d4 Merge "py3: Stop using stdlib's putrequest(); it only does ASCII" 2019-05-04 00:13:42 +00:00
Kota Tsuyuzaki
f5b472cdc7 Fix wrong tox env name from func-post-ec to func-ec
Change-Id: I789fb2413985942460d3275b479d6410c62c380f
2019-05-04 07:28:44 +09:00
Zuul
4ee9545805 Merge "py3: Be able to parse non-RFC-compliant request lines" 2019-05-03 21:08:45 +00:00
Zuul
8585c8742e Merge "Make log format for requests configurable" 2019-05-03 21:08:43 +00:00
Tim Burke
359d136c29 Unify auth_host/port/ssl/prefix func test options into auth_uri
Retain support for the old way, though (at least for now).

Change-Id: Id3c2ed87230c5918c18e2c01d086df8157f036b1
2019-05-03 20:55:52 +00:00
Tim Burke
fbf6d84796 Make py36 job voting
Change-Id: I4768fe41d6347bcae44774730350f4be583bdf15
2019-05-03 13:36:55 -07:00
Tim Burke
a239f4ec35 Remove py35 unit test job
Change-Id: Iadb3bed811907740f85bd9277ba09a4ca17fc025
2019-05-03 13:36:42 -07:00
Zuul
ce1ba6a51d Merge "Pass logger instances to AccountBroker/ContainerBroker" 2019-05-03 10:27:27 +00:00
Gilles Biannic
a4cc353375 Make log format for requests configurable
Add the log_msg_template option in proxy-server.conf and log_format in
a/c/o-server.conf. It is a string parsable by Python's format()
function. Some fields containing user data might be anonymized by using
log_anonymization_method and log_anonymization_salt.

Change-Id: I29e30ef45fe3f8a026e7897127ffae08a6a80cd9
2019-05-02 17:43:25 -06:00
Tim Burke
3a9f3f8419 py3: port s3api
Drive-by: When passing a list or tuple to swob.Response as an app_iter,
check that it's full of byte strings.

Change-Id: Ifc35aacb2e45004f74c871f08ff3c52bc57c1463
2019-05-02 16:38:39 -07:00
Tim Burke
c0ae48ba9a py3: Stop using stdlib's putrequest(); it only does ASCII
Note that this only affects the functest client.

See also: https://bugs.python.org/issue36274

Change-Id: I1359c475fbe93db2f0fcc4b450be6dd5073f922e
2019-05-02 14:44:18 -07:00
Tim Burke
93b49c5e48 py3: Be able to parse non-RFC-compliant request lines
There's a bug in CPython [1] that causes servers to mis-parse request
lines that include the bytes \x85 or \xa0.  Naturally, we have
functional tests that (with high probability) will send such request
lines. There's a fix proposed, but the earliest it's likely to land
would be for 3.8, and we need to be able to target an earlier Python.

So, intercept the request line immediately before parsing and re-write
it to be RFC-compliant. Note that this is done for Python 2 as well,
though there should be no change in the request environment that
eventlet eventually hands to us.

[1] https://bugs.python.org/issue33973

Change-Id: Ie648f5c04d4415f3b620fb196fa567ce7575d522
2019-05-02 14:44:18 -07:00
Takashi Kajinami
4b2ac606b1 Pass logger instances to AccountBroker/ContainerBroker
There are some processes which don't pass their logger instances
when creating AccountBrokers/ContainerBrokers, which causes some
error messages with a different setting from the other logs
generated by the processes.

This patch makes them pass logger instances, to make sure they
generate all logs according to their log settings.

Change-Id: I914c3a2811e1a2b7f19ad2bc9b3d042fcba63820
2019-05-03 01:15:20 +09:00
Zuul
f55167a735 Merge "Increase node_timeout in gate" 2019-04-30 22:12:01 +00:00
Clay Gerrard
58352b285d s3api multi upload test clean up
Add a test assertion for invalid ETag of MPU manifest and make some
global constants more obvious.

Change-Id: I830c0d2e473d1740f9138ebc6efd64a39d85d1dc
Related-Change-Id: I7df13e670f702a2956a9c8057fcb971f4bfb7319
2019-04-29 14:22:36 -05:00
Tim Burke
c51db382cb s3api: Block ETag header from CompleteMultipartUpload requests
Change-Id: I7df13e670f702a2956a9c8057fcb971f4bfb7319
Closes-Bug: 1810397
2019-04-29 14:01:06 -05:00
ZhongShengping
32bf43990c Replace git.openstack.org URLs with opendev.org URLs
Thorough replacement of git.openstack.org URLs with their opendev.org
counterparts.

Change-Id: I5e77307de6a484fd69b2a5863423ceb357e8601f
2019-04-24 09:56:54 +08:00
OpenDev Sysadmins
89eced960c OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:28:47 +00:00
Tim Burke
9468e28ccf Make test_read_write_acl_listings more robust
When retrying in func tests, you may get a different status if the
previous attempt succeeded.

Change-Id: Ie9bf99af1298fae5cd91fb03d5b5f13ff4873b3c
2019-04-17 21:55:13 -07:00
Zuul
3be4d2f64b Merge "Clean up dlo unit tests" 2019-04-18 00:37:36 +00:00
Zuul
1a1b44df6f Merge "py3: port dlo" 2019-04-18 00:37:34 +00:00
Zuul
5e6b7f32ac Merge "Move from py35 to py37 for gating py3 jobs" 2019-04-17 20:25:42 +00:00
Tim Burke
b6ebabee78 Clean up dlo unit tests
We don't actually need so many py2/py3 branches, and once we
clean those up, there's hardly any reason to import six.

Change-Id: Ia3b4f02e7eb99ad1a76aa35c39dc198528fd39ad
2019-04-17 09:15:56 -07:00
Pete Zaitcev
893acffbc0 py3: port dlo
Change-Id: I7236ddea0acde93d0789ad8affa76df0097a86aa
2019-04-16 22:34:58 -05:00
Tim Burke
7cb276da42 Move from py35 to py37 for gating py3 jobs
Looking at https://governance.openstack.org/tc/reference/runtimes/train.html
OpenStack is no longer targetting Python 3.5, so it doesn't really make sense
to gate on it. Move to Python 3.7, just to make it so we can go longer before
having to do this shuffle again.

Presumably, once we actually declare support for Python 3.6, we should
make that job voting, too? We've had enough gate flaky-ness lately that
I'm not keen on it.

Change-Id: I9c9e5df76a6592ab9c512b7c1ba807f9a0053059
2019-04-16 13:50:29 -07:00
zhufl
65d3cb75f8 Use assertIn to check whether substring is in str
This is to use assertIn to check whether substring is in str,
go get clearer failure message.

Change-Id: I40aff29454c423389755a5330751d2f69a227a05
2019-04-16 08:56:36 +08:00
Zuul
5090a15f52 Merge "Make staticweb return URL-encoded Location headers" 2019-04-13 02:47:42 +00:00
Zuul
d7ef9f8f80 Merge "Add tests for unexpected names in part dir" 2019-04-13 00:43:41 +00:00
Tim Burke
e5ff405ec9 Make staticweb return URL-encoded Location headers
Also, enhance tests to verify proper encoding in generated responses.

Change-Id: Ib696b1c3a34722f3a6de010973b90ef8b1917012
Partial-Bug: 1774238
2019-04-12 12:29:00 -07:00
Zuul
b27026f6c7 Merge "py3: port staticweb middleware" 2019-04-11 18:33:09 +00:00
Zuul
f21fe6f479 Merge "Remove our urlparse wrapper" 2019-04-11 08:16:16 +00:00
Tim Burke
049e56a5d0 Remove our urlparse wrapper
It has not been necessary since we dropped support for Python 2.6.
See https://github.com/python/cpython/commit/8c6d9d7 and
https://bugs.python.org/issue2987.

Be sure to keep a `urlparse` name in utils, though; swauth (at least)
still expects there to be a swift.common.utils.urlparse.

Change-Id: If2502868f251b8a83aa929ee22b10046e708d111
2019-04-10 12:39:09 -07:00
Clay Gerrard
5b40f69ede Add tests for unexpected names in part dir
An earlier revision of the related change had a bug that this test would
have discovered.

Related-Change-Id: I85dcaf65b9f19ac4659fa5937f9b0b0e804fc54e
Change-Id: Iedd85ec65a11de189ce73e650d674aee0dc7e402
2019-04-10 09:13:13 -05:00
Matthew Oliver
b86bc51607 py3: port staticweb middleware
This patch is porting the staticweb middleware to py3.

Change-Id: I5d9a13baecedd13d2b7a8ae3dd639eaff0894441
2019-04-10 17:08:21 +10:00