8423 Commits

Author SHA1 Message Date
Zuul
234b23d01a Merge "Dockerfile cleanup" 2019-06-19 03:21:30 +00:00
Zuul
f75167d925 Merge "Remove nobarrier option" 2019-06-19 03:21:29 +00:00
Zuul
fe0dd5733c Merge "py3: increase _MAXHEADERS for eventlet, too" 2019-06-19 02:25:46 +00:00
Tim Burke
b10f4bae28 func tests: tolerate more 404s when deleting
Change-Id: I3129e4f94ac39964f2f17ea05b6b2dd807fba82a
2019-06-18 09:54:02 -07:00
Zuul
1ae4952af4 Merge "Enable s3api in saio docker container" 2019-06-18 16:29:54 +00:00
Thiago da Silva
4ab3706b75 Remove nobarrier option
XFS no longer supports nobarrier mount option.
It has been deprecated for a long time[1] and removed in
recent kernel versions resulting in an error when trying to
mount: "kernel: XFS (loop0): unknown mount option [nobarrier]."

[1] - https://patchwork.kernel.org/patch/9486549/

Change-Id: Iaa9208fb20545ae9ac990f0e180899108d983123
2019-06-18 14:47:39 +01:00
Zuul
761a16acba Merge "Fix :param: in docstring" 2019-06-17 22:10:44 +00:00
Tim Burke
876ce231d6 py3: increase _MAXHEADERS for eventlet, too
eventlet.wsgi goes straight to eventlet.green.* instead of stdlib.

Change-Id: Ie1f8808c04f975ccc7cea532c4a566d6984f1757
2019-06-17 14:41:19 -07:00
Tim Burke
5573354655 Move calls to self.app outside of error handling
On py3, if/when you hit an error, you can get very noisy tracebacks like

  <traceback coming out of split_path()>

  During handling of the above exception, another exception occurred:

  <meaningful traceback>

In general, I like this, but when we've used exception handling for
flow-control, it gets difficult to separate the wheat from the chaff.

Change-Id: I5f3bc6416207cab2c7e3a77ee6689360b55990e7
2019-06-17 13:43:48 -07:00
Zuul
4316c53e4a Merge "Remove invalid dict entries from hashes.pkl" 2019-06-17 20:38:45 +00:00
Zuul
2c593dda10 Merge "s3api: Fix ETag when copying a MU part from another MU" 2019-06-17 20:38:43 +00:00
Tim Burke
6374734d7d Stop manually installing eventlet for our py3 tests
0.25.0 is now included in upper-constraints.txt

Related-Change: I04be480829eaf67ce328b2e7f7d7e2f70aaed90e
Change-Id: I5b8ecd82d6915d0e1cb216c423bbcc75c50a6f3d
2019-06-17 13:22:11 -07:00
Tim Burke
014358018e py3: Decode some things in s3api
Query strings and upload IDs should be native strings.

With this, you should be able to run the s3api functional tests under
py2 against py3 services.

Change-Id: I896c84872a5d63b23d6edabeb6d48ce7bc9a533e
2019-06-17 11:39:43 -07:00
Tim Burke
ef664700d4 py3: Specify an encoding when loading db.pending pickles
Otherwise you can hit UnicodeDecodeErrors when upgrading from py2 to
py3.

Change-Id: Iab808b3d0c2051badda0450015077ea8b1e5adb8
2019-06-17 08:06:39 -07:00
Tim Burke
e6c971b7e0 Stop sending non-UTF-8 prefixes in func tests
Previously, we'd (with high likelihood) cut it off midway through
a codepoint.

Change-Id: Iba4cce213dce204d7d1b53a937bfbe537276f0f3
2019-06-17 08:04:23 -07:00
Thiago da Silva
ccd3b890cd Enable s3api in saio docker container
Change-Id: Ie759c4092408a210d32f6457142457087343d33a
2019-06-17 14:54:11 +01:00
Zuul
bc19dd1649 Merge "Installing liberasurecode from Alpine Linux repos instead of building from source." 2019-06-15 00:15:05 +00:00
Tim Burke
ef8818a639 Fix up how we memcache on py3
Previously, we stored the WSGI strings in memcached and returned them when
responding to get_account/container_info calls. This would lead to cache
corruption in a heterogenous py2/py3 cluster such as you would have during
a rolling upgrade.

Now, only store and return native strings.

Change-Id: I8d6f66dfe846493972e433f70bad76a33d204562
2019-06-14 08:20:36 -07:00
Zuul
74e1f2e053 Merge "Update mailmap for Kota Tsuyuzaki" 2019-06-14 03:41:18 +00:00
zhufl
0f46ace8a9 Fix :param: in docstring
In docstring :param should be used instead of :param:

Change-Id: I55cd3fc188dca4a357dbb599aa58ab4083d3baf5
2019-06-14 11:29:27 +08:00
Tim Burke
aa2f1db1b7 Ensure get_*_info keys are native strings
Change-Id: I29bbea48ae38cfabf449a9f4cca1f5f27769405a
2019-06-11 14:50:49 -07:00
Tim Burke
b7b92b97b1 Bump up minimum cryptography version
...not because we strictly *need* newer cryptography, but rather because
distro packages have moved forward to the point where the 1.x series
won't compile from source and PyPI doesn't have wheels for them.

See changes like:

- https://github.com/pyca/cryptography/commit/6e7ea2e
- https://github.com/pyca/cryptography/commit/f88aea5

Change-Id: I1ff5b61873cf382c7a89873ed4ba6153f299262a
2019-06-11 09:54:42 -07:00
Kota Tsuyuzaki
668e45af6d Update mailmap for Kota Tsuyuzaki
His company changes the e-mail domain and the old one has been
recieve-only so to update the author's e-mail address, add the
entry to the mailmap.

Change-Id: I0c1f77425d9cefb2e6340ffb4c65cdea956c216f
2019-06-11 13:28:32 +09:00
Tim Burke
06e4533c2e py3: Fix header_to_environ_key
When doing things for the sake of WSGI, .upper(), .title() etc. should
only be used on bytes.

Change-Id: I130ba5014b7eff458d87ab29eb42fe45607c9a12
2019-06-10 13:47:33 -07:00
Tim Burke
e8e7106d14 py3: port obj/reconstructor tests
All of the swift changes we needed for this were already done elsewhere.

Change-Id: Ib2c26fdf7bd36ed1cccd5dbd1fa208f912f4d8d5
2019-06-10 08:31:41 -07:00
Clay Gerrard
dca658103a Fix swift with python <2.7.9
Closes-Bug: #1831932

Change-Id: I0d33864f4bffa401082548ee9a52f6eb50cb1f39
2019-06-07 10:39:00 -05:00
Zuul
05bfaf7ae1 Merge "py3: port ssync" 2019-06-06 06:02:12 +00:00
Zuul
f7445dda8a Merge "Do not sync suffixes when remote rejects reconstructor sync" 2019-06-06 01:16:51 +00:00
Zuul
0058c31e1f Merge "s3api: Allow clients to upload with UNSIGNED-PAYLOAD" 2019-06-06 01:16:49 +00:00
Zuul
17d7a95547 Merge "py3: symlink follow-up" 2019-06-05 23:22:20 +00:00
Tim Burke
d9cafca246 py3: port ssync
Change-Id: I63a502be13f5dcda2a457d38f2fc5f1ca469d562
2019-06-05 13:26:18 -07:00
Tim Burke
a122b720e9 Dockerfile cleanup
- Use `set -e` in all install scripts. Clean up swift_needs to still
  work properly and exit cleanly under `set -e`.
- Drop some redundant slashes.

Change-Id: I05f38c6366e3da0aeed2b6451a8bd9d7dc531119
2019-06-05 12:12:43 -07:00
Ehud Kaldor
17fd725707 Installing liberasurecode from Alpine Linux repos instead of building from source.
closes #1831136

Change-Id: Ic9a2844fffc6dba6a4d44a9712edbb5cd91f0470
2019-06-05 11:29:28 -07:00
Zuul
23eca56ce0 Merge "Bump openstackdocstheme to 1.30.0" 2019-06-05 15:18:29 +00:00
Tim Burke
2eb2451685 Increase some middleware test coverage
This patch increases the test coverage of the following middlewares:
 - list_endpoints
 - crypto
 - crossdomain

Change-Id: I3dec85f61da07bd110bf42220d5ba46e11833a90
2019-06-05 06:19:04 +00:00
翟小君
98637dc1e7 Bump openstackdocstheme to 1.30.0
...to pick up many improvements, including the return of table borders.

Change-Id: I166211b690b08521171b489582fa419d756b1972
2019-06-05 12:38:00 +08:00
Zuul
4ffc5db5f2 Merge "Note s3api compatability requirements in sample.conf" 2019-06-05 04:11:55 +00:00
Zuul
4ae5ae100d Merge "Have a separate s3api functional test suite" 2019-06-04 18:02:49 +00:00
Clay Gerrard
79ab3a490a Note s3api compatability requirements in sample.conf
Change-Id: I12f6e4a5a880a2d5e0f4fff4618e4196be311267
2019-06-04 11:45:35 -05:00
Pete Zaitcev
eaf180c68d Fix a few Sphinx warnings
This precludes us from landing anything in the gate, because we
treat Sphinx warnings as errors in OpenStack.

Specifically, 3 warnings are fixed:

/home/zuul/src/opendev.org/openstack/swift/swift/common/middleware\
/s3api/etree.py:docstring of\
 swift.common.middleware.s3api.etree.Element:1:\
Inline strong start-string without end-string.

duplicate object description of\
 swift.common.middleware.versioned_writes, other instance in\
 middleware, use :noindex: for one of them

duplicate object description of\
 swift.common.ring.composite_builder, other instance in\
 overview_ring, use :noindex: for one of them

Change-Id: I5e0bd61a301d46b2674b4605882462575b635220
2019-06-03 22:01:13 -05:00
Kuan-Lin Chen
37fa12cd83 Do not sync suffixes when remote rejects reconstructor sync
The commit a0fcca1e makes reconstructor not sync suffixes when remote
reject reconstructor revert. However, the exact same logic should
be applied to SYNC job as well. REPLICATE requests aren't generally
needed when using SSYC (which the reconstructor always does).

If a ssync_sender fails to finish a sync the reconstructor should skip
the REPLICATE call entirely and move on to the next partition without
causing any useless remote IO.

Change-Id: Ida50539e645ea7e2950ba668c7f031a8d10da787
Closes-Bug: #1665141
2019-06-03 18:39:51 +08:00
Christian Schwede
c9e78d15e1 Remove invalid dict entries from hashes.pkl
If the data in a hashes.pkl is corrupted but still de-serialized without
errors, it will mess up the replication and gets never fixed. This
happens for example if one of the keys is a NULL byte.

This patch checks if the dict keys in hashes.pkl are valid strings and
invalidates it if not.

Closes-Bug: 1830881
Change-Id: I84b062d062ff49935feed0aee3e1963bb72eb5ea
2019-05-31 11:07:42 +02:00
Tim Burke
2e35376c6d py3: symlink follow-up
- Have the unit tests use WSGI strings, like a real system.
- Port the func tests.

Change-Id: I3a6f409208de45ebf9f55f7f59e4fe6ac6fbe163
2019-05-30 16:25:17 -07:00
Matthew Oliver
4f9595f113 py3: port symlink middleware
This patch ports the symlink middleware to py3.
The middleware itself seems to be mostly fine and most changes
are in the symlink unit tests.

Change-Id: I973c2e1bb8969cf6bffece8ce68881c393efbaef
2019-05-30 11:44:42 -07:00
Zuul
2be7dcd850 Merge "Modify the version path URL of the configuration file" 2019-05-29 23:32:24 +00:00
Zuul
854a72facf Merge "S3Api: handle non-ASCII markers in v1 listings." 2019-05-29 23:32:20 +00:00
Zuul
769c0fb7fb Merge "Rework private-request-method interface" 2019-05-24 20:06:00 +00:00
Zuul
657b022689 Merge "py3: port diskfile" 2019-05-24 02:38:09 +00:00
Zuul
5a42544bed Merge "Clean up test_write_read_metadata" 2019-05-24 02:38:05 +00:00
Zuul
36648cd2e2 Merge "Add more tests for async expirer" 2019-05-23 09:25:50 +00:00