23 Commits

Author SHA1 Message Date
Zuul
02509acc71 Merge "Add option to skip container PUT during upload" 2022-08-23 00:37:52 +00:00
Zuul
7f2649bfb7 Merge "More cleanup following py2 removal" 2022-08-18 01:49:01 +00:00
Tim Burke
9eee29d2e4 tempurl: Support sha256 and sha512 signatures
Up the default to sha256 since

* the proxy has supported (and defaulted to allowing) it for four years
  now, and
* Rackspace has supported it for even longer.

Include a note in the --help about older clusters likely requiring sha1.

Change-Id: Ibac2bb7e2e4c9946c7384f0aab8e43d0d79ba645
Related-Change: Ia9dd1a91cc3c9c946f5f029cdefc9e66bcf01046
Related-Bug: #1733634
Closes-Bug: #1977867
2022-06-08 09:30:21 -07:00
Tim Burke
5d451fb920 More cleanup following py2 removal
* Drop py2-only hacking pin from test-requirements.

* Remove quote() helper; urllib.parse.quote() works fine.

* Remove some useless code.

Change-Id: I9ffc923f58f1d11538f83ff26f7beb53cdf134c3
2022-05-24 08:33:02 -07:00
Steve Kowalik
20c97e83d3 Remove use of mock
Since Python 3.4, the unittest module has provided mock, negating the
need for the external dependancy. Switch to using unittest.mock.

Change-Id: Idec3aaed2fddd1ece3ed86ee0bcc48f7616d56fa
2022-05-24 11:56:35 +10:00
Stephen Finucane
20d837a276 Remove unnecessary unicode prefixes
All strings are unicode by default in Python 3. No need to mark them as
such.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I68fb60ef271abfddebcc9d2137424f5db2a17e92
2022-03-21 18:32:30 +00:00
Stephen Finucane
fa137a5bf1 Remove six
This mostly affects tests. Nothing too complicated

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Iabc78f651e1d48db35638280722f8019798eccd6
2022-03-21 18:31:10 +00:00
Stephen Finucane
3d10744c55 Remove __future__ imports
These aren't needed in modern Python 3 versions.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I5e81d6fb2e2cb8e4bfae4ed746da002f44e871c4
2022-03-21 18:29:25 +00:00
Tim Burke
f1858d89e0 Add option to skip container PUT during upload
Currently, a user with read/write access to a container (but without
access to creat new containers) recieves a warning every time they
upload. Now, allow them to avoid the extra request and warning by
specifying --skip-container-put on the command line.

This is also useful when testing: developers can HEAD a container to
ensure it's in memcache, shut down all container servers, then upload
and creaate a bunch of async pendings. Previously, the 503 on container
PUT would prevent the object upload from even being attempted.

Closes-Bug: 1317956
Related-Bug: 1204558
Change-Id: I3d9129a0b6b65c6c6187ae6af003b221afceef47
Related-Change: If1f8a02ee7459ea2158ffa6e958f67d299ec529e
2022-01-11 16:39:38 -08:00
Zuul
3f5d5b0252 Merge "Include storage policy when listing account with --long" 2021-10-13 01:11:03 +00:00
Zuul
e03a11c649 Merge "Allow unit tests to be run via pytest" 2021-09-23 23:56:23 +00:00
Tim Burke
373fa26ce1 Correctly aggregate totals for >10k items
Previously, we would write out totals for every page of listings, like

    $ swift list sync --prefix=09-21 --total -l
     80000000000
     80000000000
     80000000000
     58096000000

Now, roll those all into a single total:

    $ swift list sync --prefix=09-21 --total -l
    298096000000

Change-Id: Icc265636815220e33e8c9eec0a3ab80e9f899038
2021-09-21 16:06:56 -07:00
Tim Burke
ad3e8e49d0 Include storage policy when listing account with --long
Change-Id: Ibc2f9445b5a8e80cfb73d0706e20a7e4c62eec4a
2021-09-21 08:40:45 -07:00
Tim Burke
553e34ebfe Improve formatting for billions of objects
Change-Id: If8aa08c4c8c8ad6ca2c861602baf1eefa8642a8a
2021-09-20 15:49:59 -07:00
Tim Burke
99b5b81217 Allow unit tests to be run via pytest
You can run all tests by just running

   $ pytest

Or just unit tests with

   $ pytest test/unit/

Or one specific test with

   $ pytest test/unit/test_swiftclient.py::TestConnection::test_reauth

Change-Id: I1dfa239f9ee9ea85663b5c1f22631a97f87b4dfc
2021-05-11 13:34:35 -07:00
Tim Burke
74c50dee2d Have delete --all imply --versions for the CLI
Change-Id: Id5a6d4cef3d4ed76c897a099a62a4ba3ed8f8dab
2020-10-16 13:57:02 -07:00
Tim Burke
c73792c2e5 tests: Make test_delete_container_versions less flakey
Hammering that test in a tight loop, I'd often see failures due to
ordering issues. Make the delete single-threaded to avoid that.

Change-Id: Iff45be32a7c3f258214cce78001fd33ad0a39b8c
2020-09-21 16:40:09 -07:00
Tim Burke
257a7185a8 Application credential support follow-up
Following the recent v3applicationcredentials patch, if you have your
environment variables set up to work with python-openstackclient using
swiftclient's v1password plugin, swiftclient won't work:

   $ env | egrep '^(OS|ST)_'
   ST_KEY=testing
   ST_USER=test:tester
   OS_AUTH_URL=http://saio/auth/v1.0
   ST_AUTH=http://saio/auth/v1.0
   OS_USERNAME=test:tester
   OS_AUTH_TYPE=v1password
   OS_PASSWORD=testing

   $ openstack object store account show
   +------------+----------------------------+
   | Field      | Value                      |
   +------------+----------------------------+
   | Account    | AUTH_test                  |
   | Bytes      | 0                          |
   | Containers | 11                         |
   | Objects    | 0                          |
   +------------+----------------------------+

   $ swift stat
   Only "v3applicationcredential" is supported for --os-auth-type

We don't really want to allow (and mostly ignore) arbitrary OS_AUTH_TYPE
values, though -- there are a whole bunch of plugins we don't remotely
support. But it seems OK to allow any of the password plugins; while we
won't actually use them (currently), we provide roughly equivalent
functionality.

Handful of other drive-bys:
 * Use a None sentinel to determine whether keystoneauth1 is installed
   instead of trying to catch a NameError.
 * Clarify error state when keystoneauth1 is not installed.
 * Fix a typo: "sses" -> "sess".

Change-Id: Id7ea9c3ea8278ae86a04d057a472a8f8a87b8eae
Related-Change: I9190e5e7e24b6a741970fa0d0ac792deccf73d25
2020-05-27 15:17:09 -07:00
Charles Hsu
02b637cdca Support v3 application credentials auth.
Use keystoneauth1 application credential plugin and session to fetch
a token and endpoint catalog url.

$ swift --os-auth-url http://172.16.1.2:5000/v3 --auth-version 3\
      --os-application-credential-id THE_ID \
      --os-application-credential-secret THE_SECRET \
      --os-auth-type v3applicationcredential auth

Change-Id: I9190e5e7e24b6a741970fa0d0ac792deccf73d25
Closes-Bug: 1843901
Closes-Bug: 1856635
2020-04-16 12:41:04 +08:00
Tim Burke
9b0da49c0b Improve list --versions output
Have `--versions` imply `--long` and add a new column for version_id.

Also, have version-aware listings show all versions as "null" on old
Swifts that don't support object versioning (or when object versioning
is not enabled).

Change-Id: I0e009bce2471d1c140ac9b83700591cb355fee3f
2020-04-10 17:16:15 -07:00
Clay Gerrard
78edffa46c object versioning features
* add --versions to list
 * add --versions to delete
 * add --version-id to stat
 * add --version-id to delete
 * add --version-id to download

Change-Id: I89802064921778fee7efe57c7d60c976cdde3a27
2020-04-08 13:07:26 -07:00
Tim Burke
e83cd32e2a Add test for bulk-delete-attempt-counter fix
Change-Id: Ifdeefeb4a5a3fc6895bd6cda695684de02f8c602
Related-Change: If4af9141fe4f3436a4e9e0e2dfc24c6ec7292996
Related-Bug: #1852808
2019-11-15 22:08:51 +00:00
Tim Burke
1eda8f9f3e Rename "tests" directory to be "test" like in the swift repo
In addition to being less confusing for devs, this lets us actually run
tempauth tests in swiftclient dsvm jobs.

The job definition (over in the swift repo) specifies test/sample.conf,
which does not exist in this repo. As a result, those tests would skip
with

    SKIPPING FUNCTIONAL TESTS DUE TO NO CONFIG

Change-Id: I558dbf9a657d442e6e19468e543bbec855129eeb
2019-11-06 03:48:02 +00:00