Add keyword option to get_version_data() to allow passing
of the version header so that we can get the microversions.
Specifically, this is so that we can re-use this function
in barbican, which recently implemented microversions, but
doesn't return them by default, for backward compatibility
with old clients.
Change-Id: I909750381a559f9dc61650c9f98c88d4481012b7
Bump linter requirements - follow the changes
made in keystone in commits
6dfde5b48b388e32e34a385c3a9ef48da7c7c49b and
5c71ebd7a92d25df83e2e7cc5fad9990e9eebbf5 in
order to fix compatibility with Python 3.10.
Remove python-dev from bindep - it's no longer
supported by jammy and lead us to the following
errors with the announce-release job:
```
No package matching 'python-dev' is available
```
Co-Authored-By: Herve Beraud <hberaud@redhat.com>
Change-Id: If687a2678733ce018bd31c602140f073ab1a1a65
Check if the last url segment matches the project id.
Previously the check only confirmed whether the last url segment
endswith the project id which could cause problems with spurious
matches of some legacy integer project ids.
Closes-Bug: 1968793
Change-Id: I7c6c22e41bde2a73508635b7e964c58a02c12146
The passage about discovery document and allow_version_hack makes little
sense for people unfamiliar with keystoneauth internals. What it
actually means in most cases is that the remote service is not
available. Rephrase the error message and add some debug logging.
Change-Id: I156dbb45bd8c07ace1900894f6779ed9f38cf3c6
Some services, like Nova, default to requiring auth for their
versioned discovery documents. This means strict discovery
does not work on them, because discovery as it is now defaults
to not sending auth. Just changing the default would be a behavior
change resulting in sending unneeded data with *every* request.
Instead, respond to Unauthorized exceptions by retrying the request
with auth token. This way discovery will work for services that
are otherwise blocking unauthenticated access, and will get more
efficient over time as those services improve.
Depends-On: https://review.opendev.org/#/c/685999
Change-Id: I8a33e8a05bed0f18e4e42431f6d16b8a6a5270ef
On a cloud that has inaccessible version discovery documents AND uses
integer project ids, the discovery fallback logic can fail because the
project id parses as a (very large) version.
Check to see that the url segment in the fallback code begins with a v,
so that we're only attempting to parse versions from actual candidate
segments.
Closes-Bug: #1806109
Change-Id: Id90b3b9e4852494a4678b0a9bb67362babdc971c
For (unclear) historical reasons the root single version endpoint also
contains "id" and "links" fields. This makes the current workaround
for old-style endpoints take priority over the correct algorithm.
This change reorders the code, so that if "version" is present, it
always take priority over the workaround.
The unit tests are updated to be closer to real output from ironic.
Change-Id: I743b954c6c5b2f986c213acb6ec6af7e08c9f5f8
The trailng slash on a pathless url is not meaningful, but we were
treating the url given to the discovery cache as if it were. In some
circumstances, such as an endpoint_override that didn't match the
found discovery document perfectly, a double-request could be made.
Normalize root urls in the caching code so that https://example.com and
https://example.com/ would be the same.
Change-Id: I70a5911cf0f213a7816fe8d58c6cca4702ff71bb
The ironic payload looks like:
{'id': 'v1',
'links': [{"href": "https://bare-metal.example.com/v1/",
"rel": "self"}]}
This does not have version info in it, nor min/max ranges for
microversion discovery. We can't really get any useful information from
this document, but we can at least not fail when trying to deal with it.
This should then be upwards-compatible with ironic adding version discovery
information to the document that is returned.
Change-Id: I47e0f9b295c24ef168f4a033faf573b953025d4c
If the url netloc in the catalog and service's response
are not the same, we should choose the catalog's and
add the version info to it if needed.
Change-Id: If78d368bd505156a5416bb9cbfaf988204925c79
Closes-bug: #1733052
We expose version_to_string and version_match but not version_between.
openstacksdk would really like to use version_between too for matching
microversion suitability. Turn it in to a public function.
Change-Id: I710f9e1441f4caeb9bd9830f9d4a3398a71249ec
The last piece of service type alias support is to handle volumev2,
volumev3, workflowv2, workflowv3 and friends.
Although it's an annoying scenario, luckily legacy code that uses them
has a clear meaning. volumev2, version='3' is just legit not a thing.
Needed-By: https://review.openstack.org/564494
Change-Id: Iec09bcb16d8e9b09e09bf12d03c2a55e679ad70c
For better readability, make normalize_status a class that has constants
for each of the status values.
While in there, add a test for unknown status values.
Co-Authored-By: Eric Fried <efried@us.ibm.com>
Change-Id: I93ee971125bc0c7a497e1fb839df38ebd38340e1
We're repeating ourselves a bunch with a plain dict that contains the
version data. Make a class to encapsulate it. Make the class a subclass
of dict so that json translation works.
Change-Id: Ic7d122487174a5b0378a8dfaa39514c5cb2a604b
We've got great discovery support, but if someone is wanting to find out
what is available and doesn't otherwise know what they're looking for,
they're out of luck.
Add a method to EndpointData which will return all of the version data
for a given service, and then add a method to the base auth plugin that
will use that method to collect all of the version discovery documents
for every service in the cloud.
This commit adds os-service-types so that the resulting datastructure
can return only official service type keys. A followup patch will also
use os-service-types to allow catalog lookups by service-type alias.
There is a change to the test_identity_common.V2.get_auth_data method
to remove the public and internal urls for keystone from the catalog.
The V3 catalog only has keystone on admin, so this makes them have
equivilent data.
Change-Id: I07243edb939865a5df8b283e7c626874ffd830db
The Service Types Authority has grown support for aliases, and the
os-service-types library exposes the data. Add support for matching
known aliases when matching endpoints for a user.
Change-Id: Ie90c265cb17905981d877abfaaa52354a3e63692
We collect the status when doing discovery, but it's not exposed
anywhere to the use when they look at the EndpointData for an endpoint.
Add a function to normalize the statuses and then add normalized status
to the EndpointData object.
Change-Id: Icf855d7892335b093c1083cd0106946d8911010d
It is useful to be able to dump the contents of an EndpointData for
debugging purposes. This change adds a __str__ method that joins up all
the public attributes/properties.
Change-Id: Ib8985f0fa48a613ab8fca7faffbdf60c19c7cd22
Partly in response to [1], and partly for the sake of making its API
consistent and understandable, discover._version_between now allows
un-normalized input, and responds sensibly to unspecified upper & lower
bounds.
Previously the method treated None as follows:
min_version max_version behavior
None None Always False
not None None ValueError
not None not None Compare min to candidate
None not None Compare max to candidate
That's whack, yo. Took me like ten minutes just to come up with that
chart.
This change allows us to explain clearly what happens when upper or
lower bounds are None: it translates naturally to "there is no bound".
Given that, the first line of the chart (None/None) now always returns
True, which is a behavior change the callers need to be updated to
expect. This is an acceptable sacrifice for making this method sane.
[1] https://review.openstack.org/#/c/483604/9/keystoneauth1/discover.py@306
Change-Id: I09b6cf692c8bfa290b73c8d7498bc12a5e91d690
There's an unfortunately common deployment config issue where the
per-service version discovery document doesn't return scheme and netloc
properly. (Especially common with glance, as there was an actually
upstream bug in the version discovery documents until very recently)
Since up until now nobody has actually been doing version discovery
(all of the python-client libs skip it and cheat by appending strings
to URLs locally) it's a pervassive issue with existing clouds.
The workaround is to grab the scheme and netloc from the catalog url,
since the service_url is a resource on the endpoint described by the
catalog_url, and since the catalog_url has to be correct or else nothing
works. Do this in the url normalization so that it persists in the
cache, and so that things like endpoint_override are not affected.
The need for the workaround and the description of it are documented in
the API-SIG spec on consuming version discovery.
Needed-By: I78019717cdee79cab43b0d11e737327aa281fd03
Change-Id: I29102e08998b662db8136bee32217532a316f263
The _get_discovery_url_choices generator works by taking a starting URL,
splitting it on '/', and working through the parts trying to get a
matching discovery document from it. It makes assumptions about what the
URL might look like: it might have a project ID on the end of it, and a
version before that. If the starting URL has a trailing '/', splitting
the URL results in an empty string at the end of the list of parts,
which is then treated as a version. The real version is left on the URL
while the generator assumes it has already trimmed the URL down to an
unversioned endpoint. If that version does not match the version we're
seeking, the resulting discovery document will be mismatched and the
generator will fail to yield the right endpoint.
This patch normalizes the starting URL by removing the trailing '/', if
there is one. This way every part of the split URL will be meaningful.
Closes-bug: #1709658
Change-Id: I28c48f78d6f07804d6ea228f163dd37b0fcfcd58
The positional decorator results in poorly maintainable code in
a misguided effort to emulate python3's key-word-arg only notation
and functionality. This patch removes keysteonauth's dependance
on the positional decorator.
Change-Id: I20106345747860365cd0203ba1b33a2900e045b9
With the addition of input ranges for versions it's important for the
user to be able to know what version was discovered. This is already in
the EndpointData model as "api_version" - the value was just never
populated.
In cases where an Endpoint has been discovered and the user has an
EndpointData that now correctly stores the api_version of the discovered
endpoint, and where that user now wants to get microversion metadata,
getting rid of api_version as an input to get_versoined_data made
it awkward to fetch metadata about the already discovered endpoint.
Add a method so that the user can say "please run discovery on this
endpoint to fill in the data, don't look for different endpoints, and
don't make me do the api-version -> min/max transformation myself".
Change-Id: I3031d1eb0967eaa8b911e8e8a06ba722f9e69063
The previous behavior in keystoneauth for constructing an unversioned
endpoint if there's a versioned endpoint in the catalog would result in
a trailing slash. Ensure that we end up with trailing slashses on
endpoints we construct.
On the other hand, do not add trailing slashes to urls we're actually
using. If someone wants a / the url they register with the catalog, they
can put one there.
Change-Id: I2798216b5b93d49fd5d3d865193cc0be6a1f6f0d
Closes-Bug: #1705770
Among the zillion permutations being tested, there was no test case for
discovery URLs lacking a path component (e.g. 'http://foo:123' as
opposed to 'http://foo:123/v2'). A new discovery code path was indexing
the second-to-last part of the path component, resulting in IndexError
when the discovery data contained a pathless URL.
This change set fixes the IndexError and adds unit test coverage for
pathless URLs in discovery data.
Change-Id: I990a24ea32d0f7123566053046c80e48bce0536b
Partial-Bug: #1705770
There's some ongoing concern about exactly how discover.version_between
should work. Since it's new, make it private initially in the release
to give us time to work it out.
Change-Id: I349423f1b3da74b139f5fec838597c9a824ba571
We're discouraging the use of the ambiguous and difficult-to-understand
'version' parameter in new discovery methods, instead encouraging the
use of min_version and max_version.
In order to make it possible to get the same functionality, though, we
need a way to say the same thing as version="M.m", which actually means,
"min version is M.m, and max version is the latest within major version
M".
Introducing 'latest' syntax, which can be used in various ways,
including:
min_version='2.3', max_version='2.latest'
...which is equivalent to the old school version='2.3'
Change-Id: Ife842333e25c33e54bbae4c1adb101014cb8e8db
The API-WG just approved the spec for version discovery documents to
optionally provide "next_min_version" and "not_before" information.
http://specs.openstack.org/openstack/api-wg/guidelines/microversion_specification.html#version-discovery
The intended use of these is to communicate that at a point in the
future the service plans to raise the minimum microversion. It can't say
when that will happen, as a service does not know when deployers will
decide to upgrade their services. But it can communicate the earliest
date it's possible to happen, which would be the first date the service
itself would raise the minimum.
This can be used to emit warnings to users who are using a microversion
less than the next_min_version and to tell them how long they have to
think about it.
Currently keystoneauth will not consume these for that purpose. This
patch is merely about collecting the information from the discovery
document if it is there so a consumer can take action on it if they
wish.
Change-Id: Ibc404ef55eeae721a0d1d16e4e3e51ad77b5a75c
Make Discover.version_data a little more lenient in the JSON data it
accepts with respect to "max_version" and "version". Previously, if the
"max_version" field was present but empty, the "version" field would be
ignored, even if populated with a proper version number. This goes
against the intent of the API. This change causes version_data to
ignore a blank "max_version" if "version" is populated.
This change also introduces a unit test harness for the various
permutations of version_data translating input version data to
min/max_microversion, including cases for the above.
Change-Id: I1cb4ebdd57a6eadc1a16efc0fc1ceeda640f26c2
Closes-Bug: #1703438
Remove some unused variables.
Remove a couple of unused parameters from private method
EndpointData._run_discovery.
Fix some formatting issues to reclaim a line or two, or because my IDE
flags them.
Change-Id: Idcc1f1e4dc6068125ea30ce0ab482a99a9e0b1cc
Fix an edge case where discover.normalize_version_number could return a
one-member tuple rather than the expected >=2-member tuple.
Fix up the docstring for the same, including the above behavior.
Change-Id: Ibe54da05705846e47063f8fc639b31df773bed9d
Closes-Bug: #1703414
EndpointData._run_discovery thought it was caching already-tried URLs,
but it was really caching each letter of each URL, so the cache wasn't
working:
tried = set()
...
if vers_url in tried:
continue
tried.update(vers_url) # Adds each char of `vers_url` to `tried`
Changed that last line to:
tried.add(vers_url)
Added a unit test for same.
Change-Id: I894636d846de4a3b63000f9e2f79f378134c1de8
Closes-Bug: #1703447
Remove unused `authenticated` kwarg from as-yet-unreleased public API
EndpointData.get_versioned_data, and private method
EndpointData._set_version_info.
Change-Id: I03a9969df4586c79b6b63d44fcb5474dd94e6ba2
Closes-Bug: #1703446
Flesh out docstrings for the following in discover.py:
- get_version_data
- version_to_string
- Discover.version_data
- EndpointData._get_version_discover_hack
and keystoneauth1.fixture.discovery.MicroversionDiscovery
Change-Id: I0358a86b13c8a9d2179d5bbf66a7ded7e6c777d1
Remove four unneeded things in unit tests, updated a docstring,
and changed TypeError to ValueError.
Change-Id: Iad43ab2afa3767739ef3137b2691875520d052b3
There are a two interrelated pieces in this patch which are around
fixing up places where discovery was being re-run inappropriately.
They fall out from adding tests for the functionality and couldn't
be sanely shifted back further in the stack without a big dance.
Switch the default for "discover_versions" on all of the calls that
return an EndpointData to "True". It's a new feature and is a thing that
doesn't make a ton of sense to call if you don't want discovery run.
However, get_endpoint uses it, so needs to be able to pass in
discover_version=False, so the option is still useful. Make sure that
get_endpoint and other places where ksa calls get_endpoint_data on
behalf of the user work as before without unneeded discovery.
Add tests to show that we use actually use the discovery cache properly when
we've previously done discovery that can satisfy the new request. This
works from the microversion optimization patch, but we had to clean up
a couple of things to show it fully in a test.
Change-Id: I54053336edf1b3c2bd35a77dbd78f56388b8e806
In the case where we're just asking for an endpoint, and the endpoint in
the catalog matches the requested version, there is no need to fetch the
version discovery document.
Change-Id: I2c14337a3fcb1369652d43ca68e6a572fef6d425
Grzegorz Grasza