In case of global-request-id request, Adapter
send two global request id header
- "X-OpenStack-Request-ID"
- "X-Openstack-Request-Id".
Example: https://zuul.opendev.org/t/openstack/build/c5b1debf78df4aa3bdda34f0b4c53c37/log/testrepository.subunit#2385
This is becasue of the header not being Case Insensitive
and end up with two different name of same header with difference
of cap 'D'.
Unit test for whether request global-request-id has precedence
over adapter fail many times because of how different python version
treat the dict. py3.6 and above are all good as dict maintain the
insertion ordered but py3.5 can fail it any time.
We can see consistent failure in py35 jobs:
- https://review.opendev.org/#/c/730687/
Let's make the headers always Case Insensitive which is
what RFC says.
Change-Id: Iba707dd0506d22e144aca4fdfc9b140c8e37ae02
Closes-Bug: #1881351
Clients like ironicclient and swiftclient use fixed delay for their
build-in retry functionality. To replace it without changing behavior
we need a similar feature.
Change-Id: I1f9de98dae5719842f03d45e5a9d724199d5718b
When using keystoneclient sessions, the new parameter is not available
and breaks the keystoneclient unit tests[1]. Only use the semaphore
kwarg when using keystoneauth sessions.
[1] https://review.openstack.org/640953
Change-Id: I0cc7f2514e143ec532d8fb895618f7cf1fea9cc3
shade/openstacksdk has implemented client-side rate limiting on top of
keystoneauth for ages and uses it extensively in nodepool. As part of an
effort to refactor that code a new approach was devised which was much
simpler and therfore suitable for inclusion in keystoneauth directly.
The underlying goal is two-fold, but fundamentally is about allowing a
user to add some settings so that they can avoid slamming their cloud.
First, allow a user to express that they never want to exceed a given
rate. Second, allow a user to limit the number of concurrent requests
allowed to be in flight.
The settings and logic are added to Adapter and not Session so that the
settings can easily be per-service. There is no need to block requests
to nova on a neutron rate limit, after all.
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
Needed-By: https://review.openstack.org/604926
Change-Id: Ic831e03a37d804f45b7ee58c87f92fa0f4411ad8
Typing docstrings for the next patch was annoying. Reformat the
docstrings to wrap at the front rather than in visual blocks.
Change-Id: I08fc1e45e032197f3bb0b8311c032b471494ef80
The get_all_version_data method is useful for getting a full listing of
what's going on with version discovery on a cloud. Sometimes though
people just want to see the versions for a specific service. Add a
filter to allow skipping making the version discovery call in the first
place, instead of needing to do that as a post-filtering step.
Change-Id: Ia3ca4be2976d1a5e7914fa8f2adbf7297e8cb1e1
It can be annoying to have to say raise_exc=False (or use try/except) on
every call when talking to an API where 4xx response codes are
useful/normal/informative or where the preferred coding style is to use
conditionals rather than try/except.
With this change, the Adapter constructor takes a new kwarg, raise_exc.
It defaults to None, and the existing behavior is unchanged. If set to
a boolean value, that is used as the default for requests. Specifying
raise_exc to the primitives (get, head, put, post, patch, delete,
request) at any point along the chain will still take precedence.
Change-Id: Ie291c3cb891467728d8ca33cf62afdab37c82f34
Closes-Bug: #1776501
Ironic commonly returns HTTP 409 when a node is locked by another routine
and HTTP 503 when the conductor has no free threads to process the request.
Currently it is managed by custom code in ironicclient and openstacksdk,
this change will allow to move it to Session itself.
Change-Id: I04e356e7856b020cd20aa598e291ef31e02730d2
This used to use a mutable default which would result in self.allow
always being a dict. Since allow is a public attribute, even though
keystoneauth1/identity/base.py does allow = allow or {} at the usage
place, we need to make sure it's a dict here too.
Change-Id: I388b8d7859744a1e7bd454641aaf98e4b6caa3b0
Similar to get_endpoint, which knows it doesn't need full endpoint_data,
if a user just wants to know what major version the discovery process
wound up with, there are cases in which we do not need to fetch
discovery documents. Provide an API call that a user can use when this
is the information they need to avoid them having to play games with
discover_versions settings.
Change-Id: I204a45d1d139a90176bcc2ef8d46decd09b2cd5b
The positional decorator results in poorly maintainable code in
a misguided effort to emulate python3's key-word-arg only notation
and functionality. This patch removes keysteonauth's dependance
on the positional decorator.
Change-Id: I20106345747860365cd0203ba1b33a2900e045b9
The user now has the ability to know what microversions are available,
but needs to be able to send a microversion header with their request.
Add a microversion parameter to Session that will construct and send the
header. The microversion header requires a service_type. One should be
available but it's possible for it to be missing if someone is using an
endpoint_override. Provide a parameter to let the user specify a
service_type for the microversion call in such cases.
Change-Id: I63cdd67701749630228f9496eda82b3c8747a608
get_endpoint_data on an adapater is intended to return the endpoint_data
for the endpoint the adapter is mounted to, so passing in additional
kwargs doesn't make any sense. What's more, the interaction between the
existing values and the passed in values is hard to reason about.
Update the docs on using get_endpoint_data to highlight this.
Change-Id: I851c615407bc3e22af4350a4facf8488fa9c7945
They should be here as an Adapter is essentially a codified
endpoint_filter.
Add them to the conf options for Adapter, since that is how Adapters get
defined in services which is one of the reasons for doing all of this
work.
Change-Id: I8c6613bac09f28169e903b303c7330b1e90fe72d
There are a two interrelated pieces in this patch which are around
fixing up places where discovery was being re-run inappropriately.
They fall out from adding tests for the functionality and couldn't
be sanely shifted back further in the stack without a big dance.
Switch the default for "discover_versions" on all of the calls that
return an EndpointData to "True". It's a new feature and is a thing that
doesn't make a ton of sense to call if you don't want discovery run.
However, get_endpoint uses it, so needs to be able to pass in
discover_version=False, so the option is still useful. Make sure that
get_endpoint and other places where ksa calls get_endpoint_data on
behalf of the user work as before without unneeded discovery.
Add tests to show that we use actually use the discovery cache properly when
we've previously done discovery that can satisfy the new request. This
works from the microversion optimization patch, but we had to clean up
a couple of things to show it fully in a test.
Change-Id: I54053336edf1b3c2bd35a77dbd78f56388b8e806
If a user has provided an endpoint_override, they may still be
interested in version discovery data for the endpoint. Doing that is
always an opt-in behavior, so we set the strictness flag to prevent any
URL manipulations. We'll either return data or None.
Change-Id: I673beafd1e55fd096bb221b2ca6794bb124653b8
All of this "get endpoint data" stuff is great, but it's no good if the
user can't ask "hey - what are we working with here". It's safe enough
to not cache this data on the session or adapter objects, because the
source data is all cached anyway. That way calling the method with
different filters will always return the correct data.
Change-Id: I1feb505b2d838bd4b6505336c23b68041a938008
In order to make it easier for projects to enable global_request_id
passing, make it something the Adapter understands directly, so that
the logic of adding extra headers doesn't need to be in every client.
Part of the push to enable global_request_id throughout OpenStack.
oslo spec I65de8261746b25d45e105394f4eeb95b9cb3bd42
Change-Id: Ic75be3acb8b77aae8da631e3c4cd6f545a9a35cb
In the spirit of keystoneauth1.loading.session,
keystoneauth1.loading.adapter.Adapter is a BaseLoader subclass providing
oslo_config* options suitable for inclusion by config groups wishing to
support keystoneauth1.adapter.Adapter operations such as endpoint
discovery.
*Future work should be done to move the argparse options from
keystoneauth1.adapter.Adapter into the new loading.adapter.Adapter class
for consistency.
Change-Id: Icb8c295799cc5ef4ca16f44c238838668fb7ab93
Partial-Implements: bp use-service-catalog-for-endpoints
Whilst historically we always wanted keystoneauth to do the most
permissive thing and allow a versioned or unversioned entry in a service
catalog there are now cases where we would prefer to fail when the
catalog is misconfigured. This will allow a client to opt out of
versioned catalog endpoints to insist that the deployment is correctly
configured.
Closes-Bug: #1668484
Change-Id: Ided0e0c7409994f703175fe61bd4043b840bcf1e
Auth token middleware does a bit of a hack where it passes an Adapter in
as a session to the client. This is useful there because we need to know
much more about the authentication information than we do in most
clients.
We should look at fixing this in auth_token middleware, however for now
we shouldn't issue a deprecation warning when a user passes an Adapter
as a session object because this has always been designed to work - just
not something we recommend.
Change-Id: If7ebe59d5908275e607f32244027c8e6f3d1e157
Closes-Bug: #1647230
You can pass client_name and client_version to Adapter.__init__ but for
most clients this means overriding Adapter.__init__ and setdefault()-ing
the client_name and version.
As most clients already override the Adapter object it'd be easier if
they could just set these values on the class as they are not going to
change between instances.
Change-Id: I301a7f77c8cf423bc1d45e3dcbb2325f6853b9a9
There are still a lot of places where keystoneclient sessions are being
used and we've made a fair effort to maintain compatibility with these
sessions.
Unfortunately passing client_name and client_version for user_agent
generation is something only present in keystoneauth and passing it to a
keystoneclient session results in failure.
Whilst it would be good to just tell people to fix their code in reality
we'll probably be dealing with this for a while so just check to ensure
it really is a keystoneauth session we are passing parameters to and
warn otherwise.
Change-Id: I4d51ee08cfa9094443aca7128fe5323a95974a4d
Allow specifying a service name and version to the session and a client
name and version to the adapter. The way this will work is that
libraries such as keystoneclient will pass client_name and
client_version when creating their adapter. Then when nova or another
service creates a session it will provide the service name and version.
The combination of these will be used to provide a meaningful user
agent.
Change-Id: Ibe516d9b248513579d5e8ca94015c4ae9c00f3f9
Closes-Bug: #1614846
Allow clients and services to set additional_headers that will be sent
with all requests made by the session.
Change-Id: Idbd2e5159de5790c7db65c806b964f220bb6628e
The Discover class can fiilter API versions by experimental status, deprecated
status, and unknown status, and potentially more designations in the future.
The parameters that control this were not exposed in the Session or Adapter, so
users could not take advantage of this filtering through normal means. This
patch creates an 'allow' parameter for the Adapter that will get passed down as
keyword arguments into Discover.raw_version_data().
Now, given an unversioned endpoint like:
$ openstack endpoint show cinder
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| adminurl | http://192.168.122.183:8776 |
| enabled | True |
| id | 485107c1d92b41829c331a2dc82aaaeb |
| internalurl | http://192.168.122.183:8776 |
| publicurl | http://192.168.122.183:8776 |
| region | RegionOne |
| service_id | 01b4f36a173d4c59b31fc95763095373 |
| service_name | cinder |
| service_type | volume |
+--------------+----------------------------------+
an Adapter can be used like this (this example would be expected to fail
since it disallows the deprecated volume V1 API):
auth = Password(<auth_params>)
sess = session.Session(auth=auth)
adptr = adapter.Adapter(sess)
adptr.get('<project-id>/volumes',
endpoint_filter={'service_type': 'volume',
'interface': 'public',
'version': 1},
allow={'allow_deprecated': False}))
This is inspired by an abandoned patch to keystoneclient[1] that exposed this
information as a tuple. The problem with exposing it like that is that
raw_version_data() defaults allow_deprecated to True, so including 'deprecated'
in the tuple or not including it would have the same result. Using a dict
allows us to keep the Discover interface the same.
[1] https://review.openstack.org/#/c/130159
Co-authored-by: Endre Karlson <endre.karlson@hp.com>
Change-Id: I54c29e1c2a4a2b02a3967f4ea108b8d2533616eb
Closes-bug: #1394245
Currently tox ignores D400 (D400: First line should end with a period).
This change removes D400 ignore.
All pep8 violatios are fixed.
Change-Id: I9190a15a36c90d3c60a9c520cb53d5f182b0c4e9
The positional library was spun directly out of what keystoneauth1 was
using so this is a fairly trivial change.
Change-Id: I7931ed1547d2a05e2d248bc3240a576dc68a0a40
It's an override type, which means we need to know if it's set or not.
The default value for it is, well, obvious - since it's based on the
name. This was an oversight in the duplication of the method.
Change-Id: I5fe1d1dd5f0f1e34ec5eddf92cb6964cc389ca51
Similar to Session and auth plugins, there are a set of argparse
arguments that feed directly into the Adapter class. Add a function
to register the appropriate argparse arguments, as well as one to
create an Adapter from a session and a set of arguments.
Change-Id: Ifea90b981044009c3642b268dd639a703df1ef05
Conver from the keystoneauth namespace to keystoneauth1. This is to
ensure that is is possible to install all versions of keystoneauth
side-by-side.
Change-Id: Ibbaf11525980c8edb5968d8b8ee19c55094e77d8
Ghanshyam Mann