openstack/api-sig
Code Issues Proposed changes

Add document describing consuming version discovery

Browse Source 
The next patch actually describes desired state of version discovery.
But in an epic amount of cart-before-the-horse, we have the process for
consuming the discovery already because the process must take in to account
the present as well as the past. This process has kept in mind what consuming
the recommended discovery process _wants_ to look like in the future and in
calls that out in a few places. The intent would be that the algorithm here
would work for all clouds, but that as clouds and services adopt API-SIG
recommendations, the interactions with the clouds would become more
efficient. (so for clients using the complete algorithm they should be
upwards compatible with forthcoming API-SIG guidelines and will just
naturally do less work over time).

I believe this is consistent in defaults, fallbacks and error conditions with
what is currently implemented in keystoneauth, although there is
additional logic presented here which is not yet in keystoneauth. The
intent is for the process presented here to not change the behavior
experienced by current keystoneauth users, with the exception that when
the complete algorithm is implemented it's possible that an additional
API call may be made on older clouds. That is to say, keystoneauth
should not need to make any incompatible changes, but may need to add
some features to be a fully compliant implementation.

Apologies for the size and complexity. It turns out there are many
historical oddities still lurking out there and advice to client
authors that does not take them in to account would be incomplete. On
the other hand, as we drive guidelines forward into being implemented,
the need for this much crazy logic should go away.

Co-Authored-By: Dmitry Tantsur <divius.inside@gmail.com>
Change-Id: I241f76bca8ac27fc3d27028ae284b9012a2da7e9
This commit is contained in:
Monty Taylor 2017-04-24 11:03:48 -05:00
parent 28cce354d9
commit e809580088
6 changed files with 1783 additions and 471 deletions

669
guidelines/consuming-catalog.rst

@ -22,9 +22,18 @@ from the Service Catalog.
.. note:: The use of the word "object" in this document refers to a JSON
object, not an Object from any particular programming language.
.. _catalog-user-request:
User Request
============
.. note:: It is worth noting that 'user' is a maleable concept. For instance,
the shade library performs service discovery on behalf of its users
so does not expect its users to provide a 'service-type'. In that
case, shade is the 'user' of the keystoneauth library which is the
discovery implementation. It is definitely not required that all
consumers of OpenStack clouds know all of these things.
The ultimate goal of this process is for a user to find the information about
an endpoint for a service given some inputs. The user will start the process
knowing some number of these parameters. Each additional input expected from
@ -35,69 +44,192 @@ in helping the user ask the right question.
.. note:: Be liberal with what you accept and strict with what you emit.
There is one piece of information that is absolutely required that the
user know:
The following is a list of such pieces of information that can be provided
as user input. When an implementation exposes the ability for a user to
express these parameters it is **STRONGLY** recommended that these names
be used, as they show up across the OpenStack ecosystem and make discussion
easier.
service-type
It is assumed that the user has an ``{auth-url}`` and authentication
information. The authentication process itself is out of the scope of this
document.
Required Inputs
---------------
There is one piece of information that is absolutely required that the
user know.
``service-type``
The official name of the service, such as ``compute``, ``image`` or
``block-storage`` as listed in the `OpenStack Service Types Authority`_.
``block-storage`` as listed in the :doc:`OpenStack Service Types Authority
<consuming-catalog/authority>`.
Required. It is impossible for a user to consume service discovery without
knowing what service they want to discover.
The user may also wish to express an alteration to the general algorithm:
be-strict
Forgo lenient backwards compatibility concessions and be more strict in
input and output validation.
Optional Filters
----------------
There are several optional pieces of information that the user might know,
or additional constraints the user might wish to express.
or additional constraints the user might wish to express to control how the
endpoints for a service are selected.
region-name
``region-name``
The region of the service the user desires to work with. May be optional,
depending on whether the cloud has more than one region. Services
all exist within regions, but some clouds only have one region.
If ``{be-strict}`` has been given, ``{region-name}`` is required.
If ``{be-strict}`` (see below) has been given, ``{region-name}`` is required.
.. note:: It is highly recommended that ``{region-name}`` always be required
to protect against single-region clouds adding a region in the
future. However, keystoneauth today allows region-name to be omitted
and there are a large number of clouds in existence with a single
region named ``RegionOne``. For completely new libraries or major
versions where breaking behavior is acceptable, requiring region-name
by default would be preferred.
.. note:: It is highly recommended that ``{region-name}`` always be required
to protect against single-region clouds adding a region in the
future. However, the canonical OpenStack implementation
*keystoneauth* today allows region name to be omitted and there are
a large number of clouds in existence with a single region named
``RegionOne``. For completely new libraries or major versions
where breaking behavior is acceptable, requiring region name
by default would be preferred, but breaking users just to introduce
the restriction is discouraged.
interface
Which API interface, such as ``public``, ``internal``, or ``admin``
the user wants to use. A user can also request a list of interfaces they find
acceptable in the order of their preference, such as
``['internal', 'public']`` (Optional, defaults to ``public``.)
``interface``
Which API interface, such as ``public``, ``internal`` or ``admin``, that
the user wants to use. A user should be able to request a list of interfaces
they find acceptable in the order of their preference, such as
``['internal', 'public']`` (Optional, defaults to ``public``)
service-name
``endpoint-version`` OR ``min-endpoint-version``, ``max-endpoint-version``
The **major** version of the service the user desires to work with. Optional.
An endpoint version is inherently a range with a minimum and a maximum value.
Whether it is presented to the user as a single parameter or a pair of
parameters is an implementation detail.
Each endpoint version is a string with one (``3``) or two (``3.1``) numbers,
separated by a dot.
.. warning:: Care has to be taken to not confuse major versions consisting
of two numbers with microversions. Microversions usually exist
within a certain major version, and also have a form of ``X.Y``.
No services currently use both major versions and microversions
in the form of ``X.Y``.
.. TODO(dtantsur): so, what if a service has both major versions in the form
of ``X.Y`` and microversions?
Version strings are not decimals, the are a tuple of 2 numbers combined with
a dot. Therefore, ``3.10`` is higher than ``3.9``.
A user can omit the endpoint-version indicating that they want to use
whatever endpoint is in the ``{service-catalog}``.
A user can desire to work with the latest available version, in which
case the ``{endpoint-version}`` should be ``latest``. If s
``{min-endpoint-version}`` is ``latest``, ``{max-endpoint-version}`` must be
omitted or also ``latest``.
A version can be specified with a minor value of ``latest`` to indicate
the highest minor version of a given major version. For instance,
``3.latest`` would match the highest of ``3.3`` and ``3.4`` but not ``4.0``.
If the parameter is presented as a single string, a single value should be
interpreted as if ``{min-endpoint-version}`` is the value given and
``{max-endpoint-version}`` is ``MAJOR.latest``. For instance, if ``3.4`` is
given as a single value, ``{min-endpoint-version}`` is ``3.4`` and
``{max-endpoint-version}`` is ``3.latest``.
It may seem strange from an individual user perspective to want a range or
``latest`` - but from a library and framework perspective, things like shade
or terraform may have internal logic that can handle more than one version of
a service and want to use the best version available.
.. note:: Guidance around 'latest' is different from that found in
:ref:`the microversion specification
<microversion-client-interaction>`. It is acceptable for a client
library or framework to be interested in the latest version
available but such a specification is internal and not sent to
the server. In the client case with major versions, ``latest`` acts
as an input to the version discovery process.
``service-name``
Arbitrary name given to the service by the deployer. Optional.
.. note:: In all except the most extreme cases this should never be needed and
its use as a meaningful identifier by Deployers is strongly
discouraged. However, the Consumer has no way to otherwise mitigate
the situation if their Deployer has provided them with a catalog
where a ``service-name`` must be used, so ``service-name`` must be
accepted as input. If ``{be-strict}`` has been requested,
supplying ``{service-name}`` should be an error.
.. note:: In all except the most extreme cases this should never be needed
and its use as a meaningful identifier by Deployers is strongly
discouraged. However, the Consumer has no way to otherwise mitigate
the situation if their Deployer has provided them with a catalog
where a ``service-name`` must be used, so ``service-name`` must be
accepted as input.
If ``{be-strict}`` (see below) has been requested, a user supplying
``{service-name}`` should be an error.
service-id
``service-id``
Unique identifier for an endpoint in the catalog. Optional.
.. note:: On clouds with well-formed catalogs ``service-id`` should never be
needed. If ``{be-strict}`` has been requested, supplying
``{service-id}`` should be an error.
.. note:: On clouds with well-formed catalogs ``service-id`` should never be
needed. If ``{be-strict}`` has been requested, supplying
``{service-id}`` should be an error.
endpoint-override
``endpoint-override``
An endpoint for the service that the user has procured from some other
source. (Optional, defaults to omitted.)
At the end of the discovery process, the user should know the
``{service-endpoint}``, which is the endpoint to use as the root of the
service, and the ``{interface}`` of the endpoint that was found.
Discovery Behavior Modifiers
----------------------------
The user may also wish to express alterations to the general algorithm.
Implementations may present these flags under any name that makes sense,
or may choose to not present them as behavioral modification options at all.
``be-strict``
Forgo leniant backwards compatibility concessions and be more strict in
input and output validation. Defaults to False.
``skip-discovery``
If the user wants to completely skip the version discovery process even if
logic would otherwise do it. This is useful if the user has specified an
``{endpoint-override}`` or they know they just want to use whatever is in
the catalog and do not need additional metadata about the endpoint. Defaults
to False
``fetch-version-information``
If the user has specified an ``{endpoint-version}`` which can be known to
match just from looking at the URL, the version discovery process will not
fetch version information documents. However, the user may need the
information, such as microversion ranges. Using
``{fetch-version-information}`` allows them to request that the version
document be fetched even when an optimization in the process would otherwise
allow fetching the document to be skipped. Defaults to False.
Discovery Results
=================
At the end of the discovery process, the user should know the following:
If the process was successful:
* The actual values found for all of the input values above.
Found values will be referred to in these documents as ``found-{value}`` to
differentiate. So if a user requested an ``{endpoint-version}`` of
``latest``, ``{found-endpoint-version}`` might be ``3.5``.
* ``service-endpoint``
The endpoint to use as the root of the service.
* ``max-version``
If the service supports microversions, what is the maximum microversion the
service supports. Optional, defaults to omitted, which implies that
microversions are not supported.
* ``min-version``
If the service supports microversions, what is the minimum microversion the
service supports. Optional, defaults to omitted, which implies that
microversions are not supported.
If the process was unsuccessful, an error should be returned explaining which
part failed. For instance, was a matching service not found at all or was
a matching version not found. If a matching version was not found, the error
should contain a list of version that were found.
In the description that follows, each of the above inputs and outputs will
be referred to like ``{endpoint-override}`` so that it is clear whether a user
@ -107,459 +239,54 @@ referred to at a later point are similarly referred to like
``{service-catalog}``. Names will not be reused within the process to
hold different content at different times.
It is also assumed that the user has an ``{auth-url}`` and authentication
information. The authentication process itself is out of the scope of this
document.
Discovery Algorithm
===================
Services should be registered in the ``{service-catalog}`` using their
``{service-type}`` from the `OpenStack Service Types Authority`_. However,
for historical reasons there are some services that have old service types
found in the wild. To facilitate moving forward with the correct
``{service-type}`` names, but also support existing users and installations,
the `OpenStack Service Types Authority`_ contains a list of historical
aliases for such services. See `Consuming Service Types Authority`_ for
information on the data itself.
``{service-type}`` from the :doc:`OpenStack Service Types Authority
<consuming-catalog/authority>`. However, for historical reasons there are some
services that have old service types found in the wild. To facilitate moving
forward with the correct ``{service-type}`` names, but also support existing
users and installations, the OpenStack Service Types Authority contains a list
of historical aliases for such services.
Clients will need a copy of the data published in the
`OpenStack Service Types Authority`_ to be able to complete the full Discovery
OpenStack Service Types Authority to be able to complete the full Discovery
Algorithm. A client library could either keep a local copy or fetch the data
from https://service-types.openstack.org/service-types.json and potentially
cache it. It is recommended that client libraries handle consumption of the
historical data for their users but also allow some mechanism for the user to
provide a more up to date version of the data if necessary. See
`Consuming Service Types Authority`_ for information on how to fetch the data.
provide a more up to date verison of the data if necessary.
The basic process is:
#. If the user has provided ``{endpoint-override}``, STOP. This is the
``{service-endpoint}``.
#. Authenticate to keystone at the ``{auth-url}``, retreiving a ``token``
which contains the ``{service-catalog}``.
#. Retrieve ``{catalog-endpoint}`` from the ``{service-catalog}`` given
some combination of ``{service-type}``, ``{interface}``, ``{service-name}``,
``{region-name}`` and ``{service-id}``. (See :ref:`endpoint-from-catalog`.)
.. note:: This step is obviously skipped for clouds without authentication.
.. _endpoint-from-catalog:
#. If the user has provided ``{endpoint-override}``, it is used as
``{catalog-endpoint}``.
Endpoint from Catalog
=====================
#. If the user has not provided ``{endpoint-override}``, retrieve matching
``{catalog-endpoint}`` from the ``{service-catalog}`` using the procedure
explained in :doc:`consuming-catalog/endpoint`.
The ``{service-catalog}`` can be found in the ``token`` returned from
keystone authentication.
If v3 auth is used, the catalog will be in the ``catalog`` property of the
top-level ``token`` object. Such as:
.. code-block:: json
{
"token": {
"catalog": {}
}
}
If v2 auth is used it will be in the ``serviceCatalog`` property of the
top-level ``access`` object. Such as:
.. code-block:: json
{
"access": {
"serviceCatalog": {}
}
}
In both cases, the catalog content itself is a list of objects. Each object has
two main keys that concern discovery:
type
Matches ``{service-type}``
endpoints
List of endpoint objects for that service
Additionally, for backwards compatibility reasons, the following keys may
need to be checked.
name
Matches ``{service-name}``
id
Matches ``{service-id}``
The list of endpoints has a different format depending on whether v2 or v3 auth
was used. For both versions each endpoint object has a ``region`` key,
which should match ``{region-name}`` if one was given.
In v2 auth the endpoint object has three keys ``publicURL``,
``internalURL``, ``adminURL``. The endpoint for the ``{interface}`` requested
by the user is found in the key with the name matching ``{interface}`` plus
the string ``URL``.
In v3 auth the endpoint object has a ``url`` that is the endpoint that is
being requested if the value of ``interface`` matches ``{interface}``.
Concrete examples of tokens with catalogs:
V3 Catalog Objects:
.. code-block:: json
{
"token": {
"catalog": [
{
"endpoints": [
{
"id": "39dc322ce86c4111b4f06c2eeae0841b",
"interface": "public",
"region": "RegionOne",
"url": "https://identity.example.com"
},
{
"id": "ec642f27474842e78bf059f6c48f4e99",
"interface": "internal",
"region": "RegionOne",
"url": "https://identity.example.com"
},
{
"id": "c609fc430175452290b62a4242e8a7e8",
"interface": "admin",
"region": "RegionOne",
"url": "https://identity.example.com"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa2",
"type": "identity",
"name": "keystone"
}
],
}
V2 Catalog Objects:
.. code-block:: json
{
"access": {
"serviceCatalog": [
{
"endpoints_links": [],
"endpoints": [
{
"adminURL": "https://identity.example.com/v2.0",
"region": "RegionOne",
"publicURL": "https://identity.example.com/v2.0",
"internalURL": "https://identity.example.com/v2.0",
"id": "4deb4d0504a044a395d4480741ba628c"
}
],
"type": "identity",
"name": "keystone"
},
]
}
}
The algorithm is:
#. Find the objects in the ``{service-catalog}`` that match the requested
``{service-type}``. (See `Match Candidate Entries`_.)
#. If ``{service-name}`` was given and the objects remaining have a ``name``
field, keep only the ones where ``name`` matches ``{service-name}``.
.. note:: Catalogs from Keystone v3 before v3.3 do not have a name field. If
``{be-strict}`` was not requested and the catalog does not have a
``name`` field, ``{service-name}`` should be ignored.
#. If ``{service-id}`` was given and the objects remaining have a ``id``
field, keep only the ones where ``id`` matches ``{service-id}``.
.. note:: Catalogs from Keystone v2 do not have an id field. If
``{be-strict}`` was not requested and the catalog does not have a
``id`` field, ``{service-id}`` should be ignored.
The list of remaining objects are the ``{candidate-catalog-objects}``. If there
are no endpoints, return an error that there are no endpoints matching
``{service-type}`` and ``{service-name}``.
Use ``{candidate-catalog-objects}`` to produce the list of
``{candidate-endpoints}``.
For each endpoint object in each of the ``{candidate-catalog-objects}``:
#. If v2, if there is no key of the form ``{interface}URL`` for any of the
the ``{interface}`` values given, discard the endpoint.
#. If v3, if ``interface`` does not match any of the ``{interface}`` values
given, discard the endpoint.
If there are no endpoints left, return an error that there are no endpoints
matching any of the ``{interface}`` values, preferrably including the list of
interfaces that were found.
For each remaining endpoint in ``{candidate-endpoints}``:
#. If ``{region_name}`` was given and does not match either of ``region``
or ``region_id``, discard the endpoint.
If there are no remaining endpoints, return an error that there are no
endpoints matching ``{region_name}``, preferrably including the list of
regions that were found.
#. From the set of remaining candidate endpoints, find the ones that best
matches the requested ``{service-type}``.
(See `Find Endpoint Matching Best Service Type`_.)
The remaining ``{candidate-endpoints}`` match the request. If there is more
than one of them, use the first, but emit a warning to the user that more
than one endpoint was left. If ``{be-strict}`` has been requested, return an
error instead with information about each of the endpoints left in the list.
.. note:: It would be more correct to raise an error if there is more than one
endpoint left, but the keystoneauth library returns the first and
changing that would break a large number of existing users. If one
is writing a completely new library from scratch, or a new major
version where behavior change is acceptable, it is preferable to
raise an error here if there is more than one endpoint left.
#. If v2, the ``{catalog-endpoint}`` is the value of ``{interface}URL``.
#. If v3, the ``{catalog-endpoint}`` is the value of ``url``.
Match Candidate Entries
-----------------------
For every entry in the catalog:
#. If the entry's type matches the requested ``{service-type}``, it is a
candidate.
#. If the requested type is an official type from the
`OpenStack Service Types Authority`_ that has aliases and one of the aliases
matches the entry's type, it is a candidate.
#. If the requested type is an alias of an official type from the
`OpenStack Service Types Authority`_ and the entry's type matches the
official type, it is a candidate.
.. note:: Requesting one alias and finding a different alias is not supported
at this point because most aliases carry implied information about
major versions as well. A subsequent spec adds the process for
version discovery at which point it can be safe to attempt to return
an endpoint listed under an alias different than what was requested.
Find Endpoint Matching Best Service Type
----------------------------------------
Given a list of candidate endpoints that have matched the other criteria:
#. Check the list of candidate endpoints to see if one of them matches the
requested ``{service-type}``. If any are an exact match,
`Find Endpoint Matching Best Interface`_.
#. If the requested ``{service-type}`` is an official type in the
`OpenStack Service Types Authority`_ that has aliases, check each alias
in order of preference as listed in the Authority to see if it has a
matching endpoint from the candidate endpoints. For all endpoints that
match the first alias with matching endpoints,
`Find Endpoint Matching Best Interface`_.
#. If ``{skip-discovery}`` is true, STOP and use ``{catalog-endpoint}`` as
``{service-endpoint}``. Otherwise, discover the available API versions
and find the suitable ``{service-endpoint}`` using the version discovery
procedure from :doc:`consuming-catalog/version-discovery`.
#. If the requested ``{service-type}`` is an alias of an official type in the
`OpenStack Service Types Authority`_ and any endpoints match the official
type, `Find Endpoint Matching Best Interface`_.
OpenStack Service Types Authority and any endpoints match the official
type, :ref:`find-endpoint-matching-best-service-type`.
Find Endpoint Matching Best Interface
-------------------------------------
Given a list of candidate endpoints that have matched the other criteria:
Table of Contents
=================
#. In order of preference of ``{interface}`` list, return all endpoints that
match the first ``{interface}`` with matching endpoints.
.. toctree::
For example, given the following catalog:
.. code-block:: json
{
"token": {
"catalog": [
{
"endpoints": [
{
"interface": "public",
"region": "RegionOne",
"url": "https://block-storage.example.com/v3"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa3",
"type": "volumev3",
"name": "cinder"
},
{
"endpoints": [
{
"interface": "public",
"region": "RegionOne",
"url": "https://block-storage.example.com/v2"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa2",
"type": "volumev2",
"name": "cinder"
}
],
}
Then the following:
::
service_type = 'block-storage'
# block-storage is not found, get list of aliases
# volumev3 is found, return it
service_type = 'volumev2'
# volumev2 not an official type in authority, but is in catalog
# return volumev2 entry
service_type = 'volume'
# volume not in authority or catalog
# volume is an alias of block-storage
# block-storage is not found. Return error.
Given the following catalog:
.. code-block:: json
{
"token": {
"catalog": [
{
"endpoints": [
{
"interface": "public",
"region": "RegionOne",
"url": "https://block-storage.example.com"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa3",
"type": "block-storage",
"name": "cinder"
}
],
}
Then the following:
::
service_type = 'block-storage'
# block-storage is found, return it
service_type = 'volumev2'
# volumev2 not in authority, is an alias for block-storage
# block-storage is in the catalog, return it
Given the following catalog:
.. code-block:: json
{
"token": {
"catalog": [
{
"endpoints": [
{
"interface": "public",
"region": "RegionOne",
"url": "https://block-storage.example.com"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa3",
"type": "block-storage",
"name": "cinder"
},
{
"endpoints": [
{
"interface": "public",
"region": "RegionOne",
"url": "https://block-storage.example.com/v2"
},
{
"interface": "internal",
"region": "RegionOne",
"url": "https://block-storage.example.int/v2"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa2",
"type": "volumev2",
"name": "cinder"
}
],
}
Then the following:
::
service_type = 'block-storage'
interface = ['internal', 'public']
# block-storage is found
# block-storage does not have internal, but has public
# return block-storage public
service_type = 'volumev2'
interface = ['internal', 'public']
# volumev2 not an official type in authority, but is in catalog
# volumev2 has an internal interface
# return volumev2 internal entry
Consuming Service Types Authority
=================================
The `OpenStack Service Types Authority`_ is data about official service type
names and historical service type names commonly in use from before there was
an official list. It is made available to allow libraries and other client
API consumers to be able to provide a consistent interface based on the
official list but still support existing names. Providing this support is
highly recommended, but is ultimately optional. The first step in the matching
process is always to return direct matches between the catalog and the user
request, so the existing consumption models from before the existence of the
authority should always work.
In order to consume the information in the `OpenStack Service Types Authority`_
it is important to know a few things:
#. The data is maintained in YAML format in git. This is the ultimately
authoritative source code for the list.
#. The data is published in JSON format at
https://service-types.openstack.org/service-types.json and has a JSONSchema
at https://service-types.openstack.org/published-schema.json.
#. The published data contains a version which is date based in
`ISO Date Time Format`_, a sha which contains the git sha of the
commit the published data was built from, and pre-built forward and reverse
mappings between official types and aliases.
#. The JSON file is served with ETag support and should be considered highly
cacheable.
#. The current version of the JSON file should always be the preferred file to
use.
#. The JSON file is similar to timezone data. It should not be considered
versioned such that stable releases of distros should provide a
frozen version of it. Distro packages should instead update for all
active releases when a new version of the file is published.
.. _OpenStack Service Types Authority: https://opendev.org/openstack/service-types-authority/
.. _ISO Date Time Format: https://tools.ietf.org/html/rfc3339#section-5.6
consuming-catalog/endpoint
consuming-catalog/version-discovery
consuming-catalog/authority

42
guidelines/consuming-catalog/authority.rst Normal file

@ -0,0 +1,42 @@
Consuming Service Types Authority
=================================
The `OpenStack Service Types Authority`_ is data about official service type
names and historical service type names commonly in use from before there was
an official list. It is made available to allow libraries and other client
API consumers to be able to provide a consistent interface based on the
official list but still support existing names. Providing this support is
highly recommended, but is ultimately optional. The first step in the matching
process is always to return direct matches between the catalog and the user
request, so the existing consumption models from before the existence of the
authority should always work.
In order to consume the information in the `OpenStack Service Types Authority`_
it is important to know a few things:
#. The data is maintained in YAML format in git. This is the ultimately
authoritative source code for the list.
#. The data is published in JSON format at
https://service-types.openstack.org/service-types.json and has a JSONSchema
at https://service-types.openstack.org/published-schema.json.
#. The published data contains a version which is date based in
`ISO Date Time Format`_, a sha which contains the git sha of the
commit the published data was built from, and pre-built forward and reverse
mappings between official types and aliases.
#. The JSON file is served with ETag support and should be considered highly
cacheable.
#. The current version of the JSON file should always be the preferred file to
use.
#. The JSON file is similar to timezone data. It should not be considered
versioned such that stable releases of distros should provide a
frozen version of it. Distro packages should instead update for all
active releases when a new version of the file is published.
.. _OpenStack Service Types Authority: https://opendev.org/openstack/service-types-authority/
.. _ISO Date Time Format: https://tools.ietf.org/html/rfc3339#section-5.6

471
guidelines/consuming-catalog/endpoint.rst Normal file

@ -0,0 +1,471 @@
Endpoint Discovery
==================
Endpoint from Catalog
---------------------
The ``{service-catalog}`` can be found in the ``token`` returned from
keystone authentication.
If v3 auth is used, the catalog will be in the ``catalog`` property of the
top-level ``token`` object. Such as:
.. code-block:: json
{
"token": {
"catalog": {}
}
}
If v2 auth is used it will be in the ``serviceCatalog`` property of the
top-level ``access`` object. Such as:
.. code-block:: json
{
"access": {
"serviceCatalog": {}
}
}
In both cases, the catalog content itself is a list of objects. Each object has
two main keys that concern discovery:
``type``
Matches ``{service-type}``
``endpoints``
List of endpoint objects for that service
Additionally, for backwards compatibility reasons, the following keys may
need to be checked.
``name``
Matches ``{service-name}``
``id``
Matches ``{service-id}``
The list of endpoints has a different format depending on whether v2 or v3 auth
was used. For both versions each endpoint object has a ``region`` key,
which should match ``{region-name}`` if one was given.
In v2 auth the endpoint object has three keys ``publicURL``,
``internalURL``, ``adminURL``. The endpoint for the ``{interface}`` requested
by the user is found in the key with the name matching ``{interface}`` plus
the string ``URL``.
In v3 auth the endpoint object has a ``url`` that is the endpoint that is
being requested if the value of ``interface`` matches ``{interface}``.
Examples of Tokens with Catalogs
--------------------------------
V3 Catalog Objects:
.. code-block:: json
{
"token": {
"catalog": [
{
"endpoints": [
{
"id": "39dc322ce86c4111b4f06c2eeae0841b",
"interface": "public",
"region": "RegionOne",
"url": "https://identity.example.com"
},
{
"id": "ec642f27474842e78bf059f6c48f4e99",
"interface": "internal",
"region": "RegionOne",
"url": "https://identity.example.com"
},
{
"id": "c609fc430175452290b62a4242e8a7e8",
"interface": "admin",
"region": "RegionOne",
"url": "https://identity.example.com"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa2",
"type": "identity",
"name": "keystone"
}
],
}
V2 Catalog Objects:
.. code-block:: json
{
"access": {
"serviceCatalog": [
{
"endpoints_links": [],
"endpoints": [
{
"adminURL": "https://identity.example.com/v2.0",
"region": "RegionOne",
"publicURL": "https://identity.example.com/v2.0",
"internalURL": "https://identity.example.com/v2.0",
"id": "4deb4d0504a044a395d4480741ba628c"
}
],
"type": "identity",
"name": "keystone"
},
]
}
}
Endpoint Discovery Algorithm
----------------------------
#. If ``{endpoint-version}`` was given and ``{service-type}`` ends with a
suffix of ``v[0-9]+$`` and ``{endpoint-version}`` does not match that suffix
(see `Comparing Major Versions`_), STOP. Return an error that the user
has requested a versioned ``{service-type}`` alias and an incompatible
``{endpoint-version}``.
#. Find the objects in the ``{service-catalog}`` that match the requested
``{service-type}`` (see `Match Candidate Entries`_).
#. If ``{service-name}`` was given and the objects remaining have a ``name``
field, keep only the ones where ``name`` matches ``{service-name}``.
.. note:: Catalogs from Keystone v3 before v3.3 do not have a name field. If
``{be-strict}`` was not requested and the catalog does not have a
``name`` field, ``{service-name}`` should be ignored.
#. If ``{service-id}`` was given and the objects remaining have a ``id``
field, keep only the ones where ``id`` matches ``{service-id}``.
.. note:: Catalogs from Keystone v2 do not have an id field. If
``{be-strict}`` was not requested and the catalog does not have a
``id`` field, ``{service-id}`` should be ignored.
The list of remaining objects are the ``{candidate-catalog-objects}``. If this
list is empty, return an error that there are no endpoints matching
``{service-type}`` and ``{service-name}``.
#. Use ``{candidate-catalog-objects}`` to produce the list of
``{candidate-endpoints}``. For each endpoint object in each of the
``{candidate-catalog-objects}``:
#. If v2, if there is no key of the form ``{interface}URL`` for any of the
the ``{interface}`` values given, discard the endpoint.
#. If v3, if ``interface`` does not match any of the ``{interface}`` values
given, discard the endpoint.
#. If there are no endpoints left, return an error that there are no endpoints
matching any of the ``{interface}`` values, preferrably including the list
of interfaces that were found.
#. For each remaining endpoint in ``{candidate-endpoints}``, if
``{region_name}`` was given and does not match either of ``region`` or
``region_id``, discard the endpoint.
If there are no remaining endpoints, return an error that there are no
endpoints matching ``{region_name}``, preferrably including the list of
regions that were found.
#. From the set of remaining candidate endpoints, find the ones that best
matches the requested ``{service-type}`` (see `Find Endpoint Matching Best
Service Type`_).
#. From the set of remaining candidate endpoints, find the ones that best
matches the best available requested ``{interface}``: in order of
preference of the ``{interface}`` list, return all endpoints that match
the first ``{interface}`` that has any matching endpoints.
The remaining ``{candidate-endpoints}`` match the request. If there is more
than one of them, use the first, but emit a warning to the user that more
than one endpoint was left. If ``{be-strict}`` has been requested, return an
error instead with information about each of the endpoints left in the list.
.. note:: It would be more correct to raise an error if there is more than one
endpoint left, but the keystoneauth library returns the first and
changing that would break a large number of existing users. If one
is writing a completely new library from scratch, or a new major
version where behavior change is acceptable, it is preferable to
raise an error here if there is more than one endpoint left.
#. If v2, the ``{catalog-endpoint}`` is the value of ``{interface}URL``.
#. If v3, the ``{catalog-endpoint}`` is the value of ``url``.
Match Candidate Entries
~~~~~~~~~~~~~~~~~~~~~~~
For every entry in the catalog:
#. If the entry's type matches the requested ``{service-type}``, it is a
candidate.
#. If the requested type is an official type from the
:doc:`OpenStack Service Types Authority <authority>` that has aliases and
one of the aliases matches the entry's type, it is a candidate.
#. If the requested type is an alias of an official type from the
:doc:`OpenStack Service Types Authority <authority>` and the entry's type
matches the official type, it is a candidate.
#. If the requested type is an alias of an official type from the
:doc:`OpenStack Service Types Authority <authority>` that has aliases and
the entry's type matches one of the aliases and ``{endpoint-version}`` was
given and the found alias ends with a suffix of ``v[0-9]+$`` and
``{endpoint-version}`` matches the version in the suffix (see `Comparing
Major Versions`_) it is a candidate.
.. _find-endpoint-matching-best-service-type:
Find Endpoint Matching Best Service Type
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Given a list of candidate endpoints that have matched the other criteria:
#. Check the list of candidate endpoints to see if one of them matches the
requested ``{service-type}``. If any are an exact match, return them.
#. If the requested ``{service-type}``
* is an official type from the :doc:`OpenStack Service Types Authority
<authority>` that has aliases
* ``{endpoint-version}`` was given
Look for aliases that end with a version suffix of the form ``v[0-9]+$``.
If there are any aliases with a version suffix that matches the
``{endpoint-version}`` (see `Comparing Major Versions`_), look for those
aliases in the list of candidate endpoints. If any are a match, return them.
#. If the requested ``{service-type}``
* is an official type in the :doc:`OpenStack Service Types Authority
<authority>` that has aliases
* ``{endpoint-version}`` was not given
check each alias in the order listed to see if it has a matching endpoint
from the candidate endpoints. Return the endpoints that match the first
alias that has matching endpoints.
#. If the requested ``{service-type}``
* is an alias of an official type in the
:doc:`OpenStack Service Types Authority <authority>`
* ``{endpoint-version}`` was given
look for aliases that end with a version suffix of the form ``v[0-9]+$``. If
there are any aliases with a version suffix that matches the
``{endpoint-version}`` (see `Comparing Major Versions`_), look for those
aliases in the list of candidate endpoints.
Return the endpoints that match the alias with the highest matching version.
#. If there are no matching endpoints, return an error.
.. note:: The case where
* an alias was requested
* no ``{endpoint-version}`` was given
* there is a different alias in the catalog
is not safe and so is treated as a lack of matching endpoint on
purpose. Many of the aliases carry an implied version, so absent
a requested ``{endpoint-version}`` from the user, returning
an endpoint different than the one explicitly requested has a high
chance of not being the endpoint the user expected.
.. _comparing-major-versions:
Comparing Major Versions
~~~~~~~~~~~~~~~~~~~~~~~~
When comparing Major Versions, there is a ``required`` and a ``candidate``:
* The ``required`` is what the user has requested.
* The ``candidate`` is the possible version being tested.
To be suitable a ``candidate`` must be of the same major version as
``required`` and be at least a match in minor level: ``candidate`` ``3.3``
is a match for ``required`` ``3.1`` but ``4.1`` is not.
Leading 'v' strings should be discarded in all cases.
#. Versions with only a single number normalize to ``.0``. That is,
a version of ``2`` should be treated as if it was ``2.0``.
#. If ``required`` is the string ``latest`` or contains no value, ``candidate``
matches.
#. If ``required`` is a range, any ``candidate`` that is greater than or equal
to the first value and less than or equal to the second value is a match.
Equality is judged by the above rules. Greater than and less than are judged
as expected: first by comparing the first number, and if those match then by
comparing the second number. Thus, a ``{required}`` of ``2,4`` matches
``2``, ``2.3``, ``3``, ``4`` and ``4.7``. A ``{required}`` of ``2.1,4.0``
matches ``2.3``, ``3``, ``4`` and ``4.7`` but not ``2``.
#. If ``required`` is a range without a maximum value, maximum should be
treated as if it is ``latest``.
Examples of discovery
---------------------
For example, given the following catalog:
.. code-block:: json
{
"token": {
"catalog": [
{
"endpoints": [
{
"interface": "public",
"region": "RegionOne",
"url": "https://block-storage.example.com/v3"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa3",
"type": "volumev3",
"name": "cinder"
},
{
"endpoints": [
{
"interface": "public",
"region": "RegionOne",
"url": "https://block-storage.example.com/v2"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa2",
"type": "volumev2",
"name": "cinder"
}
],
}
Then the following:
::
service_type = 'block-storage'
# block-storage is not found, get list of aliases
# volumev3 is found, return it
service_type = 'volumev2'
# volumev2 not an official type in authority, but is in catalog
# return volumev2 entry
service_type = 'volume'
# volume not in authority or catalog
# volume is an alias of block-storage
# block-storage is not found. Return error.
service_type = 'volume'
api_version = 2
# volume not in authority or catalog
# volume is an alias of block-storage
# block-storage is not found.
# volumev2 is an alias of block-storage and ends with v2 which matches
# api_version of 2
# return volumev2
Given the following catalog:
.. code-block:: json
{
"token": {
"catalog": [
{
"endpoints": [
{
"interface": "public",
"region": "RegionOne",
"url": "https://block-storage.example.com"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa3",
"type": "block-storage",
"name": "cinder"
}
],
}
Then the following:
::
service_type = 'block-storage'
# block-storage is found, return it
service_type = 'volumev2'
# volumev2 not in authority, is an alias for block-storage
# block-storage is in the catalog, return it
service_type = 'volumev2'
api_version = '3'
# volumev2 ends with a version suffix of v2 which does not match 3
# return an error before even fetching the catalog
Given the following catalog:
.. code-block:: json
{
"token": {
"catalog": [
{
"endpoints": [
{
"interface": "public",
"region": "RegionOne",
"url": "https://block-storage.example.com"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa3",
"type": "block-storage",
"name": "cinder"
},
{
"endpoints": [
{
"interface": "public",
"region": "RegionOne",
"url": "https://block-storage.example.com/v2"
},
{
"interface": "internal",
"region": "RegionOne",
"url": "https://block-storage.example.int/v2"
}
],
"id": "4363ae44bdf34a3981fde3b823cb9aa2",
"type": "volumev2",
"name": "cinder"
}
],
}
Then the following:
::
service_type = 'block-storage'
interface = ['internal', 'public']
# block-storage is found
# block-storage does not have internal, but has public
# return block-storage public
service_type = 'volumev2'
interface = ['internal', 'public']
# volumev2 not an official type in authority, but is in catalog
# volumev2 has an internal interface
# return volumev2 internal entry

1068
guidelines/consuming-catalog/version-discovery.rst Normal file

File diff suppressed because it is too large Load Diff

2
guidelines/discoverability.rst

@ -8,6 +8,8 @@ API expose the URIs and resources to end users in a machine-readable way.
See also the topic document on :ref:`consuming-catalog`.
See also the topic document on :doc:`consuming-catalog/version-discovery`.
Guidance
--------

2
guidelines/microversion_specification.rst

@ -50,6 +50,8 @@ For example, you cannot request the feature which was introduced at
microversion 2.100 without backwards incompatible changes which were
introduced in microversion 2.99 and earlier.
.. _microversion-client-interaction:
Client Interaction
------------------