In past days there were discussions about various issues
with memcached connections [1][2][3].
After investigation it looks like common root cause for above
problems is keystonemiddleware. More precisely said the way
how keystonemiddleware is caching tokens.
Currently it's using some home-made CachePool with direct
usage of memcached library, moreover it looks like its
approach is not eventlet-safe.
Discussion can be mainly found in [4].
Fortunately keystonemiddleware can use "advanced cache pool",
which is oslo.cache's implementation and was added long time ago [5],
but it is turned on only if memcache_use_advanced_pool=True.
This patch is switching to more elaborated oslo.cache CachePool
and adding deprecation warning about eventlet-unsafe variant
of keystonemiddleware's memcache pool.
How to reproduce ?
with memcache_use_advanced_pool=False
1. Build clean ENV of openstack
2. Deploy core projects (keystone,glance,nova,placement...)
3. Run while true; do COMMAND FOR SERVICE; done
- several bashes, in parallel (5-7)
COMMAND FOR SERVICE:
- openstack network list
- openstack volume list
- openstack server list
- openstack image list
4. Check memcached connections (which will grow up):
- ss | grep 11211 | wc -l every second
How to fix and test it ?
Repeat above, to fix:
- with memcache_use_advanced_pool=True
OR
- apply this patch
Compare measurements in graph.
[1] https://bugs.launchpad.net/keystonemiddleware/+bug/1892852
[2] https://bugs.launchpad.net/oslo.cache/+bug/1888394
[3] https://bugs.launchpad.net/keystonemiddleware/+bug/1883659
[4] https://review.opendev.org/c/openstack/oslo.cache/+/742193
[5] https://review.opendev.org/c/openstack/keystonemiddleware/+/268664
Closes-Bug: #1883659
Closes-Bug: #1892852
Closes-Bug: #1888394
Change-Id: I0e96334b65a0bf369ebf1d88651d13feb8d2ecac
In [0] the ``interface``option was added in order to allow the Identity
endpoint that is being used when validating tokens to be
configured by the deployer. Change the default to using the internal
endpoint, as that should be what most deployments will end up using.
[0] https://review.opendev.org/651790
Depends-On: https://review.opendev.org/651492
Closes-Bug: 1830002
Change-Id: I0ce8b6d8cd408c7fac8107972e7be70839e337fb
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* parallelizing building of documents
Update Sphinx version as well.
Remove the doc requirements from lower-constraints, they are not
needed during installation.
openstackdocstheme renames some variables, so follow the renames. A
couple of variables are also not needed anymore, remove them.
Set openstackdocs_pdf_link to link to PDF file.
Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.
See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html
Change-Id: Ic7c901ff19aa073b6e003ccb95aaf77886f20152
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: Idc319f3f8a3ddd57cba91e4cefc66dbb18d5cc22
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Add file to the reno documentation build to show release notes for
stable/ussuri.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/ussuri.
Change-Id: Iedcc2750dc0bdfdabb9d03a0b153aeeae6c0b58e
Sem-Ver: feature
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Blacklist:
W504 line break after binary operator
W503 line break before binary operator
Fix other problems found
Change-Id: I2fb257a4f42b499df3702f3e8f3c99ecb28557d6
We literally say in (the rendered version of) the same doc:
# Authentication type to load (string value)
# Deprecated group/name - [keystone_authtoken]/auth_plugin
#auth_type = <None>
Looks like auth_plugin has been deprecated for quite some time:
https://opendev.org/openstack/keystoneauth/commit/a56ed4218
Change-Id: I2dafa0cb28f017667497e0a6585d96a8cd090d5f
This change renames the _v3_to_v2_catalog to _normalize_catalog as
part of the v2 removal effort. Several services still rely on the
converted catalog format, so this change maintains the conversion
but removes the v2 association.
Change-Id: Ic7bca16d8c6211d006fc2ba09dc2ecd83f8955db
Partial-Bug: #1845539
Partial-Bug: #1777177
This change modifies any URLs specifying v2.0 to v3. This is part
of the effort to remove v2.0 functionality from keystonemiddleware.
Change-Id: I9cde8963333ea95b4ab05d9aea4d196ab4357763
Partial-Bug: #1829453
Partial-Bug: #1845539
This change removes v2.0 functionality from
keystonemiddleware, as well as associated tests.
Partial-Bug: #1845539
Partial-Bug: #1777177
Change-Id: If47e90085d8a59c52fb23876dc329cd4f0b05ef0
This change replaces the usage of keystoneclient exceptions in the
auth_token_middleware unit tests to use the ConnectFailure exception
from keystoneauth.
This is part of the process of removing keystoneclient from
keystonemiddleware.
Change-Id: Ie00fc310728c319faf2cfdfb651f0c7a8f48d757
Some regex strings contain invalid escape sequences for normal strings,
causing newer version of Python to emit DeprecationWarning messages.
This updates those instances to raw strings so they are not interpreted
as invalid.
Change-Id: I28ac26516bacab36578a5a7f6ec7f9dcf7d7eeb1
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Add file to the reno documentation build to show release notes for
stable/train.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/train.
Change-Id: Id57d4939da1ea27351d5a5dc5542f24e0abea789
Sem-Ver: feature
Background:
As part of my work on reproducible builds for openSUSE, I check that software still gives identical build results in the future.
The usual offset is +15 years, because that is how long I expect some software will be used in some places.
This showed up failing tests in our package build.
See https://reproducible-builds.org/ for why this matters.
This makes it expire 1 year in the future to model realistic tokens.
Change-Id: I73bde68be53afff4e8dff12d756b8381f34b2adb
Currently with sphinx 2.2.0 the docs job is throwing a warning
that the html_static_path entry does not exist. We treat warnings
as errors so this causes the job to fail.
This change comments the html_static_path entry in conf.py since
the path currently does not exist so it appears to be unused.
Change-Id: Ib2c74f4f37855cec250d09b23c45b5b7fde44c8d
Some options are now automatically configured by the version 1.20:
- project
- html_last_updated_fmt
- latex_engine
- latex_elements
- version
- release.
Change-Id: I161a3983e23b0ae50c232eb63ca78f8fd230e91e
This commit adds a validation step in the auth_token middleware to check
for the presence of an access_rules attribute in an application
credential token and to validate the request against the permissions
granted for that token. During token validation it sends a header to
keystone to indicate that it is capable of validating these access
rules, and not providing this header for a token like this would result
in the token failing validation. This disregards access rules for a
service request made by a service on behalf of a user, such as nova
making a request to glance, because such a request is not under the
control of the user and is not expected to be explicitly allowed in the
access rules.
bp whitelist-extension-for-app-creds
Depends-On: https://review.opendev.org/670377
Change-Id: I185e0541d5df538d74edadf9976b3034a2470c88
Keystone server no longer supports PKI/PKIZ. This change removes
keystonemiddleware's support of PKI/PKIZ and associated code.
Change-Id: I9a6639a2aa3774be61972d57f38220f66fd5c0e8
closes-bug: #1649735
partial-bug: #1736985
OpenStack Proposal Bot
Michal Arbet
Zuul
Ghanshyam Mann
Jens Harbott
Andreas Jaeger
Sean McGinnis
OpenStack Release Bot
Tim Burke
Ben Nemec
Vishakha Agarwal
Gage Hugo
pengyuesheng
Q.hongtao
Bernhard M. Wiedemann
Colleen Murphy
Corey Bryant
Colleen Murphy
Lance Bragstad
Morgan Fainberg