Fix "dictionary changed size during iteration"

We have hit an error with
"dictionary changed size during iteration" a few times during
this week in telemetry integration tests. It seems like it's
hitting only our ubuntu based jobs, I haven't seen this error
in a centos based job yet. Example of the failed job can be
found in [1], I extracted a traceback, which leads to keystoneauth1
into [2]. According to [3] and [4] using copy() should help with
the issue. The python docs [5] indicate, that copy() should always
be used when iterating through sys.modules

[1] https://zuul.opendev.org/t/openstack/build/c99db592871a441e9cddad2f4e60c2fc/console
[2] https://paste.opendev.org/show/bpzng2EUyFh1tvBHczt7/
[3] https://github.com/python/cpython/issues/84507
[4] https://github.com/python/cpython/issues/89516
[5] https://docs.python.org/3/library/sys.html#sys.modules

Change-Id: I50500c6a21bbe60050303cea4628ca9b71a3e0eb
This commit is contained in:
Jaromir Wysoglad 2023-11-09 09:13:49 -05:00
parent 44e477dfb5
commit 3b492a7aa0
1 changed files with 6 additions and 1 deletions

View File

@ -153,7 +153,12 @@ def _determine_calling_package():
# module does this but is far less efficient. Same story with the
# frame walking below. One could use ``inspect.stack()`` but it
# has far more overhead.
mod_lookup = dict((m.__file__, n) for n, m in sys.modules.items()
# NOTE(jwysogla): According to the docs, we should always use copy(),
# because sys.modules can change during iteration, which results
# in a RuntimeError
# https://docs.python.org/3/library/sys.html#sys.modules
mod_lookup = dict((m.__file__, n) for n, m in sys.modules.copy().items()
if hasattr(m, '__file__'))
# NOTE(shaleh): these are not useful because they hide the real