openstack
/
requirements
Code Issues Proposed changes

Allow backports to have arbitrary python_version markers 

There are a number of backport packages in place for packages in stdlib
that are still evolving. One such package is 'importlib-metadata' which
is a backport of 'importlib.metadata'. The README [1] for this package
shows a mapping of versions of the 'importlib-metadata' third party
library to versions of stdlib 'importlib.metadata':

  importlib_metadata  stdlib
  4.8                 3.11
  4.4                 3.10
  1.4                 3.8

A project may want to use a shiny new feature only found in the Python
3.11 stdlib version while continuing to support Python 3.8, 3.9 and
3.10. If so, they could specify something like so:

  importlib-metadata>=4.8;python_version<3.11

Meanwhile, other packages might require features found in the Python
3.10 stdlib version. They would want to specify something like so:

  importlib-metadata>=4.4;python_version<3.10

Currently this is not possible as it is not possible to specify
arbitrary 'python_version' markers. Since 'importlib-metadata' does not
have a 'python_version' marker in 'global-requirements.txt' (not since
change I5febaed02e95ff27accd946abc32f3bcbb1a5ead anyway), both projects
would have to specify

  importlib-metadata>={required_version}

Which means they'll use importlib-metadata even where they don't need
it.

Fix this issue by allowing a list of particular modules to specify a
project-specific 'python_version' marker. Currently this list only
includes 'importlib-metadata', but in the future we may wish to enhance
it to include other rolling backport libraries (e.g. mock).

[1] https://pypi.org/project/importlib-metadata/

Change-Id: I85501b4bff97d1c1e1873c3329ef998a8e501134
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This commit is contained in:
Stephen Finucane 2022-07-15 15:42:19 +01:00 committed by Tony Breeds
parent bed46f4b7a
commit a968919ebe
1 changed files with 30 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

39
openstack_requirements/check.py
View File

@ -25,6 +25,9 @@ from openstack_requirements import requirement
MIN_PY_VERSION = '3.5'
PY3_SPECIFIER_RE = re.compile(r'python_version(==|>=|>)[\'"]3\.\d+[\'"]')
BACKPORTS = {
'importlib-metadata',
}
class RequirementsList(object):
@ -98,18 +101,36 @@ def _is_requirement_in_global_reqs(local_req, global_reqs, allow_3_only=False):
if local_req_val != global_req_val:
# if global requirements specifies a python 3 version specifier
# but a project doesn't, allow it since python 3-only is okay
if (allow_3_only and matching and
aname == 'markers' and not local_req_val):
if (
allow_3_only and
matching and
aname == 'markers' and
not local_req_val
):
if PY3_SPECIFIER_RE.match(global_req_val):
continue
print('WARNING: possible mismatch found for package '
'"{}"'.format(local_req.package))
print(' Attribute "{}" does not match'.format(aname))
print(' "{}" does not match "{}"'.format(
local_req_val, global_req_val))
print(' {}'.format(local_req))
print(' {}'.format(global_req))
# likewise, if a package is one of the backport packages then
# we're okay with a potential marker (e.g. if a package
# requires a feature that is only available in a newer Python
# library, while other packages are happy without this feeature
if (
allow_3_only and
matching and
aname == 'markers' and
local_req.package in BACKPORTS
):
if (
PY3_SPECIFIER_RE.match(global_req_val) or
PY3_SPECIFIER_RE.match(local_req_val)
):
continue
print(f'WARNING: possible mismatch found for package "{local_req.package}"') # noqa: E501
print(f' Attribute "{aname}" does not match')
print(f' "{local_req_val}" does not match "{global_req_val}"') # noqa: E501
print(f' {local_req}')
print(f' {global_req}')
matching = False
if not matching:
continue