These are detected as errors since the clean up was done[1] in
the requirements repository.
[1] 314734e938f107cbd5ebcc7af4d9167c11347406
Change-Id: I6970540de396e4aa77590aadfdfa72bf3f1862f3
This library no longer supports Python 2, thus usage of six can be
removed. This also removes workaround about pickle library used in
Python 2 only.
Change-Id: I19d298cf0f402d65f0b142dea0bf35cf992332a9
Introduced changes:
- pre-commit config and rules.
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.
Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.
pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker
(check-byte-order-marker);
- Checks that non-binary executables have a proper
shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings
(check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)
For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks
Change-Id: I3640bb690c552a428ae0e8250adffbc3a51a310e
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems
Update Sphinx version as well.
Disable openstackdocs_auto_name to use 'project' variable as name.
Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.
Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.
openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.
See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html
Change-Id: Ieb7ce05f6611ad743336a35065402888176c7de3
Sphinx 2.0 no longer works on python 2.7, so we need to start capping it
there. We remove Sphinx and doc8 from the 'test-requirements.txt' file
and '[extra] test' section in 'setup.cfg' since none of our tests
actually use this. Finally, we remove a number of dependencies from
'doc/requirements.txt' since these were already specified in
'test-requirements.txt' and we install these as part of the tox target.
Change-Id: I3981f47f4278f726e3c078723f90d4b9ba1699a6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The autodoc_default_flags option is now deprecated.[1] This warning can
cause documentation build failed. use autodoc_default_options instead.
[1]https://github.com/matplotlib/matplotlib/pull/11928
Change-Id: I6ba69ca7361fd0e42b322f1b92fe22b952a13321
There is a wrong use ' here, which causes the document
to make errors when copying the code for
experimentation.
Change-Id: I3ae19f591ffd668bcd1563bd1e317b21b799dba6
With the release of NetworkX 2.0 the reporting API was
moved to view/iterator model. Many methods were moved from
reporting lists or dicts to iterating over the information.
Methods that used to return containers now return views and
methods that returned iterators have been removed in networkx.
Because of this change in NetworkX 2.0 , taskflow code
have to be changed also to support networkx > 2.0
Change-Id: I23c226f37bd85c1e38039fbcb302a2d0de49f333
Closes-Bug: #1778115
Most notably, taskflow is hitting the sphinx issue
https://github.com/sphinx-doc/sphinx/issues/2549 which causes a
spurious warning that breaks the build with -W. There is a
workaround posted in
https://stackoverflow.com/questions/31784830/sphinx-ivar-tag-goes-looking-for-cross-references
to move :ivar: docstrings to inline comments on the member variable
itself. This is not ideal because it causes the docs to render
differently from :ivar:, but until the sphinx bug is fixed it will
allow us to keep documenting the problematic variables.
There was also a problem with one of the doctests because the
output had changed. That is now fixed.
I also noticed a typo in one of the parameter descriptions so that
is fixed too.
Change-Id: Ib44621f6c3ba2c5476ec430218a0449f9f45d18f
Duplicate the dependencies that are listed in extras and place them in
a test-requirements.txt file so we can install them using the
constraints list.
In order to add the new requirements file, we have to make the constraints
and requirements match each other and the global settings.
Change-Id: I4dae402fe37015f0f512d5e9a6426d5f39526347
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
The infra jobs expect documentation requirements in either
doc/requirements.txt or test-requirements.txt. Since neither is
available, they currently fail.
Move doc requirements to doc/requirements.txt and use that to
fix releasenotes build. Note that doc/requirements.txt needs some
requirements for code analysis of taskflow.
For the docs environment - which is a developer convenience - use the
same command that is used by OpenStack CI.
Change-Id: Iec65211e5059a4afdd88a45e2c1ac535b8ec97fe
html_last_updated_fmt option is interpreted as a
byte string in python3, causing Sphinx build to break.
This patch makes it utf-8 string.
Changing Popen to .check_output because of 3 reasons:
1. check_output() will raise CalledProcessError if
the called process returns a non-zero return code.
2. For consistency with keystone [1] and cinder [2]
3. It makes the code look much better.
[1] https://review.openstack.org/#/c/457142/
[2] https://review.openstack.org/#/c/433081
Change-Id: I6e9c2009f224abc282acbf957d7e999c7f87e9eb
This moves out the engine next to run (or revert)
selection process to a single class that just does
this and moves out the common functions the analyzer
class provided to the runtime object (which all
components can access).
This makes it easier to adjust the selection algorithm
in different ways.
Change-Id: I091c69297a7bff60729791d3ca6c3fae14d6eea5
For a local process based executor usage currently to ensure
that task emitted notifications are proxied we use the multi
processing library and use its queue concept. This sadly creates
a proxy process that gets associated, and this proxy process
handles the queue and messages sent to and from it. Instead of
doing this we can instead just create a temporary local socket
using a random socket and have tasks (which are running in
different processes) use that to communicate back any emitted
notifications instead (and we can use the asyncore module to handle
the emitted notifications since it handles the lower level socket
reading, polling and dispatching).
To ensure that the socket created is somewhat secure we use a
similar process as the multi-processing library uses where we
sign all messages with a hmac that uses a one time key that only
the main process and the child process know about (and reject
any messages that do not validate using this key).
Change-Id: Iff9180054bf14495e5667af00ae2fafbdbc23791
Using a timestamp as value for "html_last_updated_fmt" isn't good
for the reproducible builds initiative from different downstream
distributions because the build result always differs due to the
timestamp in the documentation.
Also when not setting "html_last_updated_fmt", the value can be set
via command line when running "sphinx-build -D".
Change-Id: Ic8912e95e8e298fede1d94c20d3167d1f206df1e
When building packages if git is absent, then we should
fall back to a safe default.
Closes-Bug: #1552251
Change-Id: Ife309625a380cf7d5cc42581ed6d8f9f72b86cd9
Remove some of the usage of @property as none of
these objects are publicly exposed (or have docstrings
on them) to save some space/lines of code that aren't
really adding any benefit.
Use less **kwargs when we know exactly what the keyword
arguments will or will not be. Being explicit makes it
easier to understand these functions (vs not knowing what
the arguments can or can't be).
Removes base worker finder because right now we only
have one implementation (at some point we will have
two) but we can just wait to add a base class until
then.
Change-Id: I7107ff6b77a355b4c5d301948355fb6386605388
This fixes the sporadic of tasks that would happen
under certain circumstances. What happened was that
a new worker notification would be sent to a callback
while at the same time a task submission would come in
and there would be a small race period where the task
would insert itself into the requests cache while the
callback was processing.
So to work around this the whole concept of a requests
cache was revamped and now the WBE executor just maintains
its own local dictionary of ongoing requests and accesses
it safely.
During the on_wait function that is periodically called
by kombu the previous expiry of work happens but now any
requests that are pending are matched to any new workers
that may have appeared.
This avoids the race (and ensures that even if a new
worker is found but a submission is in progress that the
duration until that submission happens will only be until
the next on_wait call happens).
Related-Bug: #1431097
Change-Id: I98b0caeedc77ab2f7214847763ae1eb0433d4a78