Commit Graph

364 Commits

Author SHA1 Message Date
Zuul
6d5a435e42 Merge "Add Etcd backend for jobboard" 2024-07-08 04:20:49 +00:00
Gregory Thiemonge
f0fd57a863 Add Etcd backend for jobboard
Add a new backend for jobboard based on Etcd3, it uses the etcd3gw
bindings.

Change-Id: I12eaa04dfb11941802200736e56a4fb5761fb368
2024-06-11 16:23:40 +02:00
Zuul
1e12df2991 Merge "Remove old excludes" 2024-04-30 14:37:57 +00:00
Takashi Kajinami
116d7f7760 Remove old excludes
These are detected as errors since the clean up was done[1] in
the requirements repository.

[1] 314734e938f107cbd5ebcc7af4d9167c11347406

Change-Id: I6970540de396e4aa77590aadfdfa72bf3f1862f3
2024-04-30 18:16:31 +09:00
Takashi Kajinami
ff45bd3d2d Fix update-states target
... and regenerate svg files.

Change-Id: I903774597e07a4184213df284207eeac5f329d5e
2024-04-15 14:43:43 +09:00
Eric Harney
a2078f4e09 Fix doc building with Sphinx 6.0
Fix Sphinx 6.0 doc build error.

Change-Id: I9a1571cf416d32d040d8ff05d1c8b1769d1d0558
2023-03-21 11:28:33 -04:00
niuke
299c2afe14 remove unicode prefix from code
Change-Id: I59e8ae94eb83ae3a1ddcc99091e53686d2d55186
2022-08-24 15:17:18 +08:00
Takashi Kajinami
44f17d005f Remove six
This library no longer supports Python 2, thus usage of six can be
removed. This also removes workaround about pickle library used in
Python 2 only.

Change-Id: I19d298cf0f402d65f0b142dea0bf35cf992332a9
2022-05-18 16:12:37 +09:00
Michael Johnson
9e3ba872d0 Updating for OFTC IRC network
Change-Id: I47faea9b3c0324d3860a5209d521bc76de03bc47
2021-06-07 20:43:11 +00:00
Hervé Beraud
7ab39946e7 Adding pre-commit
Introduced changes:
- pre-commit config and rules.
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.

Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.

pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker
  (check-byte-order-marker);
- Checks that non-binary executables have a proper
  shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings
  (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
  calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)

For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks

Change-Id: I3640bb690c552a428ae0e8250adffbc3a51a310e
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
2020-09-08 14:33:38 +02:00
Andreas Jaeger
13c1165779 Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems

Update Sphinx version as well.

Disable openstackdocs_auto_name to use 'project' variable as name.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: Ieb7ce05f6611ad743336a35065402888176c7de3
2020-06-02 13:54:29 +02:00
Hervé Beraud
7497435e54 Add release notes links to doc index
Change-Id: Ie42f30fa22c0d9864e355c4037520bcca3747a76
2020-04-16 17:55:13 +02:00
Andreas Jaeger
fc401959ec [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Remove cruft from setup.cfg.
Update hacking so that it works with python3 and silence new warnings.

Change-Id: Id224fc09ac36d2a84e82347530bb5e5d30665803
2020-02-03 21:55:57 +01:00
liuwei
5965747909 update git.openstack.org to opendev
Change-Id: Ie9f1ca14eed287f3c25553d029d4a4b2ca5274d0
2019-05-22 09:51:19 +01:00
Stephen Finucane
dd680367c5 Update Sphinx requirement
Sphinx 2.0 no longer works on python 2.7, so we need to start capping it
there. We remove Sphinx and doc8 from the 'test-requirements.txt' file
and '[extra] test' section in 'setup.cfg' since none of our tests
actually use this. Finally, we remove a number of dependencies from
'doc/requirements.txt' since these were already specified in
'test-requirements.txt' and we install these as part of the tox target.

Change-Id: I3981f47f4278f726e3c078723f90d4b9ba1699a6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-05-22 08:35:34 +00:00
zhang.lei
f9349f4301 Update doc/conf.py to avoid warnings with sphinx 1.8
The autodoc_default_flags option is now deprecated.[1] This warning can
cause documentation build failed. use autodoc_default_options instead.

[1]https://github.com/matplotlib/matplotlib/pull/11928

Change-Id: I6ba69ca7361fd0e42b322f1b92fe22b952a13321
2018-09-19 01:35:57 +00:00
jiansong
eae5477d50 Fix a symbol error
There is a wrong use ' here, which causes the document
to make errors when copying the code for
experimentation.

Change-Id: I3ae19f591ffd668bcd1563bd1e317b21b799dba6
2018-08-20 03:35:00 -07:00
Vu Cong Tuan
33685c83fc Update various links in docs
Avoid redirects, s/http/https/.

Change-Id: Id03bd49f250b48109967a0e04b92b75925f08932
2018-07-17 13:56:19 -05:00
Michal Arbet
d985c5a256 Fix code to support networkx > 1.0
With the release of NetworkX 2.0 the reporting API was
moved to view/iterator model. Many methods were moved from
reporting lists or dicts to iterating over the information.
Methods that used to return containers now return views and
methods that returned iterators have been removed in networkx.
Because of this change in NetworkX 2.0 , taskflow code
have to be changed also to support networkx > 2.0

Change-Id: I23c226f37bd85c1e38039fbcb302a2d0de49f333
Closes-Bug: #1778115
2018-07-11 13:11:51 +02:00
zhangzs
d197f43dcb replace http with https
Change-Id: Ic01648cae18bf674cf5ee2753043dac7c5fc9ed0
2018-06-13 10:58:51 +08:00
Ben Nemec
e93f40cd1c Fix doc build
Most notably, taskflow is hitting the sphinx issue
https://github.com/sphinx-doc/sphinx/issues/2549 which causes a
spurious warning that breaks the build with -W.  There is a
workaround posted in
https://stackoverflow.com/questions/31784830/sphinx-ivar-tag-goes-looking-for-cross-references
to move :ivar: docstrings to inline comments on the member variable
itself.  This is not ideal because it causes the docs to render
differently from :ivar:, but until the sphinx bug is fixed it will
allow us to keep documenting the problematic variables.

There was also a problem with one of the doctests because the
output had changed.  That is now fixed.

I also noticed a typo in one of the parameter descriptions so that
is fixed too.

Change-Id: Ib44621f6c3ba2c5476ec430218a0449f9f45d18f
2018-05-09 19:50:20 +00:00
Zuul
8fb318edb6 Merge "stop using tox_install.sh" 2018-04-25 05:25:30 +00:00
melissaml
e5aec26fb8 Trivial: Update pypi url to new url
Pypi url changed from [1] to [2]

[1] https://pypi.python.org/pypi/<package>
[2] https://pypi.org/project/<package>

Change-Id: I0c8a226317fa62658555cb9a18028f1516aeccfe
2018-04-21 03:43:08 +08:00
Doug Hellmann
0124bf4919 stop using tox_install.sh
Duplicate the dependencies that are listed in extras and place them in
a test-requirements.txt file so we can install them using the
constraints list.

In order to add the new requirements file, we have to make the constraints
and requirements match each other and the global settings.

Change-Id: I4dae402fe37015f0f512d5e9a6426d5f39526347
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-18 11:56:25 -04:00
OpenStack Proposal Bot
d65c8d856c Updated from global requirements
Change-Id: I56e4891d6b218a676dd3d257ea9ca837c9750bfa
2018-03-15 09:36:36 +00:00
OpenStack Proposal Bot
3a4f9a332f Updated from global requirements
Change-Id: Iad71ca0b48f8df310bdc10e56875784172acd94b
2018-01-24 01:38:15 +00:00
OpenStack Proposal Bot
09916cec44 Updated from global requirements
Change-Id: I048e5b96e3e1825b7ff743fafc755425fe2ed6b7
2018-01-18 03:36:57 +00:00
OpenStack Proposal Bot
6e7eb0fa01 Updated from global requirements
Change-Id: Icb21b3ea8f9ae0eb4e51c951187756ac730251fd
2017-12-23 10:17:03 +00:00
Andreas Jaeger
760f4414ea Use doc/requirements.txt
The infra jobs expect documentation requirements in either
doc/requirements.txt or test-requirements.txt. Since neither is
available, they currently fail.

Move doc requirements to doc/requirements.txt and use that to
fix releasenotes build. Note that doc/requirements.txt needs some
requirements for code analysis of taskflow.

For the docs environment - which is a developer convenience - use the
same command that is used by OpenStack CI.

Change-Id: Iec65211e5059a4afdd88a45e2c1ac535b8ec97fe
2017-12-20 10:01:39 +01:00
ChangBo Guo(gcb)
40645e19a7 Update URLs in documents according to document migration
Change-Id: I9ca92fdcec388e02462332e04fe7c1bf8b5f64b8
2017-07-13 12:05:18 +08:00
Doug Hellmann
b0ca782396 switch from oslosphinx to openstackdocstheme
Change-Id: I7522d14d4e340f0882e41bed3528f90bbf74c2af
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-08 19:11:12 -04:00
Doug Hellmann
1229eb2e8e rearrange existing documentation into the new standard layout
Change-Id: Ic53baaa62771fda6de3b3dfe861909365103d1a9
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-08 14:18:58 -04:00
Vu Cong Tuan
d565cf1a0b Fix html_last_updated_fmt for Python3
html_last_updated_fmt option is interpreted as a
byte string in python3, causing Sphinx build to break.
This patch makes it utf-8 string.

Changing Popen to .check_output because of 3 reasons:
1. check_output() will raise CalledProcessError if
the called process returns a non-zero return code.
2. For consistency with keystone [1] and cinder [2]
3. It makes the code look much better.

[1] https://review.openstack.org/#/c/457142/
[2] https://review.openstack.org/#/c/433081

Change-Id: I6e9c2009f224abc282acbf957d7e999c7f87e9eb
2017-06-06 09:54:12 +07:00
gecong1973
f6b68865e1 Use https instead of http for git.openstack.org
TrivialFix

Change-Id: I7c9f2dc507641a03717333a5e0c0479f96379326
2017-02-07 09:19:04 +08:00
Joshua Harlow
774d598134 Rename engine analyzer to be named selector
This moves out the engine next to run (or revert)
selection process to a single class that just does
this and moves out the common functions the analyzer
class provided to the runtime object (which all
components can access).

This makes it easier to adjust the selection algorithm
in different ways.

Change-Id: I091c69297a7bff60729791d3ca6c3fae14d6eea5
2017-01-03 06:37:57 +00:00
qinchunhua
5a65db9651 Replaces uuid.uuid4 with uuidutils.generate_uuid()
Change-Id: I25e6e56a270d1d99ee2ddac52abfc9f5678f0a70
Closes-Bug: #1082248
2016-11-22 08:39:02 +00:00
Jenkins
fb74ed9c17 Merge "Fix a typo in documentation" 2016-10-11 18:45:03 +00:00
Luong Anh Tuan
a102530e22 Fix a typo in documentation
remove redundant 'this'

Change-Id: I93495166526ce695a41ee58dd00cc48a1858157f
2016-10-04 10:08:54 +07:00
Luong Anh Tuan
86490ded9e Fix typo: remove redundant 'that'
Change-Id: I2206b0b1747dfb3301eef5013749cfd20b5b32df
2016-10-03 13:30:29 +07:00
Joshua Harlow
c5e9cf28df Instead of a multiprocessing queue use sockets via asyncore
For a local process based executor usage currently to ensure
that task emitted notifications are proxied we use the multi
processing library and use its queue concept. This sadly creates
a proxy process that gets associated, and this proxy process
handles the queue and messages sent to and from it. Instead of
doing this we can instead just create a temporary local socket
using a random socket and have tasks (which are running in
different processes) use that to communicate back any emitted
notifications instead (and we can use the asyncore module to handle
the emitted notifications since it handles the lower level socket
reading, polling and dispatching).

To ensure that the socket created is somewhat secure we use a
similar process as the multi-processing library uses where we
sign all messages with a hmac that uses a one time key that only
the main process and the child process know about (and reject
any messages that do not validate using this key).

Change-Id: Iff9180054bf14495e5667af00ae2fafbdbc23791
2016-05-24 16:16:56 -07:00
Greg Hill
5944ccc020 Fix documentation related to missing BaseTask class
The docs no longer matched reality.

Change-Id: I9ee98e16fc03489569bbed8355416a73f7bc02ff
Closes-Bug: 1579890
2016-05-11 19:27:58 +00:00
Joshua Harlow
afbfe77a7f Remove deprecated things for 2.0 release
Change-Id: Id9adbc50bd51adc77ce88f698ad0ea2ee63fc5e2
2016-05-10 15:26:57 -07:00
Jenkins
ea46736b49 Merge "Spice up WBE banner and add simple worker __main__ entrypoint" 2016-04-28 20:39:04 +00:00
Thomas Bechtold
7bcf508263 Don't set html_last_updated_fmt without git
Using a timestamp as value for "html_last_updated_fmt" isn't good
for the reproducible builds initiative from different downstream
distributions because the build result always differs due to the
timestamp in the documentation.
Also when not setting "html_last_updated_fmt", the value can be set
via command line when running "sphinx-build -D".

Change-Id: Ic8912e95e8e298fede1d94c20d3167d1f206df1e
2016-04-13 09:21:02 +02:00
Davanum Srinivas
a2d4731f9a Fallback if git is absent
When building packages if git is absent, then we should
fall back to a safe default.

Closes-Bug: #1552251
Change-Id: Ife309625a380cf7d5cc42581ed6d8f9f72b86cd9
2016-03-02 16:29:08 -05:00
Joshua Harlow
1ab60b7e98 Some WBE protocol/executor cleanups
Remove some of the usage of @property as none of
these objects are publicly exposed (or have docstrings
on them) to save some space/lines of code that aren't
really adding any benefit.

Use less **kwargs when we know exactly what the keyword
arguments will or will not be. Being explicit makes it
easier to understand these functions (vs not knowing what
the arguments can or can't be).

Removes base worker finder because right now we only
have one implementation (at some point we will have
two) but we can just wait to add a base class until
then.

Change-Id: I7107ff6b77a355b4c5d301948355fb6386605388
2016-02-14 11:47:06 -08:00
Joshua Harlow
3e3efc562b Spice up WBE banner and add simple worker __main__ entrypoint
Change-Id: Ifdf275a623352aa3e42fbf0aa9a4394b64b54337
2016-02-08 16:33:14 -08:00
Joshua Harlow
cea71f2799 Fix for WBE sporadic timeout of tasks
This fixes the sporadic of tasks that would happen
under certain circumstances. What happened was that
a new worker notification would be sent to a callback
while at the same time a task submission would come in
and there would be a small race period where the task
would insert itself into the requests cache while the
callback was processing.

So to work around this the whole concept of a requests
cache was revamped and now the WBE executor just maintains
its own local dictionary of ongoing requests and accesses
it safely.

During the on_wait function that is periodically called
by kombu the previous expiry of work happens but now any
requests that are pending are matched to any new workers
that may have appeared.

This avoids the race (and ensures that even if a new
worker is found but a submission is in progress that the
duration until that submission happens will only be until
the next on_wait call happens).

Related-Bug: #1431097

Change-Id: I98b0caeedc77ab2f7214847763ae1eb0433d4a78
2016-02-05 14:40:25 -08:00
Jenkins
f9a2d78bd8 Merge "Retrieve the store from flowdetails as well, if it exists" 2016-01-26 03:12:02 +00:00
Jenkins
615743c71a Merge "Use automaton's converters/pydot" 2016-01-25 23:07:15 +00:00