Commit Graph

2626 Commits (stable/ussuri)

Author SHA1 Message Date
Elod Illes 766715d310 [stable-only] Cap virtualenv for py37
py37 test started to fail with latest virtualenv (20.16.4) as it
bundles setuptools 65.3.0, which causes installation problems under
py37. (Note: with py38 the problem does not occur).

This patch sets virtualenv<20.16.4 to unblock the gate as
virtualenv 20.16.3 bundles setuptools 63.4.1, with which py37 gate
works well.

Change-Id: I29beb88dd241af72397bf0dd1fc46c98a229425c
9 months ago
Zuul 28f0301f82 Merge "Move legacy grenade jobs to Zuul v3" into stable/ussuri 2 years ago
Kenneth Giusti eb3f9bd6f0 Move legacy grenade jobs to Zuul v3
Also deletes obsolete playbook files for telemetry tests that have
been removed.

Change-Id: Ibbb811e988cc048e6c39a0c2336c716a1a0ddb30
(cherry picked from commit a5ae9f4aa0)
2 years ago
Hervé Beraud 36b5e06c92 Correctly handle missing RabbitMQ queues
Currently, setting the '[oslo_messaging] direct_mandatory_flag' config
option to 'True' (the default) will result in a 'MessageUndeliverable'
exception being raised when sending a reply if a RabbitMQ queue is
missing [1]. It was the responsibility of the application to handle
this exception, however, many applications are not doing so. This has
resulted in a number of bug reports.

Start handling this error condition, using a retry loop to attempt to
resend the message and work around any temporary glitches. Since
attempting to send a reply will will no longer raise an exception,
there is little benefit in retaining the '[oslo_messaging]
direct_mandatory_flag' config option: users setting this to False will
simply not benefit from the retry logic and improved logging added
here. This option is already deprecated though and will be fully
removed in a future release.


Change-Id: Id5cddbefbe24ef100f1cc522f44430df77d217cb
Closes-Bug: #1905965
(cherry picked from commit 4937949dff)
(cherry picked from commit 391ce7fc69)
2 years ago
Hervé Beraud 5f2a4f21d0 Deprecate the mandatory flag
It will not be possible to deactivate this functionality anymore.

Change-Id: I1cbafff03349f7da9224de46285707fbf2a81a68
(cherry picked from commit 2b89d97888)
(cherry picked from commit 06ad070cc6)
2 years ago
ushen de2f0403fb Fix some typos
Change-Id: Ic0a311d290682a923ec4f8cbe36f875d560cd41a
(cherry picked from commit 2c2779a974)
2 years ago
Ben Nemec 3bf9ba5504 Fix type of direct_mandatory_flag opt
An IntOpt with a default of True is invalid. I'm a little surprised
this doesn't fail a defaults check somewhere, but it needs to be
fixed regardless.

Looking at where it is used, it appears the boolean type is correct.
This just changes the opt type to BoolOpt to match.

Change-Id: I01a38754a31c891f2b3b9c7f8135690693df5d13
Closes-Bug: 1909036
(cherry picked from commit b8f8b17030)
(cherry picked from commit a0329a3f86)
2 years ago
Hervé Beraud 7e151b15b9 Dropping lower constraints testing
We facing errors related to the new pip resolver, this
topic was discussed on the ML and QA team proposed to
to test lower-constraints [1].

I propose to drop this test because the complexity and recurring pain needed
to maintain that now exceeds the benefits provided by this mechanismes.

Also removing lower constraints and related tox environment as they are
no longer needed.



NOTE(hberaud): Conflicts are due to different base jobs in zuul and
different constraints in lower-constraints

Change-Id: Icf48ef72fbfff88dda2714b386dbbfe355dc1acb
(cherry picked from commit 686a8b6182)
(cherry picked from commit 1682ef69ae)
2 years ago
Sean Mooney 7c9b070f5b add min of 1 to rpc_conn_pool_size
This change add a min value of 1 to
such that there is always at least 1 connection avaiable.

This change add a runtime check to ensure that
[oslo_messaging_rabbit]/rpc_conn_pool_size is greater than
or equal too [oslo_messaging_rabbit]/conn_pool_min_size

Change-Id: I2ad4b9f1d012c9f0586a932ac27d96da1bcc4e4c
Closes-Bug: #1899533
(cherry picked from commit 00d15eaeab)
(cherry picked from commit 8f5847dcbe)
3 years ago
shenjiatong 0a432c7fb1 Cancel consumer if queue down
Previously, we have switched to use default exchanges
to avoid excessive amounts of exchange not found messages.
But it does not actually solve the problem because
reply_* queue is already gone and agent will not receive callbacks.

after some debugging, I found under some circumstances
seems rabbitmq consumer does not receive basic cancel
signal when queue is already gone. This might due to
rabbitmq try to restart consumer when queue is down
(for example when split brain). In such cases,
it might be better to fail early.

by reading the code, seems like x-cancel-on-ha-failover
is not dedicated to mirror queues only,,

By failing early, in my own test setup,
I could solve a certain case of exchange not found problem.

Change-Id: I2ae53340783e4044dab58035bc0992dc08145b53
Related-bug: #1789177
(cherry picked from commit 196fa877a9)
3 years ago
shenjiatong 360deb89a7 Catch ConnectionForced Exception
Occasionally I saw large number of connectionforced exceptions.
such exceptions cannot be healed automatically and requires a reboot.
catch it and ensure connection established.

Change-Id: I5c468d10230f05438df780e045f1e92662edf070
Closes-Bug: #1883038
(cherry picked from commit 26e8bb4cc9)
3 years ago
Stephen Finucane 6727be7890 tests: Resolves issues with kombu > 4.6.8
The 'kombu.connection.Connection.ensure_connection' method has changed
from calling 'retry_over_time' on 'self.connect' to calling it on
'self._connection_factory' [1], meaning our mocks are outdated. Address
this change.


Change-Id: Ibbcf21a57ab1e3f90c21901296e5c088b645127c
Signed-off-by: Stephen Finucane <>
Closes-Bug: #1885923
(cherry picked from commit afb035d971)
3 years ago
Andreas Jaeger dfcc4b0010 Simplify tools/ is called after bindep is run in CI, and tox is not setup.

Simplify the script so that it only does what's needed - install
qdrouterd for Ubuntu from qpid/testing.

Install the other packages using bindep, add test to amqp1 profile.

Add amqp1 to bindep_profiles where needed for scenario tests, previously added it everywhere.

NOTE(elod.illes): skip annotation removed from, as
kombu is constrained in ussuri with the version that does not need any
test to skip.

Change-Id: Ic3a9e2c873619670edfbf71022d593f3cb5f70f2
Related-Bug: #1885923
(cherry picked from commit 7e406c312a)
3 years ago
shenjiatong 39f770f55b Print warning message when connection running out
I believe when connection pool running out of available
resources, there will be all kinds of weird consequences
following. So let's print warning logs when we are going
to waiting indefinitely for the lock

Change-Id: I5ca930cada523bc77c644703a0f02b9160816231
Related-bug: #1871813
(cherry picked from commit 96300a3221)
3 years ago
OpenStack Release Bot f3e4d42640 Update TOX/UPPER_CONSTRAINTS_FILE for stable/ussuri
Update the URL to the upper-constraints file to point to the redirect
rule on so that anyone working on this branch
will switch to the correct upper-constraints list automatically when
the requirements repository branches.

Until the requirements repository has as stable/ussuri branch, tests will
continue to use the upper-constraints list on master.

Change-Id: Icde5163e169fba2831a024f6f3b8c39fc35cf44d
3 years ago
OpenStack Release Bot 252ea15f92 Update .gitreview for stable/ussuri
Change-Id: I08173d81f59c3470a8322dbe6ea110544ac649b4
3 years ago
Andreas Jaeger 7f638bb493 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Update local hacking check for new flake8 version.

W504 line break after binary operator

E741 ambiguous variable name
E117 over-indented
E305 expected 2 blank lines after class or function definition, found 1
F841 local variable 'e' is assigned to but never used
W605 invalid escape sequence '\.'

Change-Id: I99d574ca6569f1f177d2c5ce1011f269f4343619
3 years ago
Zuul d87618b24f Merge "Setup backend scenarios for functional tests" 3 years ago
Hervé Beraud fed48aea44 Remove the deprecated blocking executor
The blocking executor has been deprecated in Pike and marked for removal
in Rocky, but some user like Mistral asked us to wait before. We decided
to remove this executor for Train or next cycle, now we are in the
Ussuri and after some researchs on usage I think we can go ahead.

This patch drop the deprecation warnings, related unit tests and
set the server with the threading executor is the default executor.

Change-Id: If07bab61ee2b148658b88be98b12f8539f274efe
Closes-Bug: #1715141
3 years ago
Hervé Beraud 3359c520d3 remove outdated header
Change-Id: I1ac92354fea2a70e5fe72a0e3b48d4c435b5286d
3 years ago
Hervé Beraud b55da38368 reword releasenote for py27 support dropping
Change-Id: Ic65842371eac32cec3a7b66cb6aa0f3ef52b8637
3 years ago
Andy Smith ab78c8e3dd Setup backend scenarios for functional tests
This patchset introduces scenarios for the functional tests to unify the
setup and configuration of alternate messaging backends for RPC and
Notifications. The scenarios are defined to reduce redundancy in
the testing of backends and to demonstrate functional correctness
across driver combinations.

Current driver support:
rabbit - RPC, Notify
amqp   - RPC
kafka  - Notify

                  RPC       Notify
               ---------  ----------
scenario01       rabbit     rabbit
scenario02       rabbit     kafka
scenario03        amqp      rabbit
scenario04        amqp      kafka

It is anticipated that additional scenarios will be defined as new
drivers are introduced and/or new messaging intermediarites are supported.

Note: The removal of python2 jobs are also included patch

Change-Id: I0f86416623a0b718516147f0660b4df2b74cf867
3 years ago
Hervé Beraud 32a1b6e948 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in Ussuri cycle.

Complete discussion & schedule can be found in

Ussuri Communtiy-wide goal:

Change-Id: I15cc79159b603e232e442e202c78d6c56dc73252
3 years ago
Zuul 3ad0a72cdd Merge "Add support for kafka SSL autentication" 3 years ago
Oleg Bondarev d7eb89eac8 Don't log NoSuchMethod for special non-existing methods
Such non-existing methods are used as health probes.
Please see bug for details.

Change-Id: I8f6b4a669ec11721f76cf03abcd7d802c3c11eb5
Closes-Bug: #1857319
3 years ago
Michal Arbet 5a43d4548a Add support for kafka SSL autentication
Change-Id: Idef066a2e3b4923789a6b081d5442e931aba4507
3 years ago
Zuul 04b2b5d451 Merge "Adding debug logs on AMQPListener poll" 3 years ago
Hervé Beraud e4a8900aa7 Adding debug logs on AMQPListener poll
Adding some debug logs to track execution on the AMQPListener poll
function to observe if workers incoming messages are dispatched to
workers and if listener is stuck.

Change-Id: Ia9b9ef62b0b59c78283c6a8cc5b7ce42282e73b5
3 years ago
Hervé Beraud cc6cf12c3d tox: Trivial cleanup
Move 'basepython' to the top-level 'testenv'.

Change-Id: I13e72ec6cbd006ca6eccfe21f2a0584e86a52669
3 years ago
Zuul 41a4423043 Merge "Ignore releasenote cache within git untracked files" 4 years ago
Zuul 4fc0e6241b Merge "Remove telemetry checks" 4 years ago
Zuul 4e49a286e7 Merge "Do not use threading.Event" 4 years ago
Hervé Beraud 2a4cea9e85 Ignore releasenote cache within git untracked files
Change-Id: If616ac023f578f073d9ae8596bc8ce870096ef0b
4 years ago
Zuul 524f72261c Merge "Removed unused variable pools" 4 years ago
John Eckersberg d873c0d8f5 Do not use threading.Event
Waiting on a threading.Event with eventlet can cause busy looping via
epoll_wait, see related bug for more details.

Change-Id: I007613058a2d21d1712c02fa6d1602b63705c1ab
Related-bug: #1518430
4 years ago
Chenjun Shen 2b0613b208 Removed unused variable pools
Change-Id: I8f3a22d2af3f34b368bae3465bf8433fe2d3e22b
4 years ago
Kenneth Giusti 859e0d4eaa Revert "Add RPC incoming and reply log"
This broke non-rabbitmq message bus drivers. See:

This reverts commit b104f254ab.

Change-Id: I17a448a768b544482b375b0076889db989e03e8c
4 years ago
Andy Smith 1057cfc60f Remove telemetry checks
The telemetry tests are not actively maintained and have been failing
checks for quite some time. Remove the non-voting tests for now and
plan to introduce new tests in the future for better coverage of

Change-Id: I6c30c28ee4c8be4677ccecf8822af15b060cf804
4 years ago
Ghanshyam Mann 37fabdd895 Migrate grenade jobs to py3
As part of community goal of dropping py27 support[1], we are
moving the devstack to py3 by default[2]. That will make grenade job
to perform upgrade from py2 to py3 which will not work (we have seen
the failure in neutron-grenade job).

To avoid existing grenade py2 job break, this commit moves grenade
jobs to py3 which is what we planned as part dropping the py2 support.


Change-Id: I128827b77ed791ea0084008cdecbd7ac37fa9079
4 years ago
Eyal c8d6fed762 Make sure minimum amqp is 2.5.2
amqp fixed a bug in 2.5.2 that is needed
also update kombu to support amqp 2.5.2


Change-Id: I4b72d8feb85c2b9b4657510c356cd21e22fe40c2
Closes-bug: #1852058
4 years ago
Zuul da3c3e7d1d Merge "Switch to Ussuri jobs" 4 years ago
Zuul b2658bafb6 Merge "Add RPC incoming and reply log" 4 years ago
Zuul f1ca5ee6b7 Merge "tox: Keeping going with docs" 4 years ago
Zuul 19cba65785 Merge "Modify some comments to make them clickable" 4 years ago
Chenjun Shen 6bca848f5b Remove unused variable WAKE_UP
Change-Id: I65f57973b66f353a1e6fef3d0bf7793f821c1c5c
Closes-Bug: 1850259
4 years ago
pengyuesheng 4529981d8c Switch to Ussuri jobs
Change-Id: I4297685cbe5c5ec71b68467a54746c2f4814f104
4 years ago
caoyuan 1e7a0c6fbb tox: Keeping going with docs
Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be


Change-Id: If9885a1f064226909181d8b69241eb814deb2105
4 years ago
LIU Yulong b104f254ab Add RPC incoming and reply log
Typically a simple log will not narrow down the
performance, but give us more information about
the service status.

Change-Id: I51c8f2743dd39cccd3d1d021d3c50dc09f70cd97
Closes-Bug: #1847747
4 years ago
Hervé Beraud 40e5be488e Modify some comments to make them clickable
Change-Id: Ib085fa551950f1c1461a11bb80f4909fb8425484
4 years ago
Zuul c01b03e87c Merge "Update master for stable/train" 4 years ago