Commit Graph

11 Commits (740ab628f4d060f7caa33d9c3dc1e189255468c1)

Author SHA1 Message Date
Brian Haley 7594bb0627 Remove the dependency on the "mock" package
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.

This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.

Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.

Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.

Unit tests only, removing from tests/base.py affects
functional tests which need additional work.

Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
3 years ago
ushen 3781825edf fix test nit
although 2G of ram is rare, but test should not
break if vm got that small ram.

Change-Id: Ic3efbe0767abca9c7ac25b58a7fb2af97e210d45
4 years ago
Adam Spiers 61d149c012 Fix default RPC worker count
The help for the rpc_workers config option is:

    Number of RPC worker processes for service.  If not specified, the
    default is equal to half the number of API workers.

However, this does not accurately describe the current behaviour,
which is to default to half the _default_ number of API workers.  This
can make a big difference; for example on a 256-CPU machine with 256GB
of RAM which has api_workers configured to 8 but rpc_workers not
configured to anything, this will result in 64 RPC workers, which is 8
for every API worker!

Therefore tweak the default to rely on the actual value of
api_workers, which may be different than the default value.

Change-Id: I26115932ef4775f157297be1637ee26a4fca4666
Related-Bug: #1838688
Closes-Bug: #1838689
4 years ago
Doug Wiegley 61b231a999 Change process name of neutron-server to match worker role
We spawn a lot of neutron-servers, on all but the smallest systems.
It's often hard to tell which are busy/overloaded or spinning.
Add an option to set the process names to their role.

This has a small chance of breaking existing scripting, depending
how they're parsing ps output.

Sample output:
$ ps xw | grep neutron-server
 1126 pts/2    S+     0:00 grep --color=auto neutron-server
25355 ?        Ss     0:26 /usr/bin/python /usr/local/bin/neutron-server \
  --config-file /etc/neutron/neutron.conf \
  --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
25368 ?        S      0:00 neutron-server: api worker
25369 ?        S      0:00 neutron-server: api worker
25370 ?        S      0:00 neutron-server: api worker
25371 ?        S      0:00 neutron-server: api worker
25372 ?        S      0:02 neutron-server: rpc worker
25373 ?        S      0:02 neutron-server: rpc worker
25374 ?        S      0:02 neutron-server: services worker

The "normal" looking ps output is the main parent.

Partial-Bug: #1816485
Depends-On: https://review.openstack.org/637119
Change-Id: I0e664a5f8e792d85b8f5483fb8c6f1cd59a677cd
4 years ago
Doug Wiegley 7e09b25b96
Modify api and rpc default number of workers
- Limit number of api workers to roughly using half of system
  RAM. Spawning a bunch, just to have the OOM killer nuke them
  regularly is not useful.
- Bump the rpc_workers default to half of the api_workers.
  A default of 1 falls behind on any reasonably sized node.

Change-Id: I8b84a359f83133014b3d4414aafc10e6b7c6a876
Closes-bug: #1815629
4 years ago
Boden R 10644ad65b use callback payloads for _SPAWN events
This patch switches callbacks over to the payload object style events
[1] for BEFORE_SPAWN and AFTER_SPAWN based notifications. As no event
data is passed for SPAWN notifications, an actual payload object is
not needed and thus this change uses publish() rather than notify()
and updates and callback receivers to accept the payload parameter.

NeutronLibImpact

[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads

Change-Id: Ifad4781d82aa07f213f4e075b0c9455aa95e94bb
6 years ago
Isaku Yamahata 5f02ed398a use neutron-lib.callback.events.AFTER_SPAWN
Change-Id: Ia3d9dbe751964ded7cf5b99bfcd7e28c30a5d4a6
Depends-On: I50513229b207c6a0a0b0b0a5e00873fb634f6b67
6 years ago
Isaku Yamahata 22b3e36010 service: add callback AFTER_SPAWN
Add callback (PRCESS, AFTER_SPAWN) to do housekeeping task.
ML2 driver(in fact, networking-odl) wants to run house keeping task
which needs to run only in api server.

The following is very netwokring-odl specific as FYI:
The task is to run a timer which is extended when rest requests are
handled. The timer is fired to kick journal thread only when rest
request is idle in order to check if there is journal entries that was
created by other neutron servers that crash before processing them.
Such timer isn't needed for each workers. Only single timer among api
worker and rpc workers is enough.

cf https://review.openstack.org/#/c/461620/
The current approach is to run timer unconditionally for all processes
and stop it if it's worker process. With this patch, the timer can be
simply run for main process(api worker).

Change-Id: I9c07bc528c3a2fade0c835797889fc169f9bd1a6
6 years ago
Jakub Libosvar ffeae31505 Fix that api_workers=0 doesn't spawn any api workers
Change-Id: Iadb9b35bc189244dee293efe042fee77a9a2d7da
Closes-Bug: 1607691
7 years ago
Elena Ezhova 0488ac7074 Refactor TestRpcWorker and TestWorkerService
TestRpcWorker and TestWorkerService have a duplicate test_reset.
This patch introduces a base class from which tests for service
workers can inherit.

Change-Id: Ic4690c3b066b03c2fbb463f1329208ad1307d83d
8 years ago
Elena Ezhova 6d0d729731 Handle SIGHUP: neutron-server (multiprocess) and metadata agent
All launchers implemented in common.service require each service to
implement reset method because it is called in case a process
receives a SIGHUP.

This change adds the reset method to neutron.service.RpcWorker and
neutron.wsgi.WorkerService which are used to wrap rpc and api
workers correspondingly.

Now neutron-server running in multiprocess mode (api_workers > 0 and
rpc_workers > 0) and metadata agent don't die on receiving SIGHUP and support
reloading policy_path and logging options in config.

Note that reset is called only in case a service is running in daemon mode.

Other changes made in the scope of this patch that need to be mentioned:

* Don't empty self._servers list in RpcWorker's stop method

  When a service is restarted all services are gracefully shutdowned,
  resetted and started again (see openstack.common.service code).
  As graceful shutdown implies calling service.stop() and then
  service.wait() we don't want to clean self._servers list because
  it would be impossible to wait for them to stop processing
  requests and cleaning up their resources.
  Otherwise, this would lead to problems with rpc after starting
  the rpc server again.

* Create a duplicate socket each time WorkerService starts

  When api worker is stopped it kills the eventlet wsgi server
  which internally closes the wsgi server socket object. This server
  socket object becomes not usable which leads to "Bad file
  descriptor" errors on service restart.

Added functional and unit tests.

DocImpact
Partial-Bug: #1276694
Change-Id: I75b00946b7cae891c6eb192e853118e7d49e4a24
8 years ago